* [RFC PATCH 0/3] RISC-V: KVM: Preserve resident guest Vector state
@ 2026-10-27 13:13 ` slack up
0 siblings, 0 replies; 15+ messages in thread
From: slack up @ 2026-10-27 13:13 UTC (permalink / raw)
To: Anup Patel, Atish Patra, Palmer Dabbelt, kvm-riscv, linux-riscv
Cc: Paul Walmsley, Albert Ou, Alexandre Ghiti, Paolo Bonzini,
Sean Christopherson, Shuah Khan, kvm, linux-kernel,
linux-kselftest, Xiaochun Yu
From: Xiaochun Yu <yxcnew@outlook.com>
kvm_arch_vcpu_put() saves the guest Vector state and restores the host
register file. A later vCPU load on the same hart saves the host again and
requests a full guest restore, even when no host user touched the Vector
data registers in between.
This RFC keeps the guest Vector register file resident across a vCPU
put/load pair when it is safe to do so. A matching load restores only the
control CSRs; all uncertain cases retain the existing full-restore path.
The series is intentionally conservative:
- every dirty vCPU put still saves the guest state, so guest memory remains
authoritative for migration and userspace access;
- a per-hart owner records an opaque vCPU identity, an allocation cookie and a
state generation, without retaining a dereferenceable vCPU pointer;
- task Vector save/restore/discard, kernel-mode Vector use, preemptible kernel
Vector task resume and CPU state changes invalidate a resident guest before
the registers can be reused;
- reset, dirty saves and KVM_SET_ONE_REG advance the generation;
- KVM_SET_ONE_REG advances the generation before copy_from_user(), because a
partial copy can modify the destination and still return EFAULT;
- the optimization is guarded by a read-only module parameter and defaults to
disabled. The generic invalidation sites are guarded by a static key.
Patch 1 establishes the generic invalidation boundary without changing KVM
behavior. Patch 2 adds the owner tracking, CSR-only restore and tracepoints.
Patch 3 adds focused lifetime, ONE_REG and partial-fault regression tests.
This series currently depends on the separately posted build fix:
Message-ID: <OSQPR01MB6326DDE9C2F390EA3F0F444EC2662@OSQPR01MB6326.apcprd01.prod.exchangelabs.com>
Link: https://lore.kernel.org/all/OSQPR01MB6326DDE9C2F390EA3F0F444EC2662@OSQPR01MB6326.apcprd01.prod.exchangelabs.com/
Ze Huang <ze.huang@oss.qualcomm.com> independently tested that prerequisite
with GCC 16.2.1 and Clang 23.1.3 and supplied a Tested-by tag in this reply:
Link: https://lore.kernel.org/all/DMCBQSJ0B22A.2TDXAXPJNPZL2@oss.qualcomm.com/
Validation on Torvalds master 21a7f44fd076:
- CONFIG_RISCV_ISA_V=y, CONFIG_KVM=m: full Image + modules build;
- CONFIG_RISCV_ISA_V=n, CONFIG_KVM=m: full Image + modules build;
- lifetime, ONE_REG and partial-fault tests pass with the fast path disabled
and enabled;
- nested KVM/RVV smoke passes 3/3 with the fast path enabled;
- the smoke trace recorded 785 owner hits, 784 CSR-only restores and eight
conservative full fallbacks, with no trace-buffer overwrite;
- no emergency, alert, critical or error messages were present in dmesg.
Development note: OpenAI Codex was used in an extended interactive session to
assist with source analysis, implementation and selftest drafting, experiment
orchestration, patch review and changelog preparation. The prompts focused on
modeling per-hart ownership as a cache-validity problem, enumerating Vector
register invalidation boundaries, reproducing partial KVM_SET_ONE_REG writes
and reviewing the resulting patches. The build and runtime results above were
collected from the stated test environment rather than inferred by the model.
The nested QEMU result demonstrates functional behavior and event counts only;
it is not presented as a performance result. Hardware performance measurement
remains future work.
Feedback is especially welcome on whether the generic invalidation callback is
an acceptable RISC-V Vector/KVM boundary, whether the default-off module
parameter is useful for the first version, and whether the behavior patch
should be split further after the interface direction is agreed.
Xiaochun Yu (3):
riscv: vector: Add a guest owner invalidation hook
RISC-V: KVM: Preserve guest Vector state across vCPU put/load
KVM: riscv: selftests: Add Vector owner regression tests
arch/riscv/include/asm/kvm_host.h | 4 +
arch/riscv/include/asm/kvm_vcpu_vector.h | 54 +++-
arch/riscv/include/asm/processor.h | 1 +
arch/riscv/include/asm/vector.h | 31 +++
arch/riscv/kernel/kernel_mode_vector.c | 38 +++
arch/riscv/kvm/trace.h | 98 +++++++
arch/riscv/kvm/vcpu.c | 54 +++-
arch/riscv/kvm/vcpu_vector.c | 252 +++++++++++++++++-
tools/testing/selftests/kvm/Makefile.kvm | 3 +
.../kvm/riscv/vector_owner_fault_test.c | 147 ++++++++++
.../kvm/riscv/vector_owner_lifetime_test.c | 127 +++++++++
.../kvm/riscv/vector_owner_onereg_test.c | 191 +++++++++++++
12 files changed, 976 insertions(+), 24 deletions(-)
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_onereg_test.c
base-commit: 21a7f44fd0765fcc5194130b28767786eba1d1e7
prerequisite-patch-id: 370f0468ecd4354950ab075162058e4b7c31768b
--
2.43.0
^ permalink raw reply [flat|nested] 15+ messages in thread
* [RFC PATCH 0/3] RISC-V: KVM: Preserve resident guest Vector state
@ 2026-10-27 13:13 ` slack up
0 siblings, 0 replies; 15+ messages in thread
From: slack up @ 2026-10-27 13:13 UTC (permalink / raw)
To: Anup Patel, Atish Patra, Palmer Dabbelt, kvm-riscv, linux-riscv
Cc: Paul Walmsley, Albert Ou, Alexandre Ghiti, Paolo Bonzini,
Sean Christopherson, Shuah Khan, kvm, linux-kernel,
linux-kselftest, Xiaochun Yu
From: Xiaochun Yu <yxcnew@outlook.com>
kvm_arch_vcpu_put() saves the guest Vector state and restores the host
register file. A later vCPU load on the same hart saves the host again and
requests a full guest restore, even when no host user touched the Vector
data registers in between.
This RFC keeps the guest Vector register file resident across a vCPU
put/load pair when it is safe to do so. A matching load restores only the
control CSRs; all uncertain cases retain the existing full-restore path.
The series is intentionally conservative:
- every dirty vCPU put still saves the guest state, so guest memory remains
authoritative for migration and userspace access;
- a per-hart owner records an opaque vCPU identity, an allocation cookie and a
state generation, without retaining a dereferenceable vCPU pointer;
- task Vector save/restore/discard, kernel-mode Vector use, preemptible kernel
Vector task resume and CPU state changes invalidate a resident guest before
the registers can be reused;
- reset, dirty saves and KVM_SET_ONE_REG advance the generation;
- KVM_SET_ONE_REG advances the generation before copy_from_user(), because a
partial copy can modify the destination and still return EFAULT;
- the optimization is guarded by a read-only module parameter and defaults to
disabled. The generic invalidation sites are guarded by a static key.
Patch 1 establishes the generic invalidation boundary without changing KVM
behavior. Patch 2 adds the owner tracking, CSR-only restore and tracepoints.
Patch 3 adds focused lifetime, ONE_REG and partial-fault regression tests.
This series currently depends on the separately posted build fix:
Message-ID: <OSQPR01MB6326DDE9C2F390EA3F0F444EC2662@OSQPR01MB6326.apcprd01.prod.exchangelabs.com>
Link: https://lore.kernel.org/all/OSQPR01MB6326DDE9C2F390EA3F0F444EC2662@OSQPR01MB6326.apcprd01.prod.exchangelabs.com/
Ze Huang <ze.huang@oss.qualcomm.com> independently tested that prerequisite
with GCC 16.2.1 and Clang 23.1.3 and supplied a Tested-by tag in this reply:
Link: https://lore.kernel.org/all/DMCBQSJ0B22A.2TDXAXPJNPZL2@oss.qualcomm.com/
Validation on Torvalds master 21a7f44fd076:
- CONFIG_RISCV_ISA_V=y, CONFIG_KVM=m: full Image + modules build;
- CONFIG_RISCV_ISA_V=n, CONFIG_KVM=m: full Image + modules build;
- lifetime, ONE_REG and partial-fault tests pass with the fast path disabled
and enabled;
- nested KVM/RVV smoke passes 3/3 with the fast path enabled;
- the smoke trace recorded 785 owner hits, 784 CSR-only restores and eight
conservative full fallbacks, with no trace-buffer overwrite;
- no emergency, alert, critical or error messages were present in dmesg.
Development note: OpenAI Codex was used in an extended interactive session to
assist with source analysis, implementation and selftest drafting, experiment
orchestration, patch review and changelog preparation. The prompts focused on
modeling per-hart ownership as a cache-validity problem, enumerating Vector
register invalidation boundaries, reproducing partial KVM_SET_ONE_REG writes
and reviewing the resulting patches. The build and runtime results above were
collected from the stated test environment rather than inferred by the model.
The nested QEMU result demonstrates functional behavior and event counts only;
it is not presented as a performance result. Hardware performance measurement
remains future work.
Feedback is especially welcome on whether the generic invalidation callback is
an acceptable RISC-V Vector/KVM boundary, whether the default-off module
parameter is useful for the first version, and whether the behavior patch
should be split further after the interface direction is agreed.
Xiaochun Yu (3):
riscv: vector: Add a guest owner invalidation hook
RISC-V: KVM: Preserve guest Vector state across vCPU put/load
KVM: riscv: selftests: Add Vector owner regression tests
arch/riscv/include/asm/kvm_host.h | 4 +
arch/riscv/include/asm/kvm_vcpu_vector.h | 54 +++-
arch/riscv/include/asm/processor.h | 1 +
arch/riscv/include/asm/vector.h | 31 +++
arch/riscv/kernel/kernel_mode_vector.c | 38 +++
arch/riscv/kvm/trace.h | 98 +++++++
arch/riscv/kvm/vcpu.c | 54 +++-
arch/riscv/kvm/vcpu_vector.c | 252 +++++++++++++++++-
tools/testing/selftests/kvm/Makefile.kvm | 3 +
.../kvm/riscv/vector_owner_fault_test.c | 147 ++++++++++
.../kvm/riscv/vector_owner_lifetime_test.c | 127 +++++++++
.../kvm/riscv/vector_owner_onereg_test.c | 191 +++++++++++++
12 files changed, 976 insertions(+), 24 deletions(-)
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_onereg_test.c
base-commit: 21a7f44fd0765fcc5194130b28767786eba1d1e7
prerequisite-patch-id: 370f0468ecd4354950ab075162058e4b7c31768b
--
2.43.0
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 15+ messages in thread
* [RFC PATCH 0/3] RISC-V: KVM: Preserve resident guest Vector state
@ 2026-10-27 13:13 ` slack up
0 siblings, 0 replies; 15+ messages in thread
From: slack up @ 2026-10-27 13:13 UTC (permalink / raw)
To: Anup Patel, Atish Patra, Palmer Dabbelt, kvm-riscv, linux-riscv
Cc: Paul Walmsley, Albert Ou, Alexandre Ghiti, Paolo Bonzini,
Sean Christopherson, Shuah Khan, kvm, linux-kernel,
linux-kselftest, Xiaochun Yu
From: Xiaochun Yu <yxcnew@outlook.com>
kvm_arch_vcpu_put() saves the guest Vector state and restores the host
register file. A later vCPU load on the same hart saves the host again and
requests a full guest restore, even when no host user touched the Vector
data registers in between.
This RFC keeps the guest Vector register file resident across a vCPU
put/load pair when it is safe to do so. A matching load restores only the
control CSRs; all uncertain cases retain the existing full-restore path.
The series is intentionally conservative:
- every dirty vCPU put still saves the guest state, so guest memory remains
authoritative for migration and userspace access;
- a per-hart owner records an opaque vCPU identity, an allocation cookie and a
state generation, without retaining a dereferenceable vCPU pointer;
- task Vector save/restore/discard, kernel-mode Vector use, preemptible kernel
Vector task resume and CPU state changes invalidate a resident guest before
the registers can be reused;
- reset, dirty saves and KVM_SET_ONE_REG advance the generation;
- KVM_SET_ONE_REG advances the generation before copy_from_user(), because a
partial copy can modify the destination and still return EFAULT;
- the optimization is guarded by a read-only module parameter and defaults to
disabled. The generic invalidation sites are guarded by a static key.
Patch 1 establishes the generic invalidation boundary without changing KVM
behavior. Patch 2 adds the owner tracking, CSR-only restore and tracepoints.
Patch 3 adds focused lifetime, ONE_REG and partial-fault regression tests.
This series currently depends on the separately posted build fix:
Message-ID: <OSQPR01MB6326DDE9C2F390EA3F0F444EC2662@OSQPR01MB6326.apcprd01.prod.exchangelabs.com>
Link: https://lore.kernel.org/all/OSQPR01MB6326DDE9C2F390EA3F0F444EC2662@OSQPR01MB6326.apcprd01.prod.exchangelabs.com/
Ze Huang <ze.huang@oss.qualcomm.com> independently tested that prerequisite
with GCC 16.2.1 and Clang 23.1.3 and supplied a Tested-by tag in this reply:
Link: https://lore.kernel.org/all/DMCBQSJ0B22A.2TDXAXPJNPZL2@oss.qualcomm.com/
Validation on Torvalds master 21a7f44fd076:
- CONFIG_RISCV_ISA_V=y, CONFIG_KVM=m: full Image + modules build;
- CONFIG_RISCV_ISA_V=n, CONFIG_KVM=m: full Image + modules build;
- lifetime, ONE_REG and partial-fault tests pass with the fast path disabled
and enabled;
- nested KVM/RVV smoke passes 3/3 with the fast path enabled;
- the smoke trace recorded 785 owner hits, 784 CSR-only restores and eight
conservative full fallbacks, with no trace-buffer overwrite;
- no emergency, alert, critical or error messages were present in dmesg.
Development note: OpenAI Codex was used in an extended interactive session to
assist with source analysis, implementation and selftest drafting, experiment
orchestration, patch review and changelog preparation. The prompts focused on
modeling per-hart ownership as a cache-validity problem, enumerating Vector
register invalidation boundaries, reproducing partial KVM_SET_ONE_REG writes
and reviewing the resulting patches. The build and runtime results above were
collected from the stated test environment rather than inferred by the model.
The nested QEMU result demonstrates functional behavior and event counts only;
it is not presented as a performance result. Hardware performance measurement
remains future work.
Feedback is especially welcome on whether the generic invalidation callback is
an acceptable RISC-V Vector/KVM boundary, whether the default-off module
parameter is useful for the first version, and whether the behavior patch
should be split further after the interface direction is agreed.
Xiaochun Yu (3):
riscv: vector: Add a guest owner invalidation hook
RISC-V: KVM: Preserve guest Vector state across vCPU put/load
KVM: riscv: selftests: Add Vector owner regression tests
arch/riscv/include/asm/kvm_host.h | 4 +
arch/riscv/include/asm/kvm_vcpu_vector.h | 54 +++-
arch/riscv/include/asm/processor.h | 1 +
arch/riscv/include/asm/vector.h | 31 +++
arch/riscv/kernel/kernel_mode_vector.c | 38 +++
arch/riscv/kvm/trace.h | 98 +++++++
arch/riscv/kvm/vcpu.c | 54 +++-
arch/riscv/kvm/vcpu_vector.c | 252 +++++++++++++++++-
tools/testing/selftests/kvm/Makefile.kvm | 3 +
.../kvm/riscv/vector_owner_fault_test.c | 147 ++++++++++
.../kvm/riscv/vector_owner_lifetime_test.c | 127 +++++++++
.../kvm/riscv/vector_owner_onereg_test.c | 191 +++++++++++++
12 files changed, 976 insertions(+), 24 deletions(-)
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_onereg_test.c
base-commit: 21a7f44fd0765fcc5194130b28767786eba1d1e7
prerequisite-patch-id: 370f0468ecd4354950ab075162058e4b7c31768b
--
2.43.0
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 15+ messages in thread
* [RFC PATCH 1/3] riscv: vector: Add a guest owner invalidation hook
2026-10-27 13:13 ` slack up
(?)
@ 2026-10-27 13:13 ` slack up
-1 siblings, 0 replies; 15+ messages in thread
From: slack up @ 2026-10-27 13:13 UTC (permalink / raw)
To: Anup Patel, Atish Patra, Palmer Dabbelt, kvm-riscv, linux-riscv
Cc: Paul Walmsley, Albert Ou, Alexandre Ghiti, Paolo Bonzini,
Sean Christopherson, Shuah Khan, kvm, linux-kernel,
linux-kselftest, Xiaochun Yu
From: Xiaochun Yu <yxcnew@outlook.com>
KVM can avoid restoring a guest Vector register file when it remains
resident on the same hart, but only if every generic Vector user
invalidates that ownership before touching the registers.
Add a single RCU-protected invalidation callback and guard its call sites
with a static key. Hook task save, restore and discard, kernel Vector
entry, and the resume path for preemptible kernel Vector contexts. With
no registered consumer, the static key leaves the generic paths disabled.
This patch only establishes the invalidation boundary; it does not change
KVM behavior.
Development note: OpenAI Codex assisted with source analysis,
implementation drafting, test orchestration and changelog preparation.
Signed-off-by: Xiaochun Yu <yxcnew@outlook.com>
---
arch/riscv/include/asm/vector.h | 31 +++++++++++++++++++++
arch/riscv/kernel/kernel_mode_vector.c | 38 ++++++++++++++++++++++++++
2 files changed, 69 insertions(+)
diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h
index 1f8dcae3af51..967dd70f4ea4 100644
--- a/arch/riscv/include/asm/vector.h
+++ b/arch/riscv/include/asm/vector.h
@@ -9,8 +9,17 @@
#include <linux/types.h>
#include <uapi/asm-generic/errno.h>
+enum riscv_v_invalidate_reason {
+ RISCV_V_INVALIDATE_TASK_SAVE,
+ RISCV_V_INVALIDATE_TASK_RESTORE,
+ RISCV_V_INVALIDATE_TASK_DISCARD,
+ RISCV_V_INVALIDATE_KERNEL_VECTOR,
+ RISCV_V_INVALIDATE_CPU_STATE,
+};
+
#ifdef CONFIG_RISCV_ISA_V
+#include <linux/jump_label.h>
#include <linux/stringify.h>
#include <linux/sched.h>
#include <linux/sched/task_stack.h>
@@ -52,6 +61,17 @@ void riscv_v_thread_free(struct task_struct *tsk);
void __init riscv_v_setup_ctx_cache(void);
void riscv_v_thread_alloc(struct task_struct *tsk);
void __init update_regset_vector_info(unsigned long size);
+DECLARE_STATIC_KEY_FALSE(riscv_v_owner_invalidate_key);
+void __riscv_v_owner_invalidate(enum riscv_v_invalidate_reason reason);
+void kvm_riscv_register_vctx_invalidate(void (*func)(enum riscv_v_invalidate_reason reason));
+void kvm_riscv_unregister_vctx_invalidate(void);
+
+static __always_inline void
+riscv_v_owner_invalidate(enum riscv_v_invalidate_reason reason)
+{
+ if (static_branch_unlikely(&riscv_v_owner_invalidate_key))
+ __riscv_v_owner_invalidate(reason);
+}
static inline u32 riscv_v_flags(void)
{
@@ -303,6 +323,7 @@ static inline void __riscv_v_vstate_discard(void)
static inline void riscv_v_vstate_discard(struct pt_regs *regs)
{
if (riscv_v_vstate_query(regs)) {
+ riscv_v_owner_invalidate(RISCV_V_INVALIDATE_TASK_DISCARD);
__riscv_v_vstate_discard();
__riscv_v_vstate_dirty(regs);
}
@@ -312,6 +333,7 @@ static inline void riscv_v_vstate_save(struct __riscv_v_ext_state *vstate,
struct pt_regs *regs)
{
if (__riscv_v_vstate_check(regs->status, DIRTY)) {
+ riscv_v_owner_invalidate(RISCV_V_INVALIDATE_TASK_SAVE);
__riscv_v_vstate_save(vstate, vstate->datap);
__riscv_v_vstate_clean(regs);
}
@@ -321,6 +343,7 @@ static inline void riscv_v_vstate_restore(struct __riscv_v_ext_state *vstate,
struct pt_regs *regs)
{
if (riscv_v_vstate_query(regs)) {
+ riscv_v_owner_invalidate(RISCV_V_INVALIDATE_TASK_RESTORE);
__riscv_v_vstate_restore(vstate, vstate->datap);
__riscv_v_vstate_clean(regs);
}
@@ -393,6 +416,13 @@ static inline void __switch_to_vector(struct task_struct *prev,
}
if (riscv_preempt_v_started(next)) {
+ /*
+ * A preemptible kernel Vector user can be scheduled back after a
+ * vCPU published a guest owner on this hart. Drop that owner before
+ * the kernel context resumes or restores and overwrites the register
+ * file.
+ */
+ riscv_v_owner_invalidate(RISCV_V_INVALIDATE_KERNEL_VECTOR);
if (next->thread.riscv_v_flags & RISCV_PREEMPT_V_IN_SCHEDULE) {
next->thread.riscv_v_flags &= ~RISCV_PREEMPT_V_IN_SCHEDULE;
riscv_v_enable();
@@ -423,6 +453,7 @@ static inline bool riscv_v_vstate_ctrl_user_allowed(void) { return false; }
#define riscv_v_vstate_discard(regs) do {} while (0)
#define riscv_v_vstate_save(vstate, regs) do {} while (0)
#define riscv_v_vstate_restore(vstate, regs) do {} while (0)
+#define riscv_v_owner_invalidate(reason) ((void)(reason))
#define __switch_to_vector(__prev, __next) do {} while (0)
#define riscv_v_vstate_off(regs) do {} while (0)
#define riscv_v_vstate_on(regs) do {} while (0)
diff --git a/arch/riscv/kernel/kernel_mode_vector.c b/arch/riscv/kernel/kernel_mode_vector.c
index 77e98b504485..9e87bdf02028 100644
--- a/arch/riscv/kernel/kernel_mode_vector.c
+++ b/arch/riscv/kernel/kernel_mode_vector.c
@@ -22,6 +22,13 @@
static void (* __rcu kvm_flush_vector_ctx_callback)(void);
+/*
+ * KVM may leave a guest Vector register file resident on a hart. Keep the
+ * generic Vector paths unchanged until KVM registers the invalidation hook.
+ */
+static void (* __rcu kvm_invalidate_vector_ctx_callback)(enum riscv_v_invalidate_reason reason);
+DEFINE_STATIC_KEY_FALSE(riscv_v_owner_invalidate_key);
+
void kvm_riscv_register_vctx_callback(void (*func)(void))
{
if (WARN_ON_ONCE(rcu_access_pointer(kvm_flush_vector_ctx_callback)))
@@ -38,6 +45,35 @@ void kvm_riscv_unregister_vctx_callback(void)
}
EXPORT_SYMBOL_GPL(kvm_riscv_unregister_vctx_callback);
+void kvm_riscv_register_vctx_invalidate(void (*func)(enum riscv_v_invalidate_reason reason))
+{
+ if (WARN_ON_ONCE(rcu_access_pointer(kvm_invalidate_vector_ctx_callback)))
+ return;
+
+ rcu_assign_pointer(kvm_invalidate_vector_ctx_callback, func);
+ static_branch_enable(&riscv_v_owner_invalidate_key);
+}
+EXPORT_SYMBOL_GPL(kvm_riscv_register_vctx_invalidate);
+
+void kvm_riscv_unregister_vctx_invalidate(void)
+{
+ static_branch_disable(&riscv_v_owner_invalidate_key);
+ rcu_assign_pointer(kvm_invalidate_vector_ctx_callback, NULL);
+ synchronize_rcu();
+}
+EXPORT_SYMBOL_GPL(kvm_riscv_unregister_vctx_invalidate);
+
+void __riscv_v_owner_invalidate(enum riscv_v_invalidate_reason reason)
+{
+ void (*invalidate_callback)(enum riscv_v_invalidate_reason reason);
+
+ rcu_read_lock();
+ invalidate_callback = rcu_dereference(kvm_invalidate_vector_ctx_callback);
+ if (invalidate_callback)
+ invalidate_callback(reason);
+ rcu_read_unlock();
+}
+
static inline void riscv_v_start(u32 flags)
{
@@ -106,6 +142,8 @@ static void __riscv_flush_vector_context(void)
{
void (*vcpu_flush_v_callback)(void);
+ riscv_v_owner_invalidate(RISCV_V_INVALIDATE_KERNEL_VECTOR);
+
if (riscv_v_flags() & RISCV_V_VCPU_CTX) {
rcu_read_lock();
vcpu_flush_v_callback = rcu_dereference(kvm_flush_vector_ctx_callback);
--
2.43.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [RFC PATCH 1/3] riscv: vector: Add a guest owner invalidation hook
@ 2026-10-27 13:13 ` slack up
0 siblings, 0 replies; 15+ messages in thread
From: slack up @ 2026-10-27 13:13 UTC (permalink / raw)
To: Anup Patel, Atish Patra, Palmer Dabbelt, kvm-riscv, linux-riscv
Cc: Paul Walmsley, Albert Ou, Alexandre Ghiti, Paolo Bonzini,
Sean Christopherson, Shuah Khan, kvm, linux-kernel,
linux-kselftest, Xiaochun Yu
From: Xiaochun Yu <yxcnew@outlook.com>
KVM can avoid restoring a guest Vector register file when it remains
resident on the same hart, but only if every generic Vector user
invalidates that ownership before touching the registers.
Add a single RCU-protected invalidation callback and guard its call sites
with a static key. Hook task save, restore and discard, kernel Vector
entry, and the resume path for preemptible kernel Vector contexts. With
no registered consumer, the static key leaves the generic paths disabled.
This patch only establishes the invalidation boundary; it does not change
KVM behavior.
Development note: OpenAI Codex assisted with source analysis,
implementation drafting, test orchestration and changelog preparation.
Signed-off-by: Xiaochun Yu <yxcnew@outlook.com>
---
arch/riscv/include/asm/vector.h | 31 +++++++++++++++++++++
arch/riscv/kernel/kernel_mode_vector.c | 38 ++++++++++++++++++++++++++
2 files changed, 69 insertions(+)
diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h
index 1f8dcae3af51..967dd70f4ea4 100644
--- a/arch/riscv/include/asm/vector.h
+++ b/arch/riscv/include/asm/vector.h
@@ -9,8 +9,17 @@
#include <linux/types.h>
#include <uapi/asm-generic/errno.h>
+enum riscv_v_invalidate_reason {
+ RISCV_V_INVALIDATE_TASK_SAVE,
+ RISCV_V_INVALIDATE_TASK_RESTORE,
+ RISCV_V_INVALIDATE_TASK_DISCARD,
+ RISCV_V_INVALIDATE_KERNEL_VECTOR,
+ RISCV_V_INVALIDATE_CPU_STATE,
+};
+
#ifdef CONFIG_RISCV_ISA_V
+#include <linux/jump_label.h>
#include <linux/stringify.h>
#include <linux/sched.h>
#include <linux/sched/task_stack.h>
@@ -52,6 +61,17 @@ void riscv_v_thread_free(struct task_struct *tsk);
void __init riscv_v_setup_ctx_cache(void);
void riscv_v_thread_alloc(struct task_struct *tsk);
void __init update_regset_vector_info(unsigned long size);
+DECLARE_STATIC_KEY_FALSE(riscv_v_owner_invalidate_key);
+void __riscv_v_owner_invalidate(enum riscv_v_invalidate_reason reason);
+void kvm_riscv_register_vctx_invalidate(void (*func)(enum riscv_v_invalidate_reason reason));
+void kvm_riscv_unregister_vctx_invalidate(void);
+
+static __always_inline void
+riscv_v_owner_invalidate(enum riscv_v_invalidate_reason reason)
+{
+ if (static_branch_unlikely(&riscv_v_owner_invalidate_key))
+ __riscv_v_owner_invalidate(reason);
+}
static inline u32 riscv_v_flags(void)
{
@@ -303,6 +323,7 @@ static inline void __riscv_v_vstate_discard(void)
static inline void riscv_v_vstate_discard(struct pt_regs *regs)
{
if (riscv_v_vstate_query(regs)) {
+ riscv_v_owner_invalidate(RISCV_V_INVALIDATE_TASK_DISCARD);
__riscv_v_vstate_discard();
__riscv_v_vstate_dirty(regs);
}
@@ -312,6 +333,7 @@ static inline void riscv_v_vstate_save(struct __riscv_v_ext_state *vstate,
struct pt_regs *regs)
{
if (__riscv_v_vstate_check(regs->status, DIRTY)) {
+ riscv_v_owner_invalidate(RISCV_V_INVALIDATE_TASK_SAVE);
__riscv_v_vstate_save(vstate, vstate->datap);
__riscv_v_vstate_clean(regs);
}
@@ -321,6 +343,7 @@ static inline void riscv_v_vstate_restore(struct __riscv_v_ext_state *vstate,
struct pt_regs *regs)
{
if (riscv_v_vstate_query(regs)) {
+ riscv_v_owner_invalidate(RISCV_V_INVALIDATE_TASK_RESTORE);
__riscv_v_vstate_restore(vstate, vstate->datap);
__riscv_v_vstate_clean(regs);
}
@@ -393,6 +416,13 @@ static inline void __switch_to_vector(struct task_struct *prev,
}
if (riscv_preempt_v_started(next)) {
+ /*
+ * A preemptible kernel Vector user can be scheduled back after a
+ * vCPU published a guest owner on this hart. Drop that owner before
+ * the kernel context resumes or restores and overwrites the register
+ * file.
+ */
+ riscv_v_owner_invalidate(RISCV_V_INVALIDATE_KERNEL_VECTOR);
if (next->thread.riscv_v_flags & RISCV_PREEMPT_V_IN_SCHEDULE) {
next->thread.riscv_v_flags &= ~RISCV_PREEMPT_V_IN_SCHEDULE;
riscv_v_enable();
@@ -423,6 +453,7 @@ static inline bool riscv_v_vstate_ctrl_user_allowed(void) { return false; }
#define riscv_v_vstate_discard(regs) do {} while (0)
#define riscv_v_vstate_save(vstate, regs) do {} while (0)
#define riscv_v_vstate_restore(vstate, regs) do {} while (0)
+#define riscv_v_owner_invalidate(reason) ((void)(reason))
#define __switch_to_vector(__prev, __next) do {} while (0)
#define riscv_v_vstate_off(regs) do {} while (0)
#define riscv_v_vstate_on(regs) do {} while (0)
diff --git a/arch/riscv/kernel/kernel_mode_vector.c b/arch/riscv/kernel/kernel_mode_vector.c
index 77e98b504485..9e87bdf02028 100644
--- a/arch/riscv/kernel/kernel_mode_vector.c
+++ b/arch/riscv/kernel/kernel_mode_vector.c
@@ -22,6 +22,13 @@
static void (* __rcu kvm_flush_vector_ctx_callback)(void);
+/*
+ * KVM may leave a guest Vector register file resident on a hart. Keep the
+ * generic Vector paths unchanged until KVM registers the invalidation hook.
+ */
+static void (* __rcu kvm_invalidate_vector_ctx_callback)(enum riscv_v_invalidate_reason reason);
+DEFINE_STATIC_KEY_FALSE(riscv_v_owner_invalidate_key);
+
void kvm_riscv_register_vctx_callback(void (*func)(void))
{
if (WARN_ON_ONCE(rcu_access_pointer(kvm_flush_vector_ctx_callback)))
@@ -38,6 +45,35 @@ void kvm_riscv_unregister_vctx_callback(void)
}
EXPORT_SYMBOL_GPL(kvm_riscv_unregister_vctx_callback);
+void kvm_riscv_register_vctx_invalidate(void (*func)(enum riscv_v_invalidate_reason reason))
+{
+ if (WARN_ON_ONCE(rcu_access_pointer(kvm_invalidate_vector_ctx_callback)))
+ return;
+
+ rcu_assign_pointer(kvm_invalidate_vector_ctx_callback, func);
+ static_branch_enable(&riscv_v_owner_invalidate_key);
+}
+EXPORT_SYMBOL_GPL(kvm_riscv_register_vctx_invalidate);
+
+void kvm_riscv_unregister_vctx_invalidate(void)
+{
+ static_branch_disable(&riscv_v_owner_invalidate_key);
+ rcu_assign_pointer(kvm_invalidate_vector_ctx_callback, NULL);
+ synchronize_rcu();
+}
+EXPORT_SYMBOL_GPL(kvm_riscv_unregister_vctx_invalidate);
+
+void __riscv_v_owner_invalidate(enum riscv_v_invalidate_reason reason)
+{
+ void (*invalidate_callback)(enum riscv_v_invalidate_reason reason);
+
+ rcu_read_lock();
+ invalidate_callback = rcu_dereference(kvm_invalidate_vector_ctx_callback);
+ if (invalidate_callback)
+ invalidate_callback(reason);
+ rcu_read_unlock();
+}
+
static inline void riscv_v_start(u32 flags)
{
@@ -106,6 +142,8 @@ static void __riscv_flush_vector_context(void)
{
void (*vcpu_flush_v_callback)(void);
+ riscv_v_owner_invalidate(RISCV_V_INVALIDATE_KERNEL_VECTOR);
+
if (riscv_v_flags() & RISCV_V_VCPU_CTX) {
rcu_read_lock();
vcpu_flush_v_callback = rcu_dereference(kvm_flush_vector_ctx_callback);
--
2.43.0
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [RFC PATCH 1/3] riscv: vector: Add a guest owner invalidation hook
@ 2026-10-27 13:13 ` slack up
0 siblings, 0 replies; 15+ messages in thread
From: slack up @ 2026-10-27 13:13 UTC (permalink / raw)
To: Anup Patel, Atish Patra, Palmer Dabbelt, kvm-riscv, linux-riscv
Cc: Paul Walmsley, Albert Ou, Alexandre Ghiti, Paolo Bonzini,
Sean Christopherson, Shuah Khan, kvm, linux-kernel,
linux-kselftest, Xiaochun Yu
From: Xiaochun Yu <yxcnew@outlook.com>
KVM can avoid restoring a guest Vector register file when it remains
resident on the same hart, but only if every generic Vector user
invalidates that ownership before touching the registers.
Add a single RCU-protected invalidation callback and guard its call sites
with a static key. Hook task save, restore and discard, kernel Vector
entry, and the resume path for preemptible kernel Vector contexts. With
no registered consumer, the static key leaves the generic paths disabled.
This patch only establishes the invalidation boundary; it does not change
KVM behavior.
Development note: OpenAI Codex assisted with source analysis,
implementation drafting, test orchestration and changelog preparation.
Signed-off-by: Xiaochun Yu <yxcnew@outlook.com>
---
arch/riscv/include/asm/vector.h | 31 +++++++++++++++++++++
arch/riscv/kernel/kernel_mode_vector.c | 38 ++++++++++++++++++++++++++
2 files changed, 69 insertions(+)
diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h
index 1f8dcae3af51..967dd70f4ea4 100644
--- a/arch/riscv/include/asm/vector.h
+++ b/arch/riscv/include/asm/vector.h
@@ -9,8 +9,17 @@
#include <linux/types.h>
#include <uapi/asm-generic/errno.h>
+enum riscv_v_invalidate_reason {
+ RISCV_V_INVALIDATE_TASK_SAVE,
+ RISCV_V_INVALIDATE_TASK_RESTORE,
+ RISCV_V_INVALIDATE_TASK_DISCARD,
+ RISCV_V_INVALIDATE_KERNEL_VECTOR,
+ RISCV_V_INVALIDATE_CPU_STATE,
+};
+
#ifdef CONFIG_RISCV_ISA_V
+#include <linux/jump_label.h>
#include <linux/stringify.h>
#include <linux/sched.h>
#include <linux/sched/task_stack.h>
@@ -52,6 +61,17 @@ void riscv_v_thread_free(struct task_struct *tsk);
void __init riscv_v_setup_ctx_cache(void);
void riscv_v_thread_alloc(struct task_struct *tsk);
void __init update_regset_vector_info(unsigned long size);
+DECLARE_STATIC_KEY_FALSE(riscv_v_owner_invalidate_key);
+void __riscv_v_owner_invalidate(enum riscv_v_invalidate_reason reason);
+void kvm_riscv_register_vctx_invalidate(void (*func)(enum riscv_v_invalidate_reason reason));
+void kvm_riscv_unregister_vctx_invalidate(void);
+
+static __always_inline void
+riscv_v_owner_invalidate(enum riscv_v_invalidate_reason reason)
+{
+ if (static_branch_unlikely(&riscv_v_owner_invalidate_key))
+ __riscv_v_owner_invalidate(reason);
+}
static inline u32 riscv_v_flags(void)
{
@@ -303,6 +323,7 @@ static inline void __riscv_v_vstate_discard(void)
static inline void riscv_v_vstate_discard(struct pt_regs *regs)
{
if (riscv_v_vstate_query(regs)) {
+ riscv_v_owner_invalidate(RISCV_V_INVALIDATE_TASK_DISCARD);
__riscv_v_vstate_discard();
__riscv_v_vstate_dirty(regs);
}
@@ -312,6 +333,7 @@ static inline void riscv_v_vstate_save(struct __riscv_v_ext_state *vstate,
struct pt_regs *regs)
{
if (__riscv_v_vstate_check(regs->status, DIRTY)) {
+ riscv_v_owner_invalidate(RISCV_V_INVALIDATE_TASK_SAVE);
__riscv_v_vstate_save(vstate, vstate->datap);
__riscv_v_vstate_clean(regs);
}
@@ -321,6 +343,7 @@ static inline void riscv_v_vstate_restore(struct __riscv_v_ext_state *vstate,
struct pt_regs *regs)
{
if (riscv_v_vstate_query(regs)) {
+ riscv_v_owner_invalidate(RISCV_V_INVALIDATE_TASK_RESTORE);
__riscv_v_vstate_restore(vstate, vstate->datap);
__riscv_v_vstate_clean(regs);
}
@@ -393,6 +416,13 @@ static inline void __switch_to_vector(struct task_struct *prev,
}
if (riscv_preempt_v_started(next)) {
+ /*
+ * A preemptible kernel Vector user can be scheduled back after a
+ * vCPU published a guest owner on this hart. Drop that owner before
+ * the kernel context resumes or restores and overwrites the register
+ * file.
+ */
+ riscv_v_owner_invalidate(RISCV_V_INVALIDATE_KERNEL_VECTOR);
if (next->thread.riscv_v_flags & RISCV_PREEMPT_V_IN_SCHEDULE) {
next->thread.riscv_v_flags &= ~RISCV_PREEMPT_V_IN_SCHEDULE;
riscv_v_enable();
@@ -423,6 +453,7 @@ static inline bool riscv_v_vstate_ctrl_user_allowed(void) { return false; }
#define riscv_v_vstate_discard(regs) do {} while (0)
#define riscv_v_vstate_save(vstate, regs) do {} while (0)
#define riscv_v_vstate_restore(vstate, regs) do {} while (0)
+#define riscv_v_owner_invalidate(reason) ((void)(reason))
#define __switch_to_vector(__prev, __next) do {} while (0)
#define riscv_v_vstate_off(regs) do {} while (0)
#define riscv_v_vstate_on(regs) do {} while (0)
diff --git a/arch/riscv/kernel/kernel_mode_vector.c b/arch/riscv/kernel/kernel_mode_vector.c
index 77e98b504485..9e87bdf02028 100644
--- a/arch/riscv/kernel/kernel_mode_vector.c
+++ b/arch/riscv/kernel/kernel_mode_vector.c
@@ -22,6 +22,13 @@
static void (* __rcu kvm_flush_vector_ctx_callback)(void);
+/*
+ * KVM may leave a guest Vector register file resident on a hart. Keep the
+ * generic Vector paths unchanged until KVM registers the invalidation hook.
+ */
+static void (* __rcu kvm_invalidate_vector_ctx_callback)(enum riscv_v_invalidate_reason reason);
+DEFINE_STATIC_KEY_FALSE(riscv_v_owner_invalidate_key);
+
void kvm_riscv_register_vctx_callback(void (*func)(void))
{
if (WARN_ON_ONCE(rcu_access_pointer(kvm_flush_vector_ctx_callback)))
@@ -38,6 +45,35 @@ void kvm_riscv_unregister_vctx_callback(void)
}
EXPORT_SYMBOL_GPL(kvm_riscv_unregister_vctx_callback);
+void kvm_riscv_register_vctx_invalidate(void (*func)(enum riscv_v_invalidate_reason reason))
+{
+ if (WARN_ON_ONCE(rcu_access_pointer(kvm_invalidate_vector_ctx_callback)))
+ return;
+
+ rcu_assign_pointer(kvm_invalidate_vector_ctx_callback, func);
+ static_branch_enable(&riscv_v_owner_invalidate_key);
+}
+EXPORT_SYMBOL_GPL(kvm_riscv_register_vctx_invalidate);
+
+void kvm_riscv_unregister_vctx_invalidate(void)
+{
+ static_branch_disable(&riscv_v_owner_invalidate_key);
+ rcu_assign_pointer(kvm_invalidate_vector_ctx_callback, NULL);
+ synchronize_rcu();
+}
+EXPORT_SYMBOL_GPL(kvm_riscv_unregister_vctx_invalidate);
+
+void __riscv_v_owner_invalidate(enum riscv_v_invalidate_reason reason)
+{
+ void (*invalidate_callback)(enum riscv_v_invalidate_reason reason);
+
+ rcu_read_lock();
+ invalidate_callback = rcu_dereference(kvm_invalidate_vector_ctx_callback);
+ if (invalidate_callback)
+ invalidate_callback(reason);
+ rcu_read_unlock();
+}
+
static inline void riscv_v_start(u32 flags)
{
@@ -106,6 +142,8 @@ static void __riscv_flush_vector_context(void)
{
void (*vcpu_flush_v_callback)(void);
+ riscv_v_owner_invalidate(RISCV_V_INVALIDATE_KERNEL_VECTOR);
+
if (riscv_v_flags() & RISCV_V_VCPU_CTX) {
rcu_read_lock();
vcpu_flush_v_callback = rcu_dereference(kvm_flush_vector_ctx_callback);
--
2.43.0
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [RFC PATCH 2/3] RISC-V: KVM: Preserve guest Vector state across vCPU put/load
2026-10-27 13:13 ` slack up
(?)
@ 2026-10-27 13:13 ` slack up
-1 siblings, 0 replies; 15+ messages in thread
From: slack up @ 2026-10-27 13:13 UTC (permalink / raw)
To: Anup Patel, Atish Patra, Palmer Dabbelt, kvm-riscv, linux-riscv
Cc: Paul Walmsley, Albert Ou, Alexandre Ghiti, Paolo Bonzini,
Sean Christopherson, Shuah Khan, kvm, linux-kernel,
linux-kselftest, Xiaochun Yu
From: Xiaochun Yu <yxcnew@outlook.com>
kvm_arch_vcpu_put() saves the guest Vector state and restores the host
register file. A later load on the same hart saves the host again and
requests a full guest restore, even when the host has not used Vector in
between.
Track a per-hart owner with an opaque vCPU identity, an allocation cookie
and a state generation. After saving a dirty guest to memory, leave its
data registers resident when the host has no Vector restore obligation. A
matching load restores only the Vector control CSRs. All mismatches and
generic Vector users fall back to a full restore.
Keep guest memory authoritative by continuing to save at every dirty put.
Advance the generation on saves, resets and SET_ONE_REG. Do so before
copy_from_user() because a fault can leave a modified prefix in the
destination even though the ioctl fails.
Register the generic invalidation hook only when the read-only
vector_owner_fastpath module parameter is enabled. The parameter defaults
to false so the existing full-restore behavior remains the default. Add
tracepoints for restore mode, owner decisions and invalidation reasons.
Development note: OpenAI Codex assisted with source analysis,
implementation drafting, test orchestration and changelog preparation.
Signed-off-by: Xiaochun Yu <yxcnew@outlook.com>
---
arch/riscv/include/asm/kvm_host.h | 4 +
arch/riscv/include/asm/kvm_vcpu_vector.h | 54 ++++-
arch/riscv/include/asm/processor.h | 1 +
arch/riscv/kvm/trace.h | 98 +++++++++
arch/riscv/kvm/vcpu.c | 54 ++++-
arch/riscv/kvm/vcpu_vector.c | 252 ++++++++++++++++++++++-
6 files changed, 439 insertions(+), 24 deletions(-)
diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
index e28f058a94d7..a6e2b3097644 100644
--- a/arch/riscv/include/asm/kvm_host.h
+++ b/arch/riscv/include/asm/kvm_host.h
@@ -204,6 +204,10 @@ struct kvm_vcpu_arch {
/* CPU context of Guest VCPU */
struct kvm_cpu_context guest_context;
+ /* Identity and version used by Vector owner tracking. */
+ u64 vector_owner_cookie;
+ u64 vector_state_generation;
+
/* CPU CSR context of Guest VCPU */
struct kvm_vcpu_csr guest_csr;
diff --git a/arch/riscv/include/asm/kvm_vcpu_vector.h b/arch/riscv/include/asm/kvm_vcpu_vector.h
index ebc619ed67dd..7f3eecf4217b 100644
--- a/arch/riscv/include/asm/kvm_vcpu_vector.h
+++ b/arch/riscv/include/asm/kvm_vcpu_vector.h
@@ -26,29 +26,47 @@ static __always_inline void __kvm_riscv_vector_restore(struct kvm_cpu_context *c
__riscv_v_vstate_restore(&context->vector, context->vector.datap);
}
+static __always_inline void __kvm_riscv_vector_restore_csrs(struct kvm_cpu_context *context)
+{
+ riscv_v_enable();
+ __vstate_csr_restore(&context->vector);
+ riscv_v_disable();
+}
+
void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu);
void kvm_riscv_vcpu_guest_vector_save(struct kvm_cpu_context *cntx,
unsigned long *isa);
void kvm_riscv_vcpu_guest_vector_restore(struct kvm_cpu_context *cntx,
- unsigned long *isa);
+ unsigned long *isa, bool csr_only);
void kvm_riscv_vcpu_host_vector_save(struct kvm_cpu_context *cntx);
-void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx);
+void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx,
+ bool preserve_guest);
int kvm_riscv_vcpu_alloc_vector_context(struct kvm_vcpu *vcpu);
void kvm_riscv_vcpu_free_vector_context(struct kvm_vcpu *vcpu);
void kvm_riscv_register_vctx_callback(void (*func)(void));
void kvm_riscv_unregister_vctx_callback(void);
+bool kvm_riscv_vcpu_vector_owner_enabled(void);
+bool kvm_riscv_vcpu_vector_owner_load(struct kvm_vcpu *vcpu, int cpu);
+bool kvm_riscv_vcpu_vector_owner_put(struct kvm_vcpu *vcpu);
+void kvm_riscv_vcpu_vector_owner_invalidate(enum riscv_v_invalidate_reason reason);
void kvm_riscv_vcpu_flush_vector(void);
static inline void kvm_riscv_v_init(void)
{
- if (has_vector())
+ if (has_vector()) {
kvm_riscv_register_vctx_callback(&kvm_riscv_vcpu_flush_vector);
+ if (kvm_riscv_vcpu_vector_owner_enabled())
+ kvm_riscv_register_vctx_invalidate(&kvm_riscv_vcpu_vector_owner_invalidate);
+ }
}
static inline void kvm_riscv_v_exit(void)
{
- if (has_vector())
+ if (has_vector()) {
+ if (kvm_riscv_vcpu_vector_owner_enabled())
+ kvm_riscv_unregister_vctx_invalidate();
kvm_riscv_unregister_vctx_callback();
+ }
}
#else
@@ -59,6 +77,10 @@ static inline void __kvm_riscv_vector_restore(struct kvm_cpu_context *context)
{
}
+static inline void __kvm_riscv_vector_restore_csrs(struct kvm_cpu_context *context)
+{
+}
+
static inline void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu)
{
}
@@ -69,7 +91,7 @@ static inline void kvm_riscv_vcpu_guest_vector_save(struct kvm_cpu_context *cntx
}
static inline void kvm_riscv_vcpu_guest_vector_restore(struct kvm_cpu_context *cntx,
- unsigned long *isa)
+ unsigned long *isa, bool csr_only)
{
}
@@ -77,7 +99,8 @@ static inline void kvm_riscv_vcpu_host_vector_save(struct kvm_cpu_context *cntx)
{
}
-static inline void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx)
+static inline void
+kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx, bool preserve_guest)
{
}
@@ -90,6 +113,25 @@ static inline void kvm_riscv_vcpu_free_vector_context(struct kvm_vcpu *vcpu)
{
}
+static inline bool kvm_riscv_vcpu_vector_owner_enabled(void)
+{
+ return false;
+}
+
+static inline bool kvm_riscv_vcpu_vector_owner_load(struct kvm_vcpu *vcpu, int cpu)
+{
+ return false;
+}
+
+static inline bool kvm_riscv_vcpu_vector_owner_put(struct kvm_vcpu *vcpu)
+{
+ return false;
+}
+
+static inline void kvm_riscv_vcpu_vector_owner_invalidate(enum riscv_v_invalidate_reason reason)
+{
+}
+
static inline void kvm_riscv_v_init(void)
{
}
diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
index 815715c67f94..abda84ca9548 100644
--- a/arch/riscv/include/asm/processor.h
+++ b/arch/riscv/include/asm/processor.h
@@ -105,6 +105,7 @@ struct pt_regs;
#define RISCV_KERNEL_MODE_V 0x00000001
#define RISCV_V_VCPU_NEED_RESTORE 0x00000002
#define RISCV_V_VCPU_CTX 0x00000004
+#define RISCV_V_VCPU_CSR_RESTORE 0x00000008
#define RISCV_PREEMPT_V 0x00000100
#define RISCV_PREEMPT_V_DIRTY 0x80000000
#define RISCV_PREEMPT_V_NEED_RESTORE 0x40000000
diff --git a/arch/riscv/kvm/trace.h b/arch/riscv/kvm/trace.h
index 23a5fd1b686e..c805606c704d 100644
--- a/arch/riscv/kvm/trace.h
+++ b/arch/riscv/kvm/trace.h
@@ -133,6 +133,104 @@ TRACE_EVENT(kvm_vcpu_irq,
__entry->vcpu_id, __entry->irq, __entry->level)
);
+TRACE_EVENT(kvm_riscv_vector_restore_request,
+ TP_PROTO(unsigned long vcpu_id, int cpu, unsigned int source,
+ unsigned long host_vs),
+ TP_ARGS(vcpu_id, cpu, source, host_vs),
+
+ TP_STRUCT__entry(
+ __field(unsigned long, vcpu_id)
+ __field(int, cpu)
+ __field(unsigned int, source)
+ __field(unsigned long, host_vs)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_id = vcpu_id;
+ __entry->cpu = cpu;
+ __entry->source = source;
+ __entry->host_vs = host_vs;
+ ),
+
+ TP_printk("VCPU: %lu, CPU: %d, source: %s, host VS: 0x%lx",
+ __entry->vcpu_id, __entry->cpu,
+ __print_symbolic(__entry->source,
+ { 0, "load" }, { 1, "kernel_vector" }),
+ __entry->host_vs)
+);
+
+TRACE_EVENT(kvm_riscv_vector_restore,
+ TP_PROTO(unsigned long vcpu_id, int cpu, bool csr_only),
+ TP_ARGS(vcpu_id, cpu, csr_only),
+
+ TP_STRUCT__entry(
+ __field(unsigned long, vcpu_id)
+ __field(int, cpu)
+ __field(bool, csr_only)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_id = vcpu_id;
+ __entry->cpu = cpu;
+ __entry->csr_only = csr_only;
+ ),
+
+ TP_printk("VCPU: %lu, CPU: %d, mode: %s",
+ __entry->vcpu_id, __entry->cpu,
+ __entry->csr_only ? "csr" : "full")
+);
+
+TRACE_EVENT(kvm_riscv_vector_owner,
+ TP_PROTO(unsigned long vcpu_id, int cpu, unsigned int action,
+ bool eligible, unsigned int reason, unsigned long host_vs,
+ unsigned long guest_vs, u64 generation),
+ TP_ARGS(vcpu_id, cpu, action, eligible, reason, host_vs, guest_vs,
+ generation),
+
+ TP_STRUCT__entry(
+ __field(unsigned long, vcpu_id)
+ __field(int, cpu)
+ __field(unsigned int, action)
+ __field(bool, eligible)
+ __field(unsigned int, reason)
+ __field(unsigned long, host_vs)
+ __field(unsigned long, guest_vs)
+ __field(u64, generation)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_id = vcpu_id;
+ __entry->cpu = cpu;
+ __entry->action = action;
+ __entry->eligible = eligible;
+ __entry->reason = reason;
+ __entry->host_vs = host_vs;
+ __entry->guest_vs = guest_vs;
+ __entry->generation = generation;
+ ),
+
+ TP_printk("VCPU: %lu, CPU: %d, action: %s, eligible: %d, reason: %s, host VS: 0x%lx, guest VS: 0x%lx, generation: %llu",
+ __entry->vcpu_id, __entry->cpu,
+ __print_symbolic(__entry->action,
+ { 0, "load" }, { 1, "put" },
+ { 2, "invalidate" }),
+ __entry->eligible,
+ __print_symbolic(__entry->reason,
+ { 0, "hit" }, { 1, "empty" },
+ { 2, "other_vcpu" }, { 3, "cookie" },
+ { 4, "generation" }, { 5, "guest_off" },
+ { 6, "host_on" }, { 7, "migration" },
+ { 8, "put_candidate" },
+ { 9, "task_save" },
+ { 10, "task_restore" },
+ { 11, "task_discard" },
+ { 12, "kernel_vector" },
+ { 13, "cpu_state" },
+ { 14, "not_resident" }),
+ __entry->host_vs, __entry->guest_vs,
+ __entry->generation)
+);
+
#endif /* _TRACE_KVM_H */
#undef TRACE_INCLUDE_PATH
diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
index 7f1636c44d41..a59cac26ca7c 100644
--- a/arch/riscv/kvm/vcpu.c
+++ b/arch/riscv/kvm/vcpu.c
@@ -35,6 +35,7 @@ void kvm_riscv_clear_former_vcpu(void)
* via the fast-path optimization.
*/
__this_cpu_write(kvm_former_vcpu, NULL);
+ kvm_riscv_vcpu_vector_owner_invalidate(RISCV_V_INVALIDATE_CPU_STATE);
}
const struct kvm_stats_desc kvm_vcpu_stats_desc[] = {
@@ -589,6 +590,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
{
void *nsh;
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
+ bool vector_owner_hit;
/*
* If VCPU is being reloaded on the same physical CPU and no
@@ -647,9 +649,11 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
kvm_riscv_vcpu_guest_fp_restore(&vcpu->arch.guest_context,
vcpu->arch.isa);
get_cpu_vector_context();
- kvm_riscv_vcpu_host_vector_save(&vcpu->arch.host_context);
+ vector_owner_hit = kvm_riscv_vcpu_vector_owner_load(vcpu, cpu);
+ if (!vector_owner_hit)
+ kvm_riscv_vcpu_host_vector_save(&vcpu->arch.host_context);
kvm_riscv_vcpu_guest_vector_restore(&vcpu->arch.guest_context,
- vcpu->arch.isa);
+ vcpu->arch.isa, vector_owner_hit);
put_cpu_vector_context();
kvm_make_request(KVM_REQ_STEAL_UPDATE, vcpu);
@@ -661,6 +665,7 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
{
void *nsh;
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
+ bool preserve_guest;
vcpu->cpu = -1;
@@ -674,7 +679,9 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
get_cpu_vector_context();
kvm_riscv_vcpu_guest_vector_save(&vcpu->arch.guest_context,
vcpu->arch.isa);
- kvm_riscv_vcpu_host_vector_restore(&vcpu->arch.host_context);
+ preserve_guest = kvm_riscv_vcpu_vector_owner_put(vcpu);
+ kvm_riscv_vcpu_host_vector_restore(&vcpu->arch.host_context,
+ preserve_guest);
put_cpu_vector_context();
if (kvm_riscv_nacl_available()) {
@@ -793,6 +800,12 @@ static __always_inline void kvm_riscv_vcpu_swap_in_host_state(struct kvm_vcpu *v
zicficsr->ssp = csr_swap(CSR_SSP, 0);
}
+enum kvm_riscv_vector_restore_type {
+ KVM_RISCV_VECTOR_RESTORE_NONE,
+ KVM_RISCV_VECTOR_RESTORE_FULL,
+ KVM_RISCV_VECTOR_RESTORE_CSR,
+};
+
/*
* Actually run the vCPU, entering an RCU extended quiescent state (EQS) while
* the vCPU is running.
@@ -800,12 +813,14 @@ static __always_inline void kvm_riscv_vcpu_swap_in_host_state(struct kvm_vcpu *v
* This must be noinstr as instrumentation may make use of RCU, and this is not
* safe during the EQS.
*/
-static void noinstr kvm_riscv_vcpu_enter_exit(struct kvm_vcpu *vcpu,
- struct kvm_cpu_trap *trap)
+static int noinstr kvm_riscv_vcpu_enter_exit(struct kvm_vcpu *vcpu,
+ struct kvm_cpu_trap *trap)
{
void *nsh;
struct kvm_cpu_context *gcntx = &vcpu->arch.guest_context;
struct kvm_cpu_context *hcntx = &vcpu->arch.host_context;
+ int vector_restore = KVM_RISCV_VECTOR_RESTORE_NONE;
+ u32 vector_flags;
/*
* We save trap CSRs (such as SEPC, SCAUSE, STVAL, HTVAL, and
@@ -818,11 +833,21 @@ static void noinstr kvm_riscv_vcpu_enter_exit(struct kvm_vcpu *vcpu,
kvm_riscv_vcpu_swap_in_guest_state(vcpu);
guest_state_enter_irqoff();
- /* sstatus.VS != SR_VS_OFF is guaranteed when NEED_RESTORE is set */
- if (current->thread.riscv_v_flags & RISCV_V_VCPU_NEED_RESTORE) {
- current->thread.riscv_v_flags &= ~RISCV_V_VCPU_NEED_RESTORE;
+ vector_flags = current->thread.riscv_v_flags;
+ /* sstatus.VS != SR_VS_OFF is guaranteed for either restore request. */
+ if (vector_flags & (RISCV_V_VCPU_NEED_RESTORE |
+ RISCV_V_VCPU_CSR_RESTORE)) {
+ current->thread.riscv_v_flags &=
+ ~(RISCV_V_VCPU_NEED_RESTORE |
+ RISCV_V_VCPU_CSR_RESTORE);
current->thread.riscv_v_flags |= RISCV_V_VCPU_CTX;
- __kvm_riscv_vector_restore(gcntx);
+ if (vector_flags & RISCV_V_VCPU_NEED_RESTORE) {
+ __kvm_riscv_vector_restore(gcntx);
+ vector_restore = KVM_RISCV_VECTOR_RESTORE_FULL;
+ } else {
+ __kvm_riscv_vector_restore_csrs(gcntx);
+ vector_restore = KVM_RISCV_VECTOR_RESTORE_CSR;
+ }
gcntx->sstatus = (gcntx->sstatus & ~SR_VS) | SR_VS_CLEAN;
}
@@ -886,6 +911,8 @@ static void noinstr kvm_riscv_vcpu_enter_exit(struct kvm_vcpu *vcpu,
vcpu->arch.last_exit_cpu = vcpu->cpu;
guest_state_exit_irqoff();
kvm_riscv_vcpu_swap_in_host_state(vcpu);
+
+ return vector_restore;
}
int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
@@ -893,6 +920,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
int ret;
struct kvm_cpu_trap trap;
struct kvm_run *run = vcpu->run;
+ int vector_restore;
if (!vcpu->arch.ran_atleast_once)
kvm_riscv_vcpu_config_ran_once(vcpu);
@@ -1001,7 +1029,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
guest_timing_enter_irqoff();
- kvm_riscv_vcpu_enter_exit(vcpu, &trap);
+ vector_restore = kvm_riscv_vcpu_enter_exit(vcpu, &trap);
vcpu->mode = OUTSIDE_GUEST_MODE;
vcpu->stat.exits++;
@@ -1026,6 +1054,12 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
local_irq_enable();
+ if (vector_restore == KVM_RISCV_VECTOR_RESTORE_FULL)
+ trace_kvm_riscv_vector_restore(vcpu->vcpu_id,
+ smp_processor_id(), false);
+ else if (vector_restore == KVM_RISCV_VECTOR_RESTORE_CSR)
+ trace_kvm_riscv_vector_restore(vcpu->vcpu_id,
+ smp_processor_id(), true);
trace_kvm_exit(&trap);
preempt_enable();
diff --git a/arch/riscv/kvm/vcpu_vector.c b/arch/riscv/kvm/vcpu_vector.c
index fefc2857f753..c197d3d7b539 100644
--- a/arch/riscv/kvm/vcpu_vector.c
+++ b/arch/riscv/kvm/vcpu_vector.c
@@ -7,9 +7,11 @@
* Greentime Hu <greentime.hu@sifive.com>
*/
+#include <linux/atomic.h>
#include <linux/errno.h>
#include <linux/err.h>
#include <linux/kvm_host.h>
+#include <linux/moduleparam.h>
#include <linux/nospec.h>
#include <linux/uaccess.h>
#include <asm/cpufeature.h>
@@ -17,7 +19,50 @@
#include <asm/kvm_vcpu_vector.h>
#include <asm/vector.h>
+#include "trace.h"
+
#ifdef CONFIG_RISCV_ISA_V
+enum kvm_riscv_vector_owner_reason {
+ KVM_RISCV_VECTOR_OWNER_HIT,
+ KVM_RISCV_VECTOR_OWNER_EMPTY,
+ KVM_RISCV_VECTOR_OWNER_OTHER_VCPU,
+ KVM_RISCV_VECTOR_OWNER_COOKIE,
+ KVM_RISCV_VECTOR_OWNER_GENERATION,
+ KVM_RISCV_VECTOR_OWNER_GUEST_OFF,
+ KVM_RISCV_VECTOR_OWNER_HOST_ON,
+ KVM_RISCV_VECTOR_OWNER_MIGRATION,
+ KVM_RISCV_VECTOR_OWNER_PUT_CANDIDATE,
+ KVM_RISCV_VECTOR_OWNER_TASK_SAVE,
+ KVM_RISCV_VECTOR_OWNER_TASK_RESTORE,
+ KVM_RISCV_VECTOR_OWNER_TASK_DISCARD,
+ KVM_RISCV_VECTOR_OWNER_KERNEL_VECTOR,
+ KVM_RISCV_VECTOR_OWNER_CPU_STATE,
+ KVM_RISCV_VECTOR_OWNER_NOT_RESIDENT,
+};
+
+struct kvm_riscv_vector_owner {
+ /* Opaque comparison token; cookie rejects address reuse after destroy. */
+ unsigned long identity;
+ u64 cookie;
+ u64 generation;
+ unsigned long vcpu_id;
+ unsigned long guest_vs;
+};
+
+static DEFINE_PER_CPU(struct kvm_riscv_vector_owner,
+ kvm_riscv_vector_owner);
+static atomic64_t kvm_riscv_vector_owner_cookie = ATOMIC64_INIT(0);
+static bool kvm_riscv_vector_owner_fastpath;
+module_param_named(vector_owner_fastpath, kvm_riscv_vector_owner_fastpath,
+ bool, 0444);
+MODULE_PARM_DESC(vector_owner_fastpath,
+ "Preserve matched guest Vector registers across vCPU put/load");
+
+bool kvm_riscv_vcpu_vector_owner_enabled(void)
+{
+ return READ_ONCE(kvm_riscv_vector_owner_fastpath);
+}
+
void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu)
{
unsigned long *isa = vcpu->arch.isa;
@@ -34,6 +79,8 @@ void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu)
} else {
cntx->sstatus |= SR_VS_OFF;
}
+
+ vcpu->arch.vector_state_generation++;
}
static void kvm_riscv_vcpu_vector_clean(struct kvm_cpu_context *cntx)
@@ -46,18 +93,35 @@ void kvm_riscv_vcpu_guest_vector_save(struct kvm_cpu_context *cntx,
unsigned long *isa)
{
if ((cntx->sstatus & SR_VS) == SR_VS_DIRTY) {
- if (riscv_isa_extension_available(isa, V))
+ if (riscv_isa_extension_available(isa, V)) {
+ struct kvm_vcpu *vcpu = container_of(cntx, struct kvm_vcpu,
+ arch.guest_context);
+
__kvm_riscv_vector_save(cntx);
+ vcpu->arch.vector_state_generation++;
+ }
kvm_riscv_vcpu_vector_clean(cntx);
}
}
void kvm_riscv_vcpu_guest_vector_restore(struct kvm_cpu_context *cntx,
- unsigned long *isa)
+ unsigned long *isa, bool csr_only)
{
if ((cntx->sstatus & SR_VS) != SR_VS_OFF) {
- if (riscv_isa_extension_available(isa, V))
- riscv_v_flags_set(riscv_v_flags() | RISCV_V_VCPU_NEED_RESTORE);
+ if (riscv_isa_extension_available(isa, V)) {
+ struct kvm_vcpu *vcpu = container_of(cntx, struct kvm_vcpu,
+ arch.guest_context);
+ unsigned long host_vs = task_pt_regs(current)->status & SR_VS;
+ u32 flags = riscv_v_flags();
+
+ flags &= ~(RISCV_V_VCPU_NEED_RESTORE |
+ RISCV_V_VCPU_CSR_RESTORE);
+ flags |= csr_only ? RISCV_V_VCPU_CSR_RESTORE :
+ RISCV_V_VCPU_NEED_RESTORE;
+ riscv_v_flags_set(flags);
+ trace_kvm_riscv_vector_restore_request(vcpu->vcpu_id,
+ smp_processor_id(), 0, host_vs);
+ }
}
}
@@ -68,11 +132,167 @@ void kvm_riscv_vcpu_host_vector_save(struct kvm_cpu_context *cntx)
__kvm_riscv_vector_save(cntx);
}
-void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx)
+void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx,
+ bool preserve_guest)
{
- if (!kvm_riscv_isa_check_host(V))
+ if (!preserve_guest && !kvm_riscv_isa_check_host(V))
__kvm_riscv_vector_restore(cntx);
- riscv_v_flags_set(riscv_v_flags() & ~(RISCV_V_VCPU_CTX | RISCV_V_VCPU_NEED_RESTORE));
+ riscv_v_flags_set(riscv_v_flags() &
+ ~(RISCV_V_VCPU_CTX | RISCV_V_VCPU_NEED_RESTORE |
+ RISCV_V_VCPU_CSR_RESTORE));
+}
+
+bool kvm_riscv_vcpu_vector_owner_load(struct kvm_vcpu *vcpu, int cpu)
+{
+ struct kvm_riscv_vector_owner *owner;
+ unsigned long guest_vs = vcpu->arch.guest_context.sstatus & SR_VS;
+ unsigned long host_vs = task_pt_regs(current)->status & SR_VS;
+ enum kvm_riscv_vector_owner_reason reason;
+ bool eligible = false;
+
+ if (!kvm_riscv_vcpu_vector_owner_enabled())
+ return false;
+
+ owner = this_cpu_ptr(&kvm_riscv_vector_owner);
+ if (guest_vs == SR_VS_OFF) {
+ reason = KVM_RISCV_VECTOR_OWNER_GUEST_OFF;
+ } else if (host_vs != SR_VS_OFF) {
+ reason = KVM_RISCV_VECTOR_OWNER_HOST_ON;
+ } else if (!owner->identity) {
+ reason = vcpu->arch.last_exit_cpu == cpu ?
+ KVM_RISCV_VECTOR_OWNER_EMPTY :
+ KVM_RISCV_VECTOR_OWNER_MIGRATION;
+ } else if (owner->identity != (unsigned long)vcpu) {
+ reason = KVM_RISCV_VECTOR_OWNER_OTHER_VCPU;
+ } else if (owner->cookie != vcpu->arch.vector_owner_cookie) {
+ reason = KVM_RISCV_VECTOR_OWNER_COOKIE;
+ } else if (owner->generation != vcpu->arch.vector_state_generation) {
+ reason = KVM_RISCV_VECTOR_OWNER_GENERATION;
+ } else {
+ reason = KVM_RISCV_VECTOR_OWNER_HIT;
+ eligible = true;
+ }
+
+ trace_kvm_riscv_vector_owner(vcpu->vcpu_id, cpu, 0, eligible,
+ reason, host_vs, guest_vs,
+ vcpu->arch.vector_state_generation);
+
+ /* A load consumes the cached owner; the next put publishes it. */
+ owner->identity = 0;
+ owner->cookie = 0;
+ owner->generation = 0;
+ owner->vcpu_id = 0;
+ owner->guest_vs = 0;
+
+ return eligible;
+}
+
+bool kvm_riscv_vcpu_vector_owner_put(struct kvm_vcpu *vcpu)
+{
+ struct kvm_riscv_vector_owner *owner;
+ unsigned long guest_vs = vcpu->arch.guest_context.sstatus & SR_VS;
+ unsigned long host_vs = task_pt_regs(current)->status & SR_VS;
+ u32 flags = riscv_v_flags();
+ enum kvm_riscv_vector_owner_reason reason;
+ bool eligible;
+
+ if (!kvm_riscv_vcpu_vector_owner_enabled())
+ return false;
+
+ owner = this_cpu_ptr(&kvm_riscv_vector_owner);
+ eligible = guest_vs != SR_VS_OFF && host_vs == SR_VS_OFF &&
+ (flags & RISCV_V_VCPU_CTX) &&
+ !(flags & (RISCV_V_VCPU_NEED_RESTORE |
+ RISCV_V_VCPU_CSR_RESTORE));
+ if (eligible) {
+ owner->identity = (unsigned long)vcpu;
+ owner->cookie = vcpu->arch.vector_owner_cookie;
+ owner->generation = vcpu->arch.vector_state_generation;
+ owner->vcpu_id = vcpu->vcpu_id;
+ owner->guest_vs = guest_vs;
+ reason = KVM_RISCV_VECTOR_OWNER_PUT_CANDIDATE;
+ } else {
+ owner->identity = 0;
+ owner->cookie = 0;
+ owner->generation = 0;
+ owner->vcpu_id = 0;
+ owner->guest_vs = 0;
+ if (guest_vs == SR_VS_OFF)
+ reason = KVM_RISCV_VECTOR_OWNER_GUEST_OFF;
+ else if (host_vs != SR_VS_OFF)
+ reason = KVM_RISCV_VECTOR_OWNER_HOST_ON;
+ else
+ reason = KVM_RISCV_VECTOR_OWNER_NOT_RESIDENT;
+ }
+
+ trace_kvm_riscv_vector_owner(vcpu->vcpu_id, smp_processor_id(), 1,
+ eligible, reason, host_vs, guest_vs,
+ vcpu->arch.vector_state_generation);
+
+ return eligible;
+}
+
+void kvm_riscv_vcpu_vector_owner_invalidate(enum riscv_v_invalidate_reason reason)
+{
+ struct kvm_riscv_vector_owner *owner;
+ struct kvm_vcpu *vcpu;
+ enum kvm_riscv_vector_owner_reason trace_reason;
+ unsigned long host_vs;
+ u32 flags;
+
+ switch (reason) {
+ case RISCV_V_INVALIDATE_TASK_SAVE:
+ trace_reason = KVM_RISCV_VECTOR_OWNER_TASK_SAVE;
+ break;
+ case RISCV_V_INVALIDATE_TASK_RESTORE:
+ trace_reason = KVM_RISCV_VECTOR_OWNER_TASK_RESTORE;
+ break;
+ case RISCV_V_INVALIDATE_TASK_DISCARD:
+ trace_reason = KVM_RISCV_VECTOR_OWNER_TASK_DISCARD;
+ break;
+ case RISCV_V_INVALIDATE_KERNEL_VECTOR:
+ trace_reason = KVM_RISCV_VECTOR_OWNER_KERNEL_VECTOR;
+ break;
+ case RISCV_V_INVALIDATE_CPU_STATE:
+ trace_reason = KVM_RISCV_VECTOR_OWNER_CPU_STATE;
+ break;
+ default:
+ return;
+ }
+
+ preempt_disable();
+ flags = riscv_v_flags();
+ if (flags & RISCV_V_VCPU_CSR_RESTORE) {
+ vcpu = *this_cpu_ptr(kvm_get_running_vcpus());
+ if (!WARN_ON_ONCE(!vcpu)) {
+ flags &= ~RISCV_V_VCPU_CSR_RESTORE;
+ flags |= RISCV_V_VCPU_NEED_RESTORE;
+ riscv_v_flags_set(flags);
+ host_vs = task_pt_regs(current)->status & SR_VS;
+ trace_kvm_riscv_vector_owner(vcpu->vcpu_id,
+ smp_processor_id(), 2, false,
+ trace_reason, host_vs,
+ vcpu->arch.guest_context.sstatus & SR_VS,
+ vcpu->arch.vector_state_generation);
+ }
+ }
+
+ owner = this_cpu_ptr(&kvm_riscv_vector_owner);
+ if (!owner->identity)
+ goto out;
+
+ host_vs = task_pt_regs(current)->status & SR_VS;
+ trace_kvm_riscv_vector_owner(owner->vcpu_id, smp_processor_id(), 2,
+ false, trace_reason, host_vs,
+ owner->guest_vs, owner->generation);
+ owner->identity = 0;
+ owner->cookie = 0;
+ owner->generation = 0;
+ owner->vcpu_id = 0;
+ owner->guest_vs = 0;
+
+out:
+ preempt_enable();
}
int kvm_riscv_vcpu_alloc_vector_context(struct kvm_vcpu *vcpu)
@@ -88,6 +308,10 @@ int kvm_riscv_vcpu_alloc_vector_context(struct kvm_vcpu *vcpu)
return -ENOMEM;
}
+ vcpu->arch.vector_owner_cookie =
+ atomic64_inc_return(&kvm_riscv_vector_owner_cookie);
+ vcpu->arch.vector_state_generation = 0;
+
return 0;
}
@@ -110,8 +334,13 @@ void kvm_riscv_vcpu_flush_vector(void)
kvm_riscv_vcpu_guest_vector_save(&vcpu->arch.guest_context, vcpu->arch.isa);
- if ((vcpu->arch.guest_context.sstatus & SR_VS) != SR_VS_OFF)
+ if ((vcpu->arch.guest_context.sstatus & SR_VS) != SR_VS_OFF) {
+ unsigned long host_vs = task_pt_regs(current)->status & SR_VS;
+
riscv_v_flags_set(riscv_v_flags() | RISCV_V_VCPU_NEED_RESTORE);
+ trace_kvm_riscv_vector_restore_request(vcpu->vcpu_id,
+ smp_processor_id(), 1, host_vs);
+ }
}
#endif
@@ -229,6 +458,13 @@ int kvm_riscv_vcpu_set_reg_vector(struct kvm_vcpu *vcpu,
if (rc)
return rc;
+ /*
+ * copy_from_user() can modify a prefix of the destination before it
+ * reports a fault. Invalidate any cached hardware copy first so a
+ * failed SET_ONE_REG cannot leave newer memory hidden behind a stale
+ * per-hart owner.
+ */
+ vcpu->arch.vector_state_generation++;
if (copy_from_user(reg_addr, uaddr, reg_size))
return -EFAULT;
--
2.43.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [RFC PATCH 2/3] RISC-V: KVM: Preserve guest Vector state across vCPU put/load
@ 2026-10-27 13:13 ` slack up
0 siblings, 0 replies; 15+ messages in thread
From: slack up @ 2026-10-27 13:13 UTC (permalink / raw)
To: Anup Patel, Atish Patra, Palmer Dabbelt, kvm-riscv, linux-riscv
Cc: Paul Walmsley, Albert Ou, Alexandre Ghiti, Paolo Bonzini,
Sean Christopherson, Shuah Khan, kvm, linux-kernel,
linux-kselftest, Xiaochun Yu
From: Xiaochun Yu <yxcnew@outlook.com>
kvm_arch_vcpu_put() saves the guest Vector state and restores the host
register file. A later load on the same hart saves the host again and
requests a full guest restore, even when the host has not used Vector in
between.
Track a per-hart owner with an opaque vCPU identity, an allocation cookie
and a state generation. After saving a dirty guest to memory, leave its
data registers resident when the host has no Vector restore obligation. A
matching load restores only the Vector control CSRs. All mismatches and
generic Vector users fall back to a full restore.
Keep guest memory authoritative by continuing to save at every dirty put.
Advance the generation on saves, resets and SET_ONE_REG. Do so before
copy_from_user() because a fault can leave a modified prefix in the
destination even though the ioctl fails.
Register the generic invalidation hook only when the read-only
vector_owner_fastpath module parameter is enabled. The parameter defaults
to false so the existing full-restore behavior remains the default. Add
tracepoints for restore mode, owner decisions and invalidation reasons.
Development note: OpenAI Codex assisted with source analysis,
implementation drafting, test orchestration and changelog preparation.
Signed-off-by: Xiaochun Yu <yxcnew@outlook.com>
---
arch/riscv/include/asm/kvm_host.h | 4 +
arch/riscv/include/asm/kvm_vcpu_vector.h | 54 ++++-
arch/riscv/include/asm/processor.h | 1 +
arch/riscv/kvm/trace.h | 98 +++++++++
arch/riscv/kvm/vcpu.c | 54 ++++-
arch/riscv/kvm/vcpu_vector.c | 252 ++++++++++++++++++++++-
6 files changed, 439 insertions(+), 24 deletions(-)
diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
index e28f058a94d7..a6e2b3097644 100644
--- a/arch/riscv/include/asm/kvm_host.h
+++ b/arch/riscv/include/asm/kvm_host.h
@@ -204,6 +204,10 @@ struct kvm_vcpu_arch {
/* CPU context of Guest VCPU */
struct kvm_cpu_context guest_context;
+ /* Identity and version used by Vector owner tracking. */
+ u64 vector_owner_cookie;
+ u64 vector_state_generation;
+
/* CPU CSR context of Guest VCPU */
struct kvm_vcpu_csr guest_csr;
diff --git a/arch/riscv/include/asm/kvm_vcpu_vector.h b/arch/riscv/include/asm/kvm_vcpu_vector.h
index ebc619ed67dd..7f3eecf4217b 100644
--- a/arch/riscv/include/asm/kvm_vcpu_vector.h
+++ b/arch/riscv/include/asm/kvm_vcpu_vector.h
@@ -26,29 +26,47 @@ static __always_inline void __kvm_riscv_vector_restore(struct kvm_cpu_context *c
__riscv_v_vstate_restore(&context->vector, context->vector.datap);
}
+static __always_inline void __kvm_riscv_vector_restore_csrs(struct kvm_cpu_context *context)
+{
+ riscv_v_enable();
+ __vstate_csr_restore(&context->vector);
+ riscv_v_disable();
+}
+
void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu);
void kvm_riscv_vcpu_guest_vector_save(struct kvm_cpu_context *cntx,
unsigned long *isa);
void kvm_riscv_vcpu_guest_vector_restore(struct kvm_cpu_context *cntx,
- unsigned long *isa);
+ unsigned long *isa, bool csr_only);
void kvm_riscv_vcpu_host_vector_save(struct kvm_cpu_context *cntx);
-void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx);
+void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx,
+ bool preserve_guest);
int kvm_riscv_vcpu_alloc_vector_context(struct kvm_vcpu *vcpu);
void kvm_riscv_vcpu_free_vector_context(struct kvm_vcpu *vcpu);
void kvm_riscv_register_vctx_callback(void (*func)(void));
void kvm_riscv_unregister_vctx_callback(void);
+bool kvm_riscv_vcpu_vector_owner_enabled(void);
+bool kvm_riscv_vcpu_vector_owner_load(struct kvm_vcpu *vcpu, int cpu);
+bool kvm_riscv_vcpu_vector_owner_put(struct kvm_vcpu *vcpu);
+void kvm_riscv_vcpu_vector_owner_invalidate(enum riscv_v_invalidate_reason reason);
void kvm_riscv_vcpu_flush_vector(void);
static inline void kvm_riscv_v_init(void)
{
- if (has_vector())
+ if (has_vector()) {
kvm_riscv_register_vctx_callback(&kvm_riscv_vcpu_flush_vector);
+ if (kvm_riscv_vcpu_vector_owner_enabled())
+ kvm_riscv_register_vctx_invalidate(&kvm_riscv_vcpu_vector_owner_invalidate);
+ }
}
static inline void kvm_riscv_v_exit(void)
{
- if (has_vector())
+ if (has_vector()) {
+ if (kvm_riscv_vcpu_vector_owner_enabled())
+ kvm_riscv_unregister_vctx_invalidate();
kvm_riscv_unregister_vctx_callback();
+ }
}
#else
@@ -59,6 +77,10 @@ static inline void __kvm_riscv_vector_restore(struct kvm_cpu_context *context)
{
}
+static inline void __kvm_riscv_vector_restore_csrs(struct kvm_cpu_context *context)
+{
+}
+
static inline void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu)
{
}
@@ -69,7 +91,7 @@ static inline void kvm_riscv_vcpu_guest_vector_save(struct kvm_cpu_context *cntx
}
static inline void kvm_riscv_vcpu_guest_vector_restore(struct kvm_cpu_context *cntx,
- unsigned long *isa)
+ unsigned long *isa, bool csr_only)
{
}
@@ -77,7 +99,8 @@ static inline void kvm_riscv_vcpu_host_vector_save(struct kvm_cpu_context *cntx)
{
}
-static inline void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx)
+static inline void
+kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx, bool preserve_guest)
{
}
@@ -90,6 +113,25 @@ static inline void kvm_riscv_vcpu_free_vector_context(struct kvm_vcpu *vcpu)
{
}
+static inline bool kvm_riscv_vcpu_vector_owner_enabled(void)
+{
+ return false;
+}
+
+static inline bool kvm_riscv_vcpu_vector_owner_load(struct kvm_vcpu *vcpu, int cpu)
+{
+ return false;
+}
+
+static inline bool kvm_riscv_vcpu_vector_owner_put(struct kvm_vcpu *vcpu)
+{
+ return false;
+}
+
+static inline void kvm_riscv_vcpu_vector_owner_invalidate(enum riscv_v_invalidate_reason reason)
+{
+}
+
static inline void kvm_riscv_v_init(void)
{
}
diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
index 815715c67f94..abda84ca9548 100644
--- a/arch/riscv/include/asm/processor.h
+++ b/arch/riscv/include/asm/processor.h
@@ -105,6 +105,7 @@ struct pt_regs;
#define RISCV_KERNEL_MODE_V 0x00000001
#define RISCV_V_VCPU_NEED_RESTORE 0x00000002
#define RISCV_V_VCPU_CTX 0x00000004
+#define RISCV_V_VCPU_CSR_RESTORE 0x00000008
#define RISCV_PREEMPT_V 0x00000100
#define RISCV_PREEMPT_V_DIRTY 0x80000000
#define RISCV_PREEMPT_V_NEED_RESTORE 0x40000000
diff --git a/arch/riscv/kvm/trace.h b/arch/riscv/kvm/trace.h
index 23a5fd1b686e..c805606c704d 100644
--- a/arch/riscv/kvm/trace.h
+++ b/arch/riscv/kvm/trace.h
@@ -133,6 +133,104 @@ TRACE_EVENT(kvm_vcpu_irq,
__entry->vcpu_id, __entry->irq, __entry->level)
);
+TRACE_EVENT(kvm_riscv_vector_restore_request,
+ TP_PROTO(unsigned long vcpu_id, int cpu, unsigned int source,
+ unsigned long host_vs),
+ TP_ARGS(vcpu_id, cpu, source, host_vs),
+
+ TP_STRUCT__entry(
+ __field(unsigned long, vcpu_id)
+ __field(int, cpu)
+ __field(unsigned int, source)
+ __field(unsigned long, host_vs)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_id = vcpu_id;
+ __entry->cpu = cpu;
+ __entry->source = source;
+ __entry->host_vs = host_vs;
+ ),
+
+ TP_printk("VCPU: %lu, CPU: %d, source: %s, host VS: 0x%lx",
+ __entry->vcpu_id, __entry->cpu,
+ __print_symbolic(__entry->source,
+ { 0, "load" }, { 1, "kernel_vector" }),
+ __entry->host_vs)
+);
+
+TRACE_EVENT(kvm_riscv_vector_restore,
+ TP_PROTO(unsigned long vcpu_id, int cpu, bool csr_only),
+ TP_ARGS(vcpu_id, cpu, csr_only),
+
+ TP_STRUCT__entry(
+ __field(unsigned long, vcpu_id)
+ __field(int, cpu)
+ __field(bool, csr_only)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_id = vcpu_id;
+ __entry->cpu = cpu;
+ __entry->csr_only = csr_only;
+ ),
+
+ TP_printk("VCPU: %lu, CPU: %d, mode: %s",
+ __entry->vcpu_id, __entry->cpu,
+ __entry->csr_only ? "csr" : "full")
+);
+
+TRACE_EVENT(kvm_riscv_vector_owner,
+ TP_PROTO(unsigned long vcpu_id, int cpu, unsigned int action,
+ bool eligible, unsigned int reason, unsigned long host_vs,
+ unsigned long guest_vs, u64 generation),
+ TP_ARGS(vcpu_id, cpu, action, eligible, reason, host_vs, guest_vs,
+ generation),
+
+ TP_STRUCT__entry(
+ __field(unsigned long, vcpu_id)
+ __field(int, cpu)
+ __field(unsigned int, action)
+ __field(bool, eligible)
+ __field(unsigned int, reason)
+ __field(unsigned long, host_vs)
+ __field(unsigned long, guest_vs)
+ __field(u64, generation)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_id = vcpu_id;
+ __entry->cpu = cpu;
+ __entry->action = action;
+ __entry->eligible = eligible;
+ __entry->reason = reason;
+ __entry->host_vs = host_vs;
+ __entry->guest_vs = guest_vs;
+ __entry->generation = generation;
+ ),
+
+ TP_printk("VCPU: %lu, CPU: %d, action: %s, eligible: %d, reason: %s, host VS: 0x%lx, guest VS: 0x%lx, generation: %llu",
+ __entry->vcpu_id, __entry->cpu,
+ __print_symbolic(__entry->action,
+ { 0, "load" }, { 1, "put" },
+ { 2, "invalidate" }),
+ __entry->eligible,
+ __print_symbolic(__entry->reason,
+ { 0, "hit" }, { 1, "empty" },
+ { 2, "other_vcpu" }, { 3, "cookie" },
+ { 4, "generation" }, { 5, "guest_off" },
+ { 6, "host_on" }, { 7, "migration" },
+ { 8, "put_candidate" },
+ { 9, "task_save" },
+ { 10, "task_restore" },
+ { 11, "task_discard" },
+ { 12, "kernel_vector" },
+ { 13, "cpu_state" },
+ { 14, "not_resident" }),
+ __entry->host_vs, __entry->guest_vs,
+ __entry->generation)
+);
+
#endif /* _TRACE_KVM_H */
#undef TRACE_INCLUDE_PATH
diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
index 7f1636c44d41..a59cac26ca7c 100644
--- a/arch/riscv/kvm/vcpu.c
+++ b/arch/riscv/kvm/vcpu.c
@@ -35,6 +35,7 @@ void kvm_riscv_clear_former_vcpu(void)
* via the fast-path optimization.
*/
__this_cpu_write(kvm_former_vcpu, NULL);
+ kvm_riscv_vcpu_vector_owner_invalidate(RISCV_V_INVALIDATE_CPU_STATE);
}
const struct kvm_stats_desc kvm_vcpu_stats_desc[] = {
@@ -589,6 +590,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
{
void *nsh;
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
+ bool vector_owner_hit;
/*
* If VCPU is being reloaded on the same physical CPU and no
@@ -647,9 +649,11 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
kvm_riscv_vcpu_guest_fp_restore(&vcpu->arch.guest_context,
vcpu->arch.isa);
get_cpu_vector_context();
- kvm_riscv_vcpu_host_vector_save(&vcpu->arch.host_context);
+ vector_owner_hit = kvm_riscv_vcpu_vector_owner_load(vcpu, cpu);
+ if (!vector_owner_hit)
+ kvm_riscv_vcpu_host_vector_save(&vcpu->arch.host_context);
kvm_riscv_vcpu_guest_vector_restore(&vcpu->arch.guest_context,
- vcpu->arch.isa);
+ vcpu->arch.isa, vector_owner_hit);
put_cpu_vector_context();
kvm_make_request(KVM_REQ_STEAL_UPDATE, vcpu);
@@ -661,6 +665,7 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
{
void *nsh;
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
+ bool preserve_guest;
vcpu->cpu = -1;
@@ -674,7 +679,9 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
get_cpu_vector_context();
kvm_riscv_vcpu_guest_vector_save(&vcpu->arch.guest_context,
vcpu->arch.isa);
- kvm_riscv_vcpu_host_vector_restore(&vcpu->arch.host_context);
+ preserve_guest = kvm_riscv_vcpu_vector_owner_put(vcpu);
+ kvm_riscv_vcpu_host_vector_restore(&vcpu->arch.host_context,
+ preserve_guest);
put_cpu_vector_context();
if (kvm_riscv_nacl_available()) {
@@ -793,6 +800,12 @@ static __always_inline void kvm_riscv_vcpu_swap_in_host_state(struct kvm_vcpu *v
zicficsr->ssp = csr_swap(CSR_SSP, 0);
}
+enum kvm_riscv_vector_restore_type {
+ KVM_RISCV_VECTOR_RESTORE_NONE,
+ KVM_RISCV_VECTOR_RESTORE_FULL,
+ KVM_RISCV_VECTOR_RESTORE_CSR,
+};
+
/*
* Actually run the vCPU, entering an RCU extended quiescent state (EQS) while
* the vCPU is running.
@@ -800,12 +813,14 @@ static __always_inline void kvm_riscv_vcpu_swap_in_host_state(struct kvm_vcpu *v
* This must be noinstr as instrumentation may make use of RCU, and this is not
* safe during the EQS.
*/
-static void noinstr kvm_riscv_vcpu_enter_exit(struct kvm_vcpu *vcpu,
- struct kvm_cpu_trap *trap)
+static int noinstr kvm_riscv_vcpu_enter_exit(struct kvm_vcpu *vcpu,
+ struct kvm_cpu_trap *trap)
{
void *nsh;
struct kvm_cpu_context *gcntx = &vcpu->arch.guest_context;
struct kvm_cpu_context *hcntx = &vcpu->arch.host_context;
+ int vector_restore = KVM_RISCV_VECTOR_RESTORE_NONE;
+ u32 vector_flags;
/*
* We save trap CSRs (such as SEPC, SCAUSE, STVAL, HTVAL, and
@@ -818,11 +833,21 @@ static void noinstr kvm_riscv_vcpu_enter_exit(struct kvm_vcpu *vcpu,
kvm_riscv_vcpu_swap_in_guest_state(vcpu);
guest_state_enter_irqoff();
- /* sstatus.VS != SR_VS_OFF is guaranteed when NEED_RESTORE is set */
- if (current->thread.riscv_v_flags & RISCV_V_VCPU_NEED_RESTORE) {
- current->thread.riscv_v_flags &= ~RISCV_V_VCPU_NEED_RESTORE;
+ vector_flags = current->thread.riscv_v_flags;
+ /* sstatus.VS != SR_VS_OFF is guaranteed for either restore request. */
+ if (vector_flags & (RISCV_V_VCPU_NEED_RESTORE |
+ RISCV_V_VCPU_CSR_RESTORE)) {
+ current->thread.riscv_v_flags &=
+ ~(RISCV_V_VCPU_NEED_RESTORE |
+ RISCV_V_VCPU_CSR_RESTORE);
current->thread.riscv_v_flags |= RISCV_V_VCPU_CTX;
- __kvm_riscv_vector_restore(gcntx);
+ if (vector_flags & RISCV_V_VCPU_NEED_RESTORE) {
+ __kvm_riscv_vector_restore(gcntx);
+ vector_restore = KVM_RISCV_VECTOR_RESTORE_FULL;
+ } else {
+ __kvm_riscv_vector_restore_csrs(gcntx);
+ vector_restore = KVM_RISCV_VECTOR_RESTORE_CSR;
+ }
gcntx->sstatus = (gcntx->sstatus & ~SR_VS) | SR_VS_CLEAN;
}
@@ -886,6 +911,8 @@ static void noinstr kvm_riscv_vcpu_enter_exit(struct kvm_vcpu *vcpu,
vcpu->arch.last_exit_cpu = vcpu->cpu;
guest_state_exit_irqoff();
kvm_riscv_vcpu_swap_in_host_state(vcpu);
+
+ return vector_restore;
}
int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
@@ -893,6 +920,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
int ret;
struct kvm_cpu_trap trap;
struct kvm_run *run = vcpu->run;
+ int vector_restore;
if (!vcpu->arch.ran_atleast_once)
kvm_riscv_vcpu_config_ran_once(vcpu);
@@ -1001,7 +1029,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
guest_timing_enter_irqoff();
- kvm_riscv_vcpu_enter_exit(vcpu, &trap);
+ vector_restore = kvm_riscv_vcpu_enter_exit(vcpu, &trap);
vcpu->mode = OUTSIDE_GUEST_MODE;
vcpu->stat.exits++;
@@ -1026,6 +1054,12 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
local_irq_enable();
+ if (vector_restore == KVM_RISCV_VECTOR_RESTORE_FULL)
+ trace_kvm_riscv_vector_restore(vcpu->vcpu_id,
+ smp_processor_id(), false);
+ else if (vector_restore == KVM_RISCV_VECTOR_RESTORE_CSR)
+ trace_kvm_riscv_vector_restore(vcpu->vcpu_id,
+ smp_processor_id(), true);
trace_kvm_exit(&trap);
preempt_enable();
diff --git a/arch/riscv/kvm/vcpu_vector.c b/arch/riscv/kvm/vcpu_vector.c
index fefc2857f753..c197d3d7b539 100644
--- a/arch/riscv/kvm/vcpu_vector.c
+++ b/arch/riscv/kvm/vcpu_vector.c
@@ -7,9 +7,11 @@
* Greentime Hu <greentime.hu@sifive.com>
*/
+#include <linux/atomic.h>
#include <linux/errno.h>
#include <linux/err.h>
#include <linux/kvm_host.h>
+#include <linux/moduleparam.h>
#include <linux/nospec.h>
#include <linux/uaccess.h>
#include <asm/cpufeature.h>
@@ -17,7 +19,50 @@
#include <asm/kvm_vcpu_vector.h>
#include <asm/vector.h>
+#include "trace.h"
+
#ifdef CONFIG_RISCV_ISA_V
+enum kvm_riscv_vector_owner_reason {
+ KVM_RISCV_VECTOR_OWNER_HIT,
+ KVM_RISCV_VECTOR_OWNER_EMPTY,
+ KVM_RISCV_VECTOR_OWNER_OTHER_VCPU,
+ KVM_RISCV_VECTOR_OWNER_COOKIE,
+ KVM_RISCV_VECTOR_OWNER_GENERATION,
+ KVM_RISCV_VECTOR_OWNER_GUEST_OFF,
+ KVM_RISCV_VECTOR_OWNER_HOST_ON,
+ KVM_RISCV_VECTOR_OWNER_MIGRATION,
+ KVM_RISCV_VECTOR_OWNER_PUT_CANDIDATE,
+ KVM_RISCV_VECTOR_OWNER_TASK_SAVE,
+ KVM_RISCV_VECTOR_OWNER_TASK_RESTORE,
+ KVM_RISCV_VECTOR_OWNER_TASK_DISCARD,
+ KVM_RISCV_VECTOR_OWNER_KERNEL_VECTOR,
+ KVM_RISCV_VECTOR_OWNER_CPU_STATE,
+ KVM_RISCV_VECTOR_OWNER_NOT_RESIDENT,
+};
+
+struct kvm_riscv_vector_owner {
+ /* Opaque comparison token; cookie rejects address reuse after destroy. */
+ unsigned long identity;
+ u64 cookie;
+ u64 generation;
+ unsigned long vcpu_id;
+ unsigned long guest_vs;
+};
+
+static DEFINE_PER_CPU(struct kvm_riscv_vector_owner,
+ kvm_riscv_vector_owner);
+static atomic64_t kvm_riscv_vector_owner_cookie = ATOMIC64_INIT(0);
+static bool kvm_riscv_vector_owner_fastpath;
+module_param_named(vector_owner_fastpath, kvm_riscv_vector_owner_fastpath,
+ bool, 0444);
+MODULE_PARM_DESC(vector_owner_fastpath,
+ "Preserve matched guest Vector registers across vCPU put/load");
+
+bool kvm_riscv_vcpu_vector_owner_enabled(void)
+{
+ return READ_ONCE(kvm_riscv_vector_owner_fastpath);
+}
+
void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu)
{
unsigned long *isa = vcpu->arch.isa;
@@ -34,6 +79,8 @@ void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu)
} else {
cntx->sstatus |= SR_VS_OFF;
}
+
+ vcpu->arch.vector_state_generation++;
}
static void kvm_riscv_vcpu_vector_clean(struct kvm_cpu_context *cntx)
@@ -46,18 +93,35 @@ void kvm_riscv_vcpu_guest_vector_save(struct kvm_cpu_context *cntx,
unsigned long *isa)
{
if ((cntx->sstatus & SR_VS) == SR_VS_DIRTY) {
- if (riscv_isa_extension_available(isa, V))
+ if (riscv_isa_extension_available(isa, V)) {
+ struct kvm_vcpu *vcpu = container_of(cntx, struct kvm_vcpu,
+ arch.guest_context);
+
__kvm_riscv_vector_save(cntx);
+ vcpu->arch.vector_state_generation++;
+ }
kvm_riscv_vcpu_vector_clean(cntx);
}
}
void kvm_riscv_vcpu_guest_vector_restore(struct kvm_cpu_context *cntx,
- unsigned long *isa)
+ unsigned long *isa, bool csr_only)
{
if ((cntx->sstatus & SR_VS) != SR_VS_OFF) {
- if (riscv_isa_extension_available(isa, V))
- riscv_v_flags_set(riscv_v_flags() | RISCV_V_VCPU_NEED_RESTORE);
+ if (riscv_isa_extension_available(isa, V)) {
+ struct kvm_vcpu *vcpu = container_of(cntx, struct kvm_vcpu,
+ arch.guest_context);
+ unsigned long host_vs = task_pt_regs(current)->status & SR_VS;
+ u32 flags = riscv_v_flags();
+
+ flags &= ~(RISCV_V_VCPU_NEED_RESTORE |
+ RISCV_V_VCPU_CSR_RESTORE);
+ flags |= csr_only ? RISCV_V_VCPU_CSR_RESTORE :
+ RISCV_V_VCPU_NEED_RESTORE;
+ riscv_v_flags_set(flags);
+ trace_kvm_riscv_vector_restore_request(vcpu->vcpu_id,
+ smp_processor_id(), 0, host_vs);
+ }
}
}
@@ -68,11 +132,167 @@ void kvm_riscv_vcpu_host_vector_save(struct kvm_cpu_context *cntx)
__kvm_riscv_vector_save(cntx);
}
-void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx)
+void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx,
+ bool preserve_guest)
{
- if (!kvm_riscv_isa_check_host(V))
+ if (!preserve_guest && !kvm_riscv_isa_check_host(V))
__kvm_riscv_vector_restore(cntx);
- riscv_v_flags_set(riscv_v_flags() & ~(RISCV_V_VCPU_CTX | RISCV_V_VCPU_NEED_RESTORE));
+ riscv_v_flags_set(riscv_v_flags() &
+ ~(RISCV_V_VCPU_CTX | RISCV_V_VCPU_NEED_RESTORE |
+ RISCV_V_VCPU_CSR_RESTORE));
+}
+
+bool kvm_riscv_vcpu_vector_owner_load(struct kvm_vcpu *vcpu, int cpu)
+{
+ struct kvm_riscv_vector_owner *owner;
+ unsigned long guest_vs = vcpu->arch.guest_context.sstatus & SR_VS;
+ unsigned long host_vs = task_pt_regs(current)->status & SR_VS;
+ enum kvm_riscv_vector_owner_reason reason;
+ bool eligible = false;
+
+ if (!kvm_riscv_vcpu_vector_owner_enabled())
+ return false;
+
+ owner = this_cpu_ptr(&kvm_riscv_vector_owner);
+ if (guest_vs == SR_VS_OFF) {
+ reason = KVM_RISCV_VECTOR_OWNER_GUEST_OFF;
+ } else if (host_vs != SR_VS_OFF) {
+ reason = KVM_RISCV_VECTOR_OWNER_HOST_ON;
+ } else if (!owner->identity) {
+ reason = vcpu->arch.last_exit_cpu == cpu ?
+ KVM_RISCV_VECTOR_OWNER_EMPTY :
+ KVM_RISCV_VECTOR_OWNER_MIGRATION;
+ } else if (owner->identity != (unsigned long)vcpu) {
+ reason = KVM_RISCV_VECTOR_OWNER_OTHER_VCPU;
+ } else if (owner->cookie != vcpu->arch.vector_owner_cookie) {
+ reason = KVM_RISCV_VECTOR_OWNER_COOKIE;
+ } else if (owner->generation != vcpu->arch.vector_state_generation) {
+ reason = KVM_RISCV_VECTOR_OWNER_GENERATION;
+ } else {
+ reason = KVM_RISCV_VECTOR_OWNER_HIT;
+ eligible = true;
+ }
+
+ trace_kvm_riscv_vector_owner(vcpu->vcpu_id, cpu, 0, eligible,
+ reason, host_vs, guest_vs,
+ vcpu->arch.vector_state_generation);
+
+ /* A load consumes the cached owner; the next put publishes it. */
+ owner->identity = 0;
+ owner->cookie = 0;
+ owner->generation = 0;
+ owner->vcpu_id = 0;
+ owner->guest_vs = 0;
+
+ return eligible;
+}
+
+bool kvm_riscv_vcpu_vector_owner_put(struct kvm_vcpu *vcpu)
+{
+ struct kvm_riscv_vector_owner *owner;
+ unsigned long guest_vs = vcpu->arch.guest_context.sstatus & SR_VS;
+ unsigned long host_vs = task_pt_regs(current)->status & SR_VS;
+ u32 flags = riscv_v_flags();
+ enum kvm_riscv_vector_owner_reason reason;
+ bool eligible;
+
+ if (!kvm_riscv_vcpu_vector_owner_enabled())
+ return false;
+
+ owner = this_cpu_ptr(&kvm_riscv_vector_owner);
+ eligible = guest_vs != SR_VS_OFF && host_vs == SR_VS_OFF &&
+ (flags & RISCV_V_VCPU_CTX) &&
+ !(flags & (RISCV_V_VCPU_NEED_RESTORE |
+ RISCV_V_VCPU_CSR_RESTORE));
+ if (eligible) {
+ owner->identity = (unsigned long)vcpu;
+ owner->cookie = vcpu->arch.vector_owner_cookie;
+ owner->generation = vcpu->arch.vector_state_generation;
+ owner->vcpu_id = vcpu->vcpu_id;
+ owner->guest_vs = guest_vs;
+ reason = KVM_RISCV_VECTOR_OWNER_PUT_CANDIDATE;
+ } else {
+ owner->identity = 0;
+ owner->cookie = 0;
+ owner->generation = 0;
+ owner->vcpu_id = 0;
+ owner->guest_vs = 0;
+ if (guest_vs == SR_VS_OFF)
+ reason = KVM_RISCV_VECTOR_OWNER_GUEST_OFF;
+ else if (host_vs != SR_VS_OFF)
+ reason = KVM_RISCV_VECTOR_OWNER_HOST_ON;
+ else
+ reason = KVM_RISCV_VECTOR_OWNER_NOT_RESIDENT;
+ }
+
+ trace_kvm_riscv_vector_owner(vcpu->vcpu_id, smp_processor_id(), 1,
+ eligible, reason, host_vs, guest_vs,
+ vcpu->arch.vector_state_generation);
+
+ return eligible;
+}
+
+void kvm_riscv_vcpu_vector_owner_invalidate(enum riscv_v_invalidate_reason reason)
+{
+ struct kvm_riscv_vector_owner *owner;
+ struct kvm_vcpu *vcpu;
+ enum kvm_riscv_vector_owner_reason trace_reason;
+ unsigned long host_vs;
+ u32 flags;
+
+ switch (reason) {
+ case RISCV_V_INVALIDATE_TASK_SAVE:
+ trace_reason = KVM_RISCV_VECTOR_OWNER_TASK_SAVE;
+ break;
+ case RISCV_V_INVALIDATE_TASK_RESTORE:
+ trace_reason = KVM_RISCV_VECTOR_OWNER_TASK_RESTORE;
+ break;
+ case RISCV_V_INVALIDATE_TASK_DISCARD:
+ trace_reason = KVM_RISCV_VECTOR_OWNER_TASK_DISCARD;
+ break;
+ case RISCV_V_INVALIDATE_KERNEL_VECTOR:
+ trace_reason = KVM_RISCV_VECTOR_OWNER_KERNEL_VECTOR;
+ break;
+ case RISCV_V_INVALIDATE_CPU_STATE:
+ trace_reason = KVM_RISCV_VECTOR_OWNER_CPU_STATE;
+ break;
+ default:
+ return;
+ }
+
+ preempt_disable();
+ flags = riscv_v_flags();
+ if (flags & RISCV_V_VCPU_CSR_RESTORE) {
+ vcpu = *this_cpu_ptr(kvm_get_running_vcpus());
+ if (!WARN_ON_ONCE(!vcpu)) {
+ flags &= ~RISCV_V_VCPU_CSR_RESTORE;
+ flags |= RISCV_V_VCPU_NEED_RESTORE;
+ riscv_v_flags_set(flags);
+ host_vs = task_pt_regs(current)->status & SR_VS;
+ trace_kvm_riscv_vector_owner(vcpu->vcpu_id,
+ smp_processor_id(), 2, false,
+ trace_reason, host_vs,
+ vcpu->arch.guest_context.sstatus & SR_VS,
+ vcpu->arch.vector_state_generation);
+ }
+ }
+
+ owner = this_cpu_ptr(&kvm_riscv_vector_owner);
+ if (!owner->identity)
+ goto out;
+
+ host_vs = task_pt_regs(current)->status & SR_VS;
+ trace_kvm_riscv_vector_owner(owner->vcpu_id, smp_processor_id(), 2,
+ false, trace_reason, host_vs,
+ owner->guest_vs, owner->generation);
+ owner->identity = 0;
+ owner->cookie = 0;
+ owner->generation = 0;
+ owner->vcpu_id = 0;
+ owner->guest_vs = 0;
+
+out:
+ preempt_enable();
}
int kvm_riscv_vcpu_alloc_vector_context(struct kvm_vcpu *vcpu)
@@ -88,6 +308,10 @@ int kvm_riscv_vcpu_alloc_vector_context(struct kvm_vcpu *vcpu)
return -ENOMEM;
}
+ vcpu->arch.vector_owner_cookie =
+ atomic64_inc_return(&kvm_riscv_vector_owner_cookie);
+ vcpu->arch.vector_state_generation = 0;
+
return 0;
}
@@ -110,8 +334,13 @@ void kvm_riscv_vcpu_flush_vector(void)
kvm_riscv_vcpu_guest_vector_save(&vcpu->arch.guest_context, vcpu->arch.isa);
- if ((vcpu->arch.guest_context.sstatus & SR_VS) != SR_VS_OFF)
+ if ((vcpu->arch.guest_context.sstatus & SR_VS) != SR_VS_OFF) {
+ unsigned long host_vs = task_pt_regs(current)->status & SR_VS;
+
riscv_v_flags_set(riscv_v_flags() | RISCV_V_VCPU_NEED_RESTORE);
+ trace_kvm_riscv_vector_restore_request(vcpu->vcpu_id,
+ smp_processor_id(), 1, host_vs);
+ }
}
#endif
@@ -229,6 +458,13 @@ int kvm_riscv_vcpu_set_reg_vector(struct kvm_vcpu *vcpu,
if (rc)
return rc;
+ /*
+ * copy_from_user() can modify a prefix of the destination before it
+ * reports a fault. Invalidate any cached hardware copy first so a
+ * failed SET_ONE_REG cannot leave newer memory hidden behind a stale
+ * per-hart owner.
+ */
+ vcpu->arch.vector_state_generation++;
if (copy_from_user(reg_addr, uaddr, reg_size))
return -EFAULT;
--
2.43.0
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [RFC PATCH 2/3] RISC-V: KVM: Preserve guest Vector state across vCPU put/load
@ 2026-10-27 13:13 ` slack up
0 siblings, 0 replies; 15+ messages in thread
From: slack up @ 2026-10-27 13:13 UTC (permalink / raw)
To: Anup Patel, Atish Patra, Palmer Dabbelt, kvm-riscv, linux-riscv
Cc: Paul Walmsley, Albert Ou, Alexandre Ghiti, Paolo Bonzini,
Sean Christopherson, Shuah Khan, kvm, linux-kernel,
linux-kselftest, Xiaochun Yu
From: Xiaochun Yu <yxcnew@outlook.com>
kvm_arch_vcpu_put() saves the guest Vector state and restores the host
register file. A later load on the same hart saves the host again and
requests a full guest restore, even when the host has not used Vector in
between.
Track a per-hart owner with an opaque vCPU identity, an allocation cookie
and a state generation. After saving a dirty guest to memory, leave its
data registers resident when the host has no Vector restore obligation. A
matching load restores only the Vector control CSRs. All mismatches and
generic Vector users fall back to a full restore.
Keep guest memory authoritative by continuing to save at every dirty put.
Advance the generation on saves, resets and SET_ONE_REG. Do so before
copy_from_user() because a fault can leave a modified prefix in the
destination even though the ioctl fails.
Register the generic invalidation hook only when the read-only
vector_owner_fastpath module parameter is enabled. The parameter defaults
to false so the existing full-restore behavior remains the default. Add
tracepoints for restore mode, owner decisions and invalidation reasons.
Development note: OpenAI Codex assisted with source analysis,
implementation drafting, test orchestration and changelog preparation.
Signed-off-by: Xiaochun Yu <yxcnew@outlook.com>
---
arch/riscv/include/asm/kvm_host.h | 4 +
arch/riscv/include/asm/kvm_vcpu_vector.h | 54 ++++-
arch/riscv/include/asm/processor.h | 1 +
arch/riscv/kvm/trace.h | 98 +++++++++
arch/riscv/kvm/vcpu.c | 54 ++++-
arch/riscv/kvm/vcpu_vector.c | 252 ++++++++++++++++++++++-
6 files changed, 439 insertions(+), 24 deletions(-)
diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
index e28f058a94d7..a6e2b3097644 100644
--- a/arch/riscv/include/asm/kvm_host.h
+++ b/arch/riscv/include/asm/kvm_host.h
@@ -204,6 +204,10 @@ struct kvm_vcpu_arch {
/* CPU context of Guest VCPU */
struct kvm_cpu_context guest_context;
+ /* Identity and version used by Vector owner tracking. */
+ u64 vector_owner_cookie;
+ u64 vector_state_generation;
+
/* CPU CSR context of Guest VCPU */
struct kvm_vcpu_csr guest_csr;
diff --git a/arch/riscv/include/asm/kvm_vcpu_vector.h b/arch/riscv/include/asm/kvm_vcpu_vector.h
index ebc619ed67dd..7f3eecf4217b 100644
--- a/arch/riscv/include/asm/kvm_vcpu_vector.h
+++ b/arch/riscv/include/asm/kvm_vcpu_vector.h
@@ -26,29 +26,47 @@ static __always_inline void __kvm_riscv_vector_restore(struct kvm_cpu_context *c
__riscv_v_vstate_restore(&context->vector, context->vector.datap);
}
+static __always_inline void __kvm_riscv_vector_restore_csrs(struct kvm_cpu_context *context)
+{
+ riscv_v_enable();
+ __vstate_csr_restore(&context->vector);
+ riscv_v_disable();
+}
+
void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu);
void kvm_riscv_vcpu_guest_vector_save(struct kvm_cpu_context *cntx,
unsigned long *isa);
void kvm_riscv_vcpu_guest_vector_restore(struct kvm_cpu_context *cntx,
- unsigned long *isa);
+ unsigned long *isa, bool csr_only);
void kvm_riscv_vcpu_host_vector_save(struct kvm_cpu_context *cntx);
-void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx);
+void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx,
+ bool preserve_guest);
int kvm_riscv_vcpu_alloc_vector_context(struct kvm_vcpu *vcpu);
void kvm_riscv_vcpu_free_vector_context(struct kvm_vcpu *vcpu);
void kvm_riscv_register_vctx_callback(void (*func)(void));
void kvm_riscv_unregister_vctx_callback(void);
+bool kvm_riscv_vcpu_vector_owner_enabled(void);
+bool kvm_riscv_vcpu_vector_owner_load(struct kvm_vcpu *vcpu, int cpu);
+bool kvm_riscv_vcpu_vector_owner_put(struct kvm_vcpu *vcpu);
+void kvm_riscv_vcpu_vector_owner_invalidate(enum riscv_v_invalidate_reason reason);
void kvm_riscv_vcpu_flush_vector(void);
static inline void kvm_riscv_v_init(void)
{
- if (has_vector())
+ if (has_vector()) {
kvm_riscv_register_vctx_callback(&kvm_riscv_vcpu_flush_vector);
+ if (kvm_riscv_vcpu_vector_owner_enabled())
+ kvm_riscv_register_vctx_invalidate(&kvm_riscv_vcpu_vector_owner_invalidate);
+ }
}
static inline void kvm_riscv_v_exit(void)
{
- if (has_vector())
+ if (has_vector()) {
+ if (kvm_riscv_vcpu_vector_owner_enabled())
+ kvm_riscv_unregister_vctx_invalidate();
kvm_riscv_unregister_vctx_callback();
+ }
}
#else
@@ -59,6 +77,10 @@ static inline void __kvm_riscv_vector_restore(struct kvm_cpu_context *context)
{
}
+static inline void __kvm_riscv_vector_restore_csrs(struct kvm_cpu_context *context)
+{
+}
+
static inline void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu)
{
}
@@ -69,7 +91,7 @@ static inline void kvm_riscv_vcpu_guest_vector_save(struct kvm_cpu_context *cntx
}
static inline void kvm_riscv_vcpu_guest_vector_restore(struct kvm_cpu_context *cntx,
- unsigned long *isa)
+ unsigned long *isa, bool csr_only)
{
}
@@ -77,7 +99,8 @@ static inline void kvm_riscv_vcpu_host_vector_save(struct kvm_cpu_context *cntx)
{
}
-static inline void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx)
+static inline void
+kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx, bool preserve_guest)
{
}
@@ -90,6 +113,25 @@ static inline void kvm_riscv_vcpu_free_vector_context(struct kvm_vcpu *vcpu)
{
}
+static inline bool kvm_riscv_vcpu_vector_owner_enabled(void)
+{
+ return false;
+}
+
+static inline bool kvm_riscv_vcpu_vector_owner_load(struct kvm_vcpu *vcpu, int cpu)
+{
+ return false;
+}
+
+static inline bool kvm_riscv_vcpu_vector_owner_put(struct kvm_vcpu *vcpu)
+{
+ return false;
+}
+
+static inline void kvm_riscv_vcpu_vector_owner_invalidate(enum riscv_v_invalidate_reason reason)
+{
+}
+
static inline void kvm_riscv_v_init(void)
{
}
diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
index 815715c67f94..abda84ca9548 100644
--- a/arch/riscv/include/asm/processor.h
+++ b/arch/riscv/include/asm/processor.h
@@ -105,6 +105,7 @@ struct pt_regs;
#define RISCV_KERNEL_MODE_V 0x00000001
#define RISCV_V_VCPU_NEED_RESTORE 0x00000002
#define RISCV_V_VCPU_CTX 0x00000004
+#define RISCV_V_VCPU_CSR_RESTORE 0x00000008
#define RISCV_PREEMPT_V 0x00000100
#define RISCV_PREEMPT_V_DIRTY 0x80000000
#define RISCV_PREEMPT_V_NEED_RESTORE 0x40000000
diff --git a/arch/riscv/kvm/trace.h b/arch/riscv/kvm/trace.h
index 23a5fd1b686e..c805606c704d 100644
--- a/arch/riscv/kvm/trace.h
+++ b/arch/riscv/kvm/trace.h
@@ -133,6 +133,104 @@ TRACE_EVENT(kvm_vcpu_irq,
__entry->vcpu_id, __entry->irq, __entry->level)
);
+TRACE_EVENT(kvm_riscv_vector_restore_request,
+ TP_PROTO(unsigned long vcpu_id, int cpu, unsigned int source,
+ unsigned long host_vs),
+ TP_ARGS(vcpu_id, cpu, source, host_vs),
+
+ TP_STRUCT__entry(
+ __field(unsigned long, vcpu_id)
+ __field(int, cpu)
+ __field(unsigned int, source)
+ __field(unsigned long, host_vs)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_id = vcpu_id;
+ __entry->cpu = cpu;
+ __entry->source = source;
+ __entry->host_vs = host_vs;
+ ),
+
+ TP_printk("VCPU: %lu, CPU: %d, source: %s, host VS: 0x%lx",
+ __entry->vcpu_id, __entry->cpu,
+ __print_symbolic(__entry->source,
+ { 0, "load" }, { 1, "kernel_vector" }),
+ __entry->host_vs)
+);
+
+TRACE_EVENT(kvm_riscv_vector_restore,
+ TP_PROTO(unsigned long vcpu_id, int cpu, bool csr_only),
+ TP_ARGS(vcpu_id, cpu, csr_only),
+
+ TP_STRUCT__entry(
+ __field(unsigned long, vcpu_id)
+ __field(int, cpu)
+ __field(bool, csr_only)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_id = vcpu_id;
+ __entry->cpu = cpu;
+ __entry->csr_only = csr_only;
+ ),
+
+ TP_printk("VCPU: %lu, CPU: %d, mode: %s",
+ __entry->vcpu_id, __entry->cpu,
+ __entry->csr_only ? "csr" : "full")
+);
+
+TRACE_EVENT(kvm_riscv_vector_owner,
+ TP_PROTO(unsigned long vcpu_id, int cpu, unsigned int action,
+ bool eligible, unsigned int reason, unsigned long host_vs,
+ unsigned long guest_vs, u64 generation),
+ TP_ARGS(vcpu_id, cpu, action, eligible, reason, host_vs, guest_vs,
+ generation),
+
+ TP_STRUCT__entry(
+ __field(unsigned long, vcpu_id)
+ __field(int, cpu)
+ __field(unsigned int, action)
+ __field(bool, eligible)
+ __field(unsigned int, reason)
+ __field(unsigned long, host_vs)
+ __field(unsigned long, guest_vs)
+ __field(u64, generation)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_id = vcpu_id;
+ __entry->cpu = cpu;
+ __entry->action = action;
+ __entry->eligible = eligible;
+ __entry->reason = reason;
+ __entry->host_vs = host_vs;
+ __entry->guest_vs = guest_vs;
+ __entry->generation = generation;
+ ),
+
+ TP_printk("VCPU: %lu, CPU: %d, action: %s, eligible: %d, reason: %s, host VS: 0x%lx, guest VS: 0x%lx, generation: %llu",
+ __entry->vcpu_id, __entry->cpu,
+ __print_symbolic(__entry->action,
+ { 0, "load" }, { 1, "put" },
+ { 2, "invalidate" }),
+ __entry->eligible,
+ __print_symbolic(__entry->reason,
+ { 0, "hit" }, { 1, "empty" },
+ { 2, "other_vcpu" }, { 3, "cookie" },
+ { 4, "generation" }, { 5, "guest_off" },
+ { 6, "host_on" }, { 7, "migration" },
+ { 8, "put_candidate" },
+ { 9, "task_save" },
+ { 10, "task_restore" },
+ { 11, "task_discard" },
+ { 12, "kernel_vector" },
+ { 13, "cpu_state" },
+ { 14, "not_resident" }),
+ __entry->host_vs, __entry->guest_vs,
+ __entry->generation)
+);
+
#endif /* _TRACE_KVM_H */
#undef TRACE_INCLUDE_PATH
diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
index 7f1636c44d41..a59cac26ca7c 100644
--- a/arch/riscv/kvm/vcpu.c
+++ b/arch/riscv/kvm/vcpu.c
@@ -35,6 +35,7 @@ void kvm_riscv_clear_former_vcpu(void)
* via the fast-path optimization.
*/
__this_cpu_write(kvm_former_vcpu, NULL);
+ kvm_riscv_vcpu_vector_owner_invalidate(RISCV_V_INVALIDATE_CPU_STATE);
}
const struct kvm_stats_desc kvm_vcpu_stats_desc[] = {
@@ -589,6 +590,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
{
void *nsh;
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
+ bool vector_owner_hit;
/*
* If VCPU is being reloaded on the same physical CPU and no
@@ -647,9 +649,11 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
kvm_riscv_vcpu_guest_fp_restore(&vcpu->arch.guest_context,
vcpu->arch.isa);
get_cpu_vector_context();
- kvm_riscv_vcpu_host_vector_save(&vcpu->arch.host_context);
+ vector_owner_hit = kvm_riscv_vcpu_vector_owner_load(vcpu, cpu);
+ if (!vector_owner_hit)
+ kvm_riscv_vcpu_host_vector_save(&vcpu->arch.host_context);
kvm_riscv_vcpu_guest_vector_restore(&vcpu->arch.guest_context,
- vcpu->arch.isa);
+ vcpu->arch.isa, vector_owner_hit);
put_cpu_vector_context();
kvm_make_request(KVM_REQ_STEAL_UPDATE, vcpu);
@@ -661,6 +665,7 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
{
void *nsh;
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
+ bool preserve_guest;
vcpu->cpu = -1;
@@ -674,7 +679,9 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
get_cpu_vector_context();
kvm_riscv_vcpu_guest_vector_save(&vcpu->arch.guest_context,
vcpu->arch.isa);
- kvm_riscv_vcpu_host_vector_restore(&vcpu->arch.host_context);
+ preserve_guest = kvm_riscv_vcpu_vector_owner_put(vcpu);
+ kvm_riscv_vcpu_host_vector_restore(&vcpu->arch.host_context,
+ preserve_guest);
put_cpu_vector_context();
if (kvm_riscv_nacl_available()) {
@@ -793,6 +800,12 @@ static __always_inline void kvm_riscv_vcpu_swap_in_host_state(struct kvm_vcpu *v
zicficsr->ssp = csr_swap(CSR_SSP, 0);
}
+enum kvm_riscv_vector_restore_type {
+ KVM_RISCV_VECTOR_RESTORE_NONE,
+ KVM_RISCV_VECTOR_RESTORE_FULL,
+ KVM_RISCV_VECTOR_RESTORE_CSR,
+};
+
/*
* Actually run the vCPU, entering an RCU extended quiescent state (EQS) while
* the vCPU is running.
@@ -800,12 +813,14 @@ static __always_inline void kvm_riscv_vcpu_swap_in_host_state(struct kvm_vcpu *v
* This must be noinstr as instrumentation may make use of RCU, and this is not
* safe during the EQS.
*/
-static void noinstr kvm_riscv_vcpu_enter_exit(struct kvm_vcpu *vcpu,
- struct kvm_cpu_trap *trap)
+static int noinstr kvm_riscv_vcpu_enter_exit(struct kvm_vcpu *vcpu,
+ struct kvm_cpu_trap *trap)
{
void *nsh;
struct kvm_cpu_context *gcntx = &vcpu->arch.guest_context;
struct kvm_cpu_context *hcntx = &vcpu->arch.host_context;
+ int vector_restore = KVM_RISCV_VECTOR_RESTORE_NONE;
+ u32 vector_flags;
/*
* We save trap CSRs (such as SEPC, SCAUSE, STVAL, HTVAL, and
@@ -818,11 +833,21 @@ static void noinstr kvm_riscv_vcpu_enter_exit(struct kvm_vcpu *vcpu,
kvm_riscv_vcpu_swap_in_guest_state(vcpu);
guest_state_enter_irqoff();
- /* sstatus.VS != SR_VS_OFF is guaranteed when NEED_RESTORE is set */
- if (current->thread.riscv_v_flags & RISCV_V_VCPU_NEED_RESTORE) {
- current->thread.riscv_v_flags &= ~RISCV_V_VCPU_NEED_RESTORE;
+ vector_flags = current->thread.riscv_v_flags;
+ /* sstatus.VS != SR_VS_OFF is guaranteed for either restore request. */
+ if (vector_flags & (RISCV_V_VCPU_NEED_RESTORE |
+ RISCV_V_VCPU_CSR_RESTORE)) {
+ current->thread.riscv_v_flags &=
+ ~(RISCV_V_VCPU_NEED_RESTORE |
+ RISCV_V_VCPU_CSR_RESTORE);
current->thread.riscv_v_flags |= RISCV_V_VCPU_CTX;
- __kvm_riscv_vector_restore(gcntx);
+ if (vector_flags & RISCV_V_VCPU_NEED_RESTORE) {
+ __kvm_riscv_vector_restore(gcntx);
+ vector_restore = KVM_RISCV_VECTOR_RESTORE_FULL;
+ } else {
+ __kvm_riscv_vector_restore_csrs(gcntx);
+ vector_restore = KVM_RISCV_VECTOR_RESTORE_CSR;
+ }
gcntx->sstatus = (gcntx->sstatus & ~SR_VS) | SR_VS_CLEAN;
}
@@ -886,6 +911,8 @@ static void noinstr kvm_riscv_vcpu_enter_exit(struct kvm_vcpu *vcpu,
vcpu->arch.last_exit_cpu = vcpu->cpu;
guest_state_exit_irqoff();
kvm_riscv_vcpu_swap_in_host_state(vcpu);
+
+ return vector_restore;
}
int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
@@ -893,6 +920,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
int ret;
struct kvm_cpu_trap trap;
struct kvm_run *run = vcpu->run;
+ int vector_restore;
if (!vcpu->arch.ran_atleast_once)
kvm_riscv_vcpu_config_ran_once(vcpu);
@@ -1001,7 +1029,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
guest_timing_enter_irqoff();
- kvm_riscv_vcpu_enter_exit(vcpu, &trap);
+ vector_restore = kvm_riscv_vcpu_enter_exit(vcpu, &trap);
vcpu->mode = OUTSIDE_GUEST_MODE;
vcpu->stat.exits++;
@@ -1026,6 +1054,12 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
local_irq_enable();
+ if (vector_restore == KVM_RISCV_VECTOR_RESTORE_FULL)
+ trace_kvm_riscv_vector_restore(vcpu->vcpu_id,
+ smp_processor_id(), false);
+ else if (vector_restore == KVM_RISCV_VECTOR_RESTORE_CSR)
+ trace_kvm_riscv_vector_restore(vcpu->vcpu_id,
+ smp_processor_id(), true);
trace_kvm_exit(&trap);
preempt_enable();
diff --git a/arch/riscv/kvm/vcpu_vector.c b/arch/riscv/kvm/vcpu_vector.c
index fefc2857f753..c197d3d7b539 100644
--- a/arch/riscv/kvm/vcpu_vector.c
+++ b/arch/riscv/kvm/vcpu_vector.c
@@ -7,9 +7,11 @@
* Greentime Hu <greentime.hu@sifive.com>
*/
+#include <linux/atomic.h>
#include <linux/errno.h>
#include <linux/err.h>
#include <linux/kvm_host.h>
+#include <linux/moduleparam.h>
#include <linux/nospec.h>
#include <linux/uaccess.h>
#include <asm/cpufeature.h>
@@ -17,7 +19,50 @@
#include <asm/kvm_vcpu_vector.h>
#include <asm/vector.h>
+#include "trace.h"
+
#ifdef CONFIG_RISCV_ISA_V
+enum kvm_riscv_vector_owner_reason {
+ KVM_RISCV_VECTOR_OWNER_HIT,
+ KVM_RISCV_VECTOR_OWNER_EMPTY,
+ KVM_RISCV_VECTOR_OWNER_OTHER_VCPU,
+ KVM_RISCV_VECTOR_OWNER_COOKIE,
+ KVM_RISCV_VECTOR_OWNER_GENERATION,
+ KVM_RISCV_VECTOR_OWNER_GUEST_OFF,
+ KVM_RISCV_VECTOR_OWNER_HOST_ON,
+ KVM_RISCV_VECTOR_OWNER_MIGRATION,
+ KVM_RISCV_VECTOR_OWNER_PUT_CANDIDATE,
+ KVM_RISCV_VECTOR_OWNER_TASK_SAVE,
+ KVM_RISCV_VECTOR_OWNER_TASK_RESTORE,
+ KVM_RISCV_VECTOR_OWNER_TASK_DISCARD,
+ KVM_RISCV_VECTOR_OWNER_KERNEL_VECTOR,
+ KVM_RISCV_VECTOR_OWNER_CPU_STATE,
+ KVM_RISCV_VECTOR_OWNER_NOT_RESIDENT,
+};
+
+struct kvm_riscv_vector_owner {
+ /* Opaque comparison token; cookie rejects address reuse after destroy. */
+ unsigned long identity;
+ u64 cookie;
+ u64 generation;
+ unsigned long vcpu_id;
+ unsigned long guest_vs;
+};
+
+static DEFINE_PER_CPU(struct kvm_riscv_vector_owner,
+ kvm_riscv_vector_owner);
+static atomic64_t kvm_riscv_vector_owner_cookie = ATOMIC64_INIT(0);
+static bool kvm_riscv_vector_owner_fastpath;
+module_param_named(vector_owner_fastpath, kvm_riscv_vector_owner_fastpath,
+ bool, 0444);
+MODULE_PARM_DESC(vector_owner_fastpath,
+ "Preserve matched guest Vector registers across vCPU put/load");
+
+bool kvm_riscv_vcpu_vector_owner_enabled(void)
+{
+ return READ_ONCE(kvm_riscv_vector_owner_fastpath);
+}
+
void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu)
{
unsigned long *isa = vcpu->arch.isa;
@@ -34,6 +79,8 @@ void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu)
} else {
cntx->sstatus |= SR_VS_OFF;
}
+
+ vcpu->arch.vector_state_generation++;
}
static void kvm_riscv_vcpu_vector_clean(struct kvm_cpu_context *cntx)
@@ -46,18 +93,35 @@ void kvm_riscv_vcpu_guest_vector_save(struct kvm_cpu_context *cntx,
unsigned long *isa)
{
if ((cntx->sstatus & SR_VS) == SR_VS_DIRTY) {
- if (riscv_isa_extension_available(isa, V))
+ if (riscv_isa_extension_available(isa, V)) {
+ struct kvm_vcpu *vcpu = container_of(cntx, struct kvm_vcpu,
+ arch.guest_context);
+
__kvm_riscv_vector_save(cntx);
+ vcpu->arch.vector_state_generation++;
+ }
kvm_riscv_vcpu_vector_clean(cntx);
}
}
void kvm_riscv_vcpu_guest_vector_restore(struct kvm_cpu_context *cntx,
- unsigned long *isa)
+ unsigned long *isa, bool csr_only)
{
if ((cntx->sstatus & SR_VS) != SR_VS_OFF) {
- if (riscv_isa_extension_available(isa, V))
- riscv_v_flags_set(riscv_v_flags() | RISCV_V_VCPU_NEED_RESTORE);
+ if (riscv_isa_extension_available(isa, V)) {
+ struct kvm_vcpu *vcpu = container_of(cntx, struct kvm_vcpu,
+ arch.guest_context);
+ unsigned long host_vs = task_pt_regs(current)->status & SR_VS;
+ u32 flags = riscv_v_flags();
+
+ flags &= ~(RISCV_V_VCPU_NEED_RESTORE |
+ RISCV_V_VCPU_CSR_RESTORE);
+ flags |= csr_only ? RISCV_V_VCPU_CSR_RESTORE :
+ RISCV_V_VCPU_NEED_RESTORE;
+ riscv_v_flags_set(flags);
+ trace_kvm_riscv_vector_restore_request(vcpu->vcpu_id,
+ smp_processor_id(), 0, host_vs);
+ }
}
}
@@ -68,11 +132,167 @@ void kvm_riscv_vcpu_host_vector_save(struct kvm_cpu_context *cntx)
__kvm_riscv_vector_save(cntx);
}
-void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx)
+void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx,
+ bool preserve_guest)
{
- if (!kvm_riscv_isa_check_host(V))
+ if (!preserve_guest && !kvm_riscv_isa_check_host(V))
__kvm_riscv_vector_restore(cntx);
- riscv_v_flags_set(riscv_v_flags() & ~(RISCV_V_VCPU_CTX | RISCV_V_VCPU_NEED_RESTORE));
+ riscv_v_flags_set(riscv_v_flags() &
+ ~(RISCV_V_VCPU_CTX | RISCV_V_VCPU_NEED_RESTORE |
+ RISCV_V_VCPU_CSR_RESTORE));
+}
+
+bool kvm_riscv_vcpu_vector_owner_load(struct kvm_vcpu *vcpu, int cpu)
+{
+ struct kvm_riscv_vector_owner *owner;
+ unsigned long guest_vs = vcpu->arch.guest_context.sstatus & SR_VS;
+ unsigned long host_vs = task_pt_regs(current)->status & SR_VS;
+ enum kvm_riscv_vector_owner_reason reason;
+ bool eligible = false;
+
+ if (!kvm_riscv_vcpu_vector_owner_enabled())
+ return false;
+
+ owner = this_cpu_ptr(&kvm_riscv_vector_owner);
+ if (guest_vs == SR_VS_OFF) {
+ reason = KVM_RISCV_VECTOR_OWNER_GUEST_OFF;
+ } else if (host_vs != SR_VS_OFF) {
+ reason = KVM_RISCV_VECTOR_OWNER_HOST_ON;
+ } else if (!owner->identity) {
+ reason = vcpu->arch.last_exit_cpu == cpu ?
+ KVM_RISCV_VECTOR_OWNER_EMPTY :
+ KVM_RISCV_VECTOR_OWNER_MIGRATION;
+ } else if (owner->identity != (unsigned long)vcpu) {
+ reason = KVM_RISCV_VECTOR_OWNER_OTHER_VCPU;
+ } else if (owner->cookie != vcpu->arch.vector_owner_cookie) {
+ reason = KVM_RISCV_VECTOR_OWNER_COOKIE;
+ } else if (owner->generation != vcpu->arch.vector_state_generation) {
+ reason = KVM_RISCV_VECTOR_OWNER_GENERATION;
+ } else {
+ reason = KVM_RISCV_VECTOR_OWNER_HIT;
+ eligible = true;
+ }
+
+ trace_kvm_riscv_vector_owner(vcpu->vcpu_id, cpu, 0, eligible,
+ reason, host_vs, guest_vs,
+ vcpu->arch.vector_state_generation);
+
+ /* A load consumes the cached owner; the next put publishes it. */
+ owner->identity = 0;
+ owner->cookie = 0;
+ owner->generation = 0;
+ owner->vcpu_id = 0;
+ owner->guest_vs = 0;
+
+ return eligible;
+}
+
+bool kvm_riscv_vcpu_vector_owner_put(struct kvm_vcpu *vcpu)
+{
+ struct kvm_riscv_vector_owner *owner;
+ unsigned long guest_vs = vcpu->arch.guest_context.sstatus & SR_VS;
+ unsigned long host_vs = task_pt_regs(current)->status & SR_VS;
+ u32 flags = riscv_v_flags();
+ enum kvm_riscv_vector_owner_reason reason;
+ bool eligible;
+
+ if (!kvm_riscv_vcpu_vector_owner_enabled())
+ return false;
+
+ owner = this_cpu_ptr(&kvm_riscv_vector_owner);
+ eligible = guest_vs != SR_VS_OFF && host_vs == SR_VS_OFF &&
+ (flags & RISCV_V_VCPU_CTX) &&
+ !(flags & (RISCV_V_VCPU_NEED_RESTORE |
+ RISCV_V_VCPU_CSR_RESTORE));
+ if (eligible) {
+ owner->identity = (unsigned long)vcpu;
+ owner->cookie = vcpu->arch.vector_owner_cookie;
+ owner->generation = vcpu->arch.vector_state_generation;
+ owner->vcpu_id = vcpu->vcpu_id;
+ owner->guest_vs = guest_vs;
+ reason = KVM_RISCV_VECTOR_OWNER_PUT_CANDIDATE;
+ } else {
+ owner->identity = 0;
+ owner->cookie = 0;
+ owner->generation = 0;
+ owner->vcpu_id = 0;
+ owner->guest_vs = 0;
+ if (guest_vs == SR_VS_OFF)
+ reason = KVM_RISCV_VECTOR_OWNER_GUEST_OFF;
+ else if (host_vs != SR_VS_OFF)
+ reason = KVM_RISCV_VECTOR_OWNER_HOST_ON;
+ else
+ reason = KVM_RISCV_VECTOR_OWNER_NOT_RESIDENT;
+ }
+
+ trace_kvm_riscv_vector_owner(vcpu->vcpu_id, smp_processor_id(), 1,
+ eligible, reason, host_vs, guest_vs,
+ vcpu->arch.vector_state_generation);
+
+ return eligible;
+}
+
+void kvm_riscv_vcpu_vector_owner_invalidate(enum riscv_v_invalidate_reason reason)
+{
+ struct kvm_riscv_vector_owner *owner;
+ struct kvm_vcpu *vcpu;
+ enum kvm_riscv_vector_owner_reason trace_reason;
+ unsigned long host_vs;
+ u32 flags;
+
+ switch (reason) {
+ case RISCV_V_INVALIDATE_TASK_SAVE:
+ trace_reason = KVM_RISCV_VECTOR_OWNER_TASK_SAVE;
+ break;
+ case RISCV_V_INVALIDATE_TASK_RESTORE:
+ trace_reason = KVM_RISCV_VECTOR_OWNER_TASK_RESTORE;
+ break;
+ case RISCV_V_INVALIDATE_TASK_DISCARD:
+ trace_reason = KVM_RISCV_VECTOR_OWNER_TASK_DISCARD;
+ break;
+ case RISCV_V_INVALIDATE_KERNEL_VECTOR:
+ trace_reason = KVM_RISCV_VECTOR_OWNER_KERNEL_VECTOR;
+ break;
+ case RISCV_V_INVALIDATE_CPU_STATE:
+ trace_reason = KVM_RISCV_VECTOR_OWNER_CPU_STATE;
+ break;
+ default:
+ return;
+ }
+
+ preempt_disable();
+ flags = riscv_v_flags();
+ if (flags & RISCV_V_VCPU_CSR_RESTORE) {
+ vcpu = *this_cpu_ptr(kvm_get_running_vcpus());
+ if (!WARN_ON_ONCE(!vcpu)) {
+ flags &= ~RISCV_V_VCPU_CSR_RESTORE;
+ flags |= RISCV_V_VCPU_NEED_RESTORE;
+ riscv_v_flags_set(flags);
+ host_vs = task_pt_regs(current)->status & SR_VS;
+ trace_kvm_riscv_vector_owner(vcpu->vcpu_id,
+ smp_processor_id(), 2, false,
+ trace_reason, host_vs,
+ vcpu->arch.guest_context.sstatus & SR_VS,
+ vcpu->arch.vector_state_generation);
+ }
+ }
+
+ owner = this_cpu_ptr(&kvm_riscv_vector_owner);
+ if (!owner->identity)
+ goto out;
+
+ host_vs = task_pt_regs(current)->status & SR_VS;
+ trace_kvm_riscv_vector_owner(owner->vcpu_id, smp_processor_id(), 2,
+ false, trace_reason, host_vs,
+ owner->guest_vs, owner->generation);
+ owner->identity = 0;
+ owner->cookie = 0;
+ owner->generation = 0;
+ owner->vcpu_id = 0;
+ owner->guest_vs = 0;
+
+out:
+ preempt_enable();
}
int kvm_riscv_vcpu_alloc_vector_context(struct kvm_vcpu *vcpu)
@@ -88,6 +308,10 @@ int kvm_riscv_vcpu_alloc_vector_context(struct kvm_vcpu *vcpu)
return -ENOMEM;
}
+ vcpu->arch.vector_owner_cookie =
+ atomic64_inc_return(&kvm_riscv_vector_owner_cookie);
+ vcpu->arch.vector_state_generation = 0;
+
return 0;
}
@@ -110,8 +334,13 @@ void kvm_riscv_vcpu_flush_vector(void)
kvm_riscv_vcpu_guest_vector_save(&vcpu->arch.guest_context, vcpu->arch.isa);
- if ((vcpu->arch.guest_context.sstatus & SR_VS) != SR_VS_OFF)
+ if ((vcpu->arch.guest_context.sstatus & SR_VS) != SR_VS_OFF) {
+ unsigned long host_vs = task_pt_regs(current)->status & SR_VS;
+
riscv_v_flags_set(riscv_v_flags() | RISCV_V_VCPU_NEED_RESTORE);
+ trace_kvm_riscv_vector_restore_request(vcpu->vcpu_id,
+ smp_processor_id(), 1, host_vs);
+ }
}
#endif
@@ -229,6 +458,13 @@ int kvm_riscv_vcpu_set_reg_vector(struct kvm_vcpu *vcpu,
if (rc)
return rc;
+ /*
+ * copy_from_user() can modify a prefix of the destination before it
+ * reports a fault. Invalidate any cached hardware copy first so a
+ * failed SET_ONE_REG cannot leave newer memory hidden behind a stale
+ * per-hart owner.
+ */
+ vcpu->arch.vector_state_generation++;
if (copy_from_user(reg_addr, uaddr, reg_size))
return -EFAULT;
--
2.43.0
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [RFC PATCH 3/3] KVM: riscv: selftests: Add Vector owner regression tests
2026-10-27 13:13 ` slack up
(?)
@ 2026-10-27 13:13 ` slack up
-1 siblings, 0 replies; 15+ messages in thread
From: slack up @ 2026-10-27 13:13 UTC (permalink / raw)
To: Anup Patel, Atish Patra, Palmer Dabbelt, kvm-riscv, linux-riscv
Cc: Paul Walmsley, Albert Ou, Alexandre Ghiti, Paolo Bonzini,
Sean Christopherson, Shuah Khan, kvm, linux-kernel,
linux-kselftest, Xiaochun Yu
From: Xiaochun Yu <yxcnew@outlook.com>
Add focused tests for the correctness boundaries of a resident guest
Vector owner.
The lifetime test destroys a vCPU with resident Vector data before
creating a new vCPU and verifies that the new vCPU cannot inherit it. The
ONE_REG test checks that GET preserves guest data and SET becomes visible
on the next entry. The fault test crosses into a protected page to verify
that a partially completed SET_ONE_REG copy cannot leave stale hardware
state visible to the guest.
Pin each test to one host CPU so the same-hart owner path is deterministic
when the optional optimization is enabled. Size the Vector buffers for
the architectural maximum VLEN so the tests are not limited to the VLEN
implemented by the development platform.
Development note: OpenAI Codex assisted with source analysis, selftest
drafting, test orchestration and changelog preparation.
Signed-off-by: Xiaochun Yu <yxcnew@outlook.com>
---
tools/testing/selftests/kvm/Makefile.kvm | 3 +
.../kvm/riscv/vector_owner_fault_test.c | 147 ++++++++++++++
.../kvm/riscv/vector_owner_lifetime_test.c | 127 ++++++++++++
.../kvm/riscv/vector_owner_onereg_test.c | 191 ++++++++++++++++++
4 files changed, 468 insertions(+)
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_onereg_test.c
diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 3cb35a6963d5..50a1f247ebc3 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -236,6 +236,9 @@ TEST_GEN_PROGS_s390 += s390/irq_injection
TEST_GEN_PROGS_riscv = $(TEST_GEN_PROGS_COMMON)
TEST_GEN_PROGS_riscv += riscv/sbi_pmu_test
TEST_GEN_PROGS_riscv += riscv/ebreak_test
+TEST_GEN_PROGS_riscv += riscv/vector_owner_lifetime_test
+TEST_GEN_PROGS_riscv += riscv/vector_owner_onereg_test
+TEST_GEN_PROGS_riscv += riscv/vector_owner_fault_test
TEST_GEN_PROGS_riscv += access_tracking_perf_test
TEST_GEN_PROGS_riscv += arch_timer
TEST_GEN_PROGS_riscv += coalesced_io_test
diff --git a/tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c b/tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c
new file mode 100644
index 000000000000..a25dd21db77f
--- /dev/null
+++ b/tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c
@@ -0,0 +1,147 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Check that failed Vector SET_ONE_REG copies cannot leave a stale owner. */
+#include <errno.h>
+#include <stdio.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <unistd.h>
+
+#include <asm/ptrace.h>
+
+#include "kvm_syscalls.h"
+#include "kvm_util.h"
+#include "processor.h"
+#include "ucall_common.h"
+
+#define OLD_PATTERN 0x31
+#define NEW_PATTERN 0xa7
+
+static unsigned char source[RISCV_MAX_VLENB] __aligned(64);
+static unsigned char observed[RISCV_MAX_VLENB] __aligned(64);
+
+static void guest_code(void)
+{
+ unsigned long i, vlenb;
+
+ csr_set(CSR_SSTATUS, SR_VS_INITIAL);
+ vlenb = csr_read(CSR_VLENB);
+ GUEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB);
+ for (i = 0; i < vlenb; i++)
+ source[i] = OLD_PATTERN;
+
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vle8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(source), "r"(vlenb)
+ : "memory");
+ GUEST_SYNC(1);
+
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vse8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(observed), "r"(vlenb)
+ : "memory");
+ GUEST_DONE();
+}
+
+static void pin_to_one_cpu(void)
+{
+ cpu_set_t allowed, pinned;
+ int cpu;
+
+ kvm_sched_getaffinity(0, sizeof(allowed), &allowed);
+ for (cpu = 0; cpu < CPU_SETSIZE; cpu++) {
+ if (CPU_ISSET(cpu, &allowed))
+ break;
+ }
+ TEST_ASSERT(cpu < CPU_SETSIZE, "no CPU available");
+ CPU_ZERO(&pinned);
+ CPU_SET(cpu, &pinned);
+ kvm_sched_setaffinity(0, sizeof(pinned), &pinned);
+}
+
+static void run_to(struct kvm_vcpu *vcpu, u64 expected)
+{
+ struct ucall uc;
+ u64 cmd;
+
+ vcpu_run(vcpu);
+ cmd = get_ucall(vcpu, &uc);
+ if (cmd == UCALL_ABORT)
+ REPORT_GUEST_ASSERT(uc);
+ TEST_ASSERT_EQ(cmd, expected);
+ if (expected == UCALL_SYNC)
+ TEST_ASSERT_EQ(uc.args[1], 1);
+}
+
+int main(void)
+{
+ unsigned char expected[RISCV_MAX_VLENB] __aligned(64) = {};
+ struct kvm_vm *vm;
+ struct kvm_vcpu *vcpu;
+ struct kvm_one_reg reg;
+ unsigned char *mapping;
+ unsigned long i, vlenb;
+ u64 vlenb_id, v0_id, size;
+ long page_size;
+ bool changed = false;
+ int ret, saved_errno;
+
+ pin_to_one_cpu();
+ vm = vm_create_with_one_vcpu(&vcpu, guest_code);
+ TEST_REQUIRE(__vcpu_has_isa_ext(vcpu, KVM_RISCV_ISA_EXT_V));
+ vm_init_vector_tables(vm);
+ vcpu_init_vector_tables(vcpu);
+ vlenb_id = KVM_REG_RISCV | KVM_REG_RISCV_VECTOR |
+ KVM_REG_SIZE_ULONG | KVM_REG_RISCV_VECTOR_CSR_REG(vlenb);
+ ret = __vcpu_get_reg(vcpu, vlenb_id, &vlenb);
+ TEST_ASSERT(!ret, "GET vlenb failed: %d", ret);
+ TEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB &&
+ !(vlenb & (vlenb - 1)), "bad vlenb: %lu", vlenb);
+ size = (u64)__builtin_ctzl(vlenb) << KVM_REG_SIZE_SHIFT;
+ v0_id = KVM_REG_RISCV | KVM_REG_RISCV_VECTOR | size |
+ KVM_REG_RISCV_VECTOR_REG(0);
+
+ page_size = sysconf(_SC_PAGESIZE);
+ TEST_ASSERT(page_size > 0 && vlenb <= 2 * page_size, "bad page size");
+ mapping = mmap(NULL, 2 * page_size, PROT_READ | PROT_WRITE,
+ MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+ TEST_ASSERT(mapping != MAP_FAILED, "mmap failed: %d", errno);
+ memset(mapping, NEW_PATTERN, page_size);
+ ret = mprotect(mapping + page_size, page_size, PROT_NONE);
+ TEST_ASSERT(!ret, "mprotect failed: %d", errno);
+ reg.id = v0_id;
+ reg.addr = (u64)(mapping + page_size - vlenb / 2);
+
+ run_to(vcpu, UCALL_SYNC);
+ ret = __vcpu_ioctl(vcpu, KVM_SET_ONE_REG, ®);
+ saved_errno = errno;
+ TEST_ASSERT(ret == -1 && saved_errno == EFAULT,
+ "SET should fault: ret=%d errno=%d", ret, saved_errno);
+ ret = __vcpu_get_reg(vcpu, v0_id, expected);
+ TEST_ASSERT(!ret, "GET v0 after failed SET: %d", ret);
+
+ /* Compare with memory, without assuming an exact partial-copy length. */
+ run_to(vcpu, UCALL_DONE);
+ sync_global_from_guest(vm, observed);
+ for (i = 0; i < vlenb; i++)
+ changed |= expected[i] != OLD_PATTERN;
+ TEST_ASSERT(changed, "failed copy did not exercise a memory update");
+ for (i = 0; i < vlenb; i++)
+ TEST_ASSERT(observed[i] == expected[i],
+ "partial fault byte %lu: guest=%#x memory=%#x",
+ i, observed[i], expected[i]);
+
+ pr_info("Vector SET_ONE_REG partial fault test passed (vlenb=%lu)\n",
+ vlenb);
+ kvm_vm_free(vm);
+ ret = munmap(mapping, 2 * page_size);
+ TEST_ASSERT(!ret, "munmap failed: %d", errno);
+ return 0;
+}
diff --git a/tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c b/tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c
new file mode 100644
index 000000000000..c7b057b27cd0
--- /dev/null
+++ b/tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c
@@ -0,0 +1,127 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Validate that a stale per-hart Vector owner from a destroyed vCPU cannot
+ * be treated as the owner of a different vCPU.
+ */
+#include <stdio.h>
+
+#include <asm/ptrace.h>
+
+#include "kvm_syscalls.h"
+#include "kvm_util.h"
+#include "processor.h"
+#include "ucall_common.h"
+
+#define OLD_PATTERN 0x5d
+
+static unsigned char old_source[RISCV_MAX_VLENB] __aligned(64);
+static unsigned char new_observed[RISCV_MAX_VLENB] __aligned(64);
+
+static void vector_load(const void *source, unsigned long vlenb)
+{
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vle8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(source), "r"(vlenb)
+ : "memory");
+}
+
+static void vector_store(void *destination, unsigned long vlenb)
+{
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vse8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(destination), "r"(vlenb)
+ : "memory");
+}
+
+static void old_guest(void)
+{
+ unsigned long i, vlenb;
+
+ csr_set(CSR_SSTATUS, SR_VS_INITIAL);
+ vlenb = csr_read(CSR_VLENB);
+ GUEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB);
+ for (i = 0; i < vlenb; i++)
+ old_source[i] = OLD_PATTERN;
+ vector_load(old_source, vlenb);
+ GUEST_DONE();
+}
+
+static void new_guest(void)
+{
+ unsigned long i, vlenb;
+
+ csr_set(CSR_SSTATUS, SR_VS_INITIAL);
+ vlenb = csr_read(CSR_VLENB);
+ GUEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB);
+ vector_store(new_observed, vlenb);
+ for (i = 0; i < vlenb; i++)
+ GUEST_ASSERT_EQ(new_observed[i], 0);
+ GUEST_DONE();
+}
+
+static void pin_to_one_cpu(void)
+{
+ cpu_set_t allowed, pinned;
+ int cpu;
+
+ kvm_sched_getaffinity(0, sizeof(allowed), &allowed);
+ for (cpu = 0; cpu < CPU_SETSIZE; cpu++) {
+ if (CPU_ISSET(cpu, &allowed))
+ break;
+ }
+ TEST_ASSERT(cpu < CPU_SETSIZE, "no CPU available");
+
+ CPU_ZERO(&pinned);
+ CPU_SET(cpu, &pinned);
+ kvm_sched_setaffinity(0, sizeof(pinned), &pinned);
+}
+
+static void run_to_done(struct kvm_vcpu *vcpu)
+{
+ struct ucall uc;
+ u64 cmd;
+
+ vcpu_run(vcpu);
+ cmd = get_ucall(vcpu, &uc);
+ if (cmd == UCALL_ABORT)
+ REPORT_GUEST_ASSERT(uc);
+ TEST_ASSERT_EQ(cmd, UCALL_DONE);
+}
+
+static struct kvm_vm *create_vector_vm(struct kvm_vcpu **vcpu,
+ void (*guest_code)(void))
+{
+ struct kvm_vm *vm;
+
+ vm = vm_create_with_one_vcpu(vcpu, guest_code);
+ TEST_REQUIRE(__vcpu_has_isa_ext(*vcpu, KVM_RISCV_ISA_EXT_V));
+ vm_init_vector_tables(vm);
+ vcpu_init_vector_tables(*vcpu);
+ return vm;
+}
+
+int main(void)
+{
+ struct kvm_vm *old_vm, *new_vm;
+ struct kvm_vcpu *old_vcpu, *new_vcpu;
+
+ pin_to_one_cpu();
+ old_vm = create_vector_vm(&old_vcpu, old_guest);
+ run_to_done(old_vcpu);
+ kvm_vm_free(old_vm);
+
+ new_vm = create_vector_vm(&new_vcpu, new_guest);
+ run_to_done(new_vcpu);
+ kvm_vm_free(new_vm);
+
+ pr_info("Vector owner lifetime test passed\n");
+ return 0;
+}
diff --git a/tools/testing/selftests/kvm/riscv/vector_owner_onereg_test.c b/tools/testing/selftests/kvm/riscv/vector_owner_onereg_test.c
new file mode 100644
index 000000000000..a1bd83e4808b
--- /dev/null
+++ b/tools/testing/selftests/kvm/riscv/vector_owner_onereg_test.c
@@ -0,0 +1,191 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Validate RISC-V KVM Vector KVM_GET/SET_ONE_REG interactions while the
+ * vCPU repeatedly runs on one physical CPU.
+ */
+#include <stdio.h>
+#include <string.h>
+
+#include <asm/ptrace.h>
+
+#include "kvm_syscalls.h"
+#include "kvm_util.h"
+#include "processor.h"
+#include "ucall_common.h"
+
+#define GUEST_PATTERN 0x31
+#define SET_PATTERN 0xa7
+
+static unsigned char guest_source[RISCV_MAX_VLENB] __aligned(64);
+static unsigned char guest_after_get[RISCV_MAX_VLENB] __aligned(64);
+static unsigned char guest_after_set[RISCV_MAX_VLENB] __aligned(64);
+
+static void guest_vector_load(const void *source, unsigned long vlenb)
+{
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vle8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(source), "r"(vlenb)
+ : "memory");
+}
+
+static void guest_vector_store(void *destination, unsigned long vlenb)
+{
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vse8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(destination), "r"(vlenb)
+ : "memory");
+}
+
+static void guest_expect_pattern(const unsigned char *data,
+ unsigned long vlenb, unsigned char pattern)
+{
+ unsigned long i;
+
+ for (i = 0; i < vlenb; i++)
+ GUEST_ASSERT_EQ(data[i], pattern);
+}
+
+static void guest_code(void)
+{
+ unsigned long i, vlenb;
+
+ csr_set(CSR_SSTATUS, SR_VS_INITIAL);
+ vlenb = csr_read(CSR_VLENB);
+ GUEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB);
+
+ for (i = 0; i < vlenb; i++)
+ guest_source[i] = GUEST_PATTERN;
+
+ guest_vector_load(guest_source, vlenb);
+ GUEST_SYNC(1);
+
+ guest_vector_store(guest_after_get, vlenb);
+ guest_expect_pattern(guest_after_get, vlenb, GUEST_PATTERN);
+ GUEST_SYNC(2);
+
+ guest_vector_store(guest_after_set, vlenb);
+ guest_expect_pattern(guest_after_set, vlenb, SET_PATTERN);
+ GUEST_DONE();
+}
+
+static void pin_to_one_cpu(void)
+{
+ cpu_set_t allowed, pinned;
+ int cpu;
+
+ kvm_sched_getaffinity(0, sizeof(allowed), &allowed);
+ for (cpu = 0; cpu < CPU_SETSIZE; cpu++) {
+ if (CPU_ISSET(cpu, &allowed))
+ break;
+ }
+ TEST_ASSERT(cpu < CPU_SETSIZE, "no CPU available");
+
+ CPU_ZERO(&pinned);
+ CPU_SET(cpu, &pinned);
+ kvm_sched_setaffinity(0, sizeof(pinned), &pinned);
+}
+
+static void run_to_sync(struct kvm_vcpu *vcpu, unsigned long stage)
+{
+ struct ucall uc;
+ u64 cmd;
+
+ vcpu_run(vcpu);
+ cmd = get_ucall(vcpu, &uc);
+ if (cmd == UCALL_ABORT)
+ REPORT_GUEST_ASSERT(uc);
+ TEST_ASSERT_EQ(cmd, UCALL_SYNC);
+ TEST_ASSERT_EQ(uc.args[1], stage);
+}
+
+static void run_to_done(struct kvm_vcpu *vcpu)
+{
+ struct ucall uc;
+ u64 cmd;
+
+ vcpu_run(vcpu);
+ cmd = get_ucall(vcpu, &uc);
+ if (cmd == UCALL_ABORT)
+ REPORT_GUEST_ASSERT(uc);
+ TEST_ASSERT_EQ(cmd, UCALL_DONE);
+}
+
+static int set_one_reg_buffer(struct kvm_vcpu *vcpu, u64 id, void *buffer)
+{
+ struct kvm_one_reg reg = {
+ .id = id,
+ .addr = (u64)buffer,
+ };
+
+ return __vcpu_ioctl(vcpu, KVM_SET_ONE_REG, ®);
+}
+
+static void expect_pattern(const unsigned char *data, unsigned long length,
+ unsigned char pattern, const char *where)
+{
+ unsigned long i;
+
+ for (i = 0; i < length; i++)
+ TEST_ASSERT(data[i] == pattern,
+ "%s byte %lu: got %#x, expected %#x",
+ where, i, data[i], pattern);
+}
+
+int main(void)
+{
+ unsigned char get_buffer[RISCV_MAX_VLENB] __aligned(64) = {};
+ unsigned char set_buffer[RISCV_MAX_VLENB] __aligned(64);
+ struct kvm_vm *vm;
+ struct kvm_vcpu *vcpu;
+ unsigned long vlenb;
+ u64 vlenb_id, v0_id, size;
+ int ret;
+
+ pin_to_one_cpu();
+ vm = vm_create_with_one_vcpu(&vcpu, guest_code);
+ TEST_REQUIRE(__vcpu_has_isa_ext(vcpu, KVM_RISCV_ISA_EXT_V));
+
+ vm_init_vector_tables(vm);
+ vcpu_init_vector_tables(vcpu);
+
+ vlenb_id = KVM_REG_RISCV | KVM_REG_RISCV_VECTOR |
+ KVM_REG_SIZE_ULONG | KVM_REG_RISCV_VECTOR_CSR_REG(vlenb);
+ ret = __vcpu_get_reg(vcpu, vlenb_id, &vlenb);
+ TEST_ASSERT(!ret, "KVM_GET_ONE_REG(vlenb) failed: %d", ret);
+ TEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB &&
+ !(vlenb & (vlenb - 1)), "unsupported vlenb: %lu", vlenb);
+
+ size = (u64)__builtin_ctzl(vlenb) << KVM_REG_SIZE_SHIFT;
+ v0_id = KVM_REG_RISCV | KVM_REG_RISCV_VECTOR | size |
+ KVM_REG_RISCV_VECTOR_REG(0);
+
+ run_to_sync(vcpu, 1);
+ ret = __vcpu_get_reg(vcpu, v0_id, get_buffer);
+ TEST_ASSERT(!ret, "KVM_GET_ONE_REG(v0) failed: %d", ret);
+ expect_pattern(get_buffer, vlenb, GUEST_PATTERN, "GET v0");
+
+ run_to_sync(vcpu, 2);
+ memset(set_buffer, SET_PATTERN, vlenb);
+ ret = set_one_reg_buffer(vcpu, v0_id, set_buffer);
+ TEST_ASSERT(!ret, "KVM_SET_ONE_REG(v0) failed: %d", ret);
+
+ run_to_done(vcpu);
+ sync_global_from_guest(vm, guest_after_get);
+ sync_global_from_guest(vm, guest_after_set);
+ expect_pattern(guest_after_get, vlenb, GUEST_PATTERN,
+ "guest after GET");
+ expect_pattern(guest_after_set, vlenb, SET_PATTERN,
+ "guest after SET");
+
+ pr_info("Vector ONE_REG owner test passed (vlenb=%lu)\n", vlenb);
+ kvm_vm_free(vm);
+ return 0;
+}
--
2.43.0
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [RFC PATCH 3/3] KVM: riscv: selftests: Add Vector owner regression tests
@ 2026-10-27 13:13 ` slack up
0 siblings, 0 replies; 15+ messages in thread
From: slack up @ 2026-10-27 13:13 UTC (permalink / raw)
To: Anup Patel, Atish Patra, Palmer Dabbelt, kvm-riscv, linux-riscv
Cc: Paul Walmsley, Albert Ou, Alexandre Ghiti, Paolo Bonzini,
Sean Christopherson, Shuah Khan, kvm, linux-kernel,
linux-kselftest, Xiaochun Yu
From: Xiaochun Yu <yxcnew@outlook.com>
Add focused tests for the correctness boundaries of a resident guest
Vector owner.
The lifetime test destroys a vCPU with resident Vector data before
creating a new vCPU and verifies that the new vCPU cannot inherit it. The
ONE_REG test checks that GET preserves guest data and SET becomes visible
on the next entry. The fault test crosses into a protected page to verify
that a partially completed SET_ONE_REG copy cannot leave stale hardware
state visible to the guest.
Pin each test to one host CPU so the same-hart owner path is deterministic
when the optional optimization is enabled. Size the Vector buffers for
the architectural maximum VLEN so the tests are not limited to the VLEN
implemented by the development platform.
Development note: OpenAI Codex assisted with source analysis, selftest
drafting, test orchestration and changelog preparation.
Signed-off-by: Xiaochun Yu <yxcnew@outlook.com>
---
tools/testing/selftests/kvm/Makefile.kvm | 3 +
.../kvm/riscv/vector_owner_fault_test.c | 147 ++++++++++++++
.../kvm/riscv/vector_owner_lifetime_test.c | 127 ++++++++++++
.../kvm/riscv/vector_owner_onereg_test.c | 191 ++++++++++++++++++
4 files changed, 468 insertions(+)
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_onereg_test.c
diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 3cb35a6963d5..50a1f247ebc3 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -236,6 +236,9 @@ TEST_GEN_PROGS_s390 += s390/irq_injection
TEST_GEN_PROGS_riscv = $(TEST_GEN_PROGS_COMMON)
TEST_GEN_PROGS_riscv += riscv/sbi_pmu_test
TEST_GEN_PROGS_riscv += riscv/ebreak_test
+TEST_GEN_PROGS_riscv += riscv/vector_owner_lifetime_test
+TEST_GEN_PROGS_riscv += riscv/vector_owner_onereg_test
+TEST_GEN_PROGS_riscv += riscv/vector_owner_fault_test
TEST_GEN_PROGS_riscv += access_tracking_perf_test
TEST_GEN_PROGS_riscv += arch_timer
TEST_GEN_PROGS_riscv += coalesced_io_test
diff --git a/tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c b/tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c
new file mode 100644
index 000000000000..a25dd21db77f
--- /dev/null
+++ b/tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c
@@ -0,0 +1,147 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Check that failed Vector SET_ONE_REG copies cannot leave a stale owner. */
+#include <errno.h>
+#include <stdio.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <unistd.h>
+
+#include <asm/ptrace.h>
+
+#include "kvm_syscalls.h"
+#include "kvm_util.h"
+#include "processor.h"
+#include "ucall_common.h"
+
+#define OLD_PATTERN 0x31
+#define NEW_PATTERN 0xa7
+
+static unsigned char source[RISCV_MAX_VLENB] __aligned(64);
+static unsigned char observed[RISCV_MAX_VLENB] __aligned(64);
+
+static void guest_code(void)
+{
+ unsigned long i, vlenb;
+
+ csr_set(CSR_SSTATUS, SR_VS_INITIAL);
+ vlenb = csr_read(CSR_VLENB);
+ GUEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB);
+ for (i = 0; i < vlenb; i++)
+ source[i] = OLD_PATTERN;
+
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vle8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(source), "r"(vlenb)
+ : "memory");
+ GUEST_SYNC(1);
+
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vse8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(observed), "r"(vlenb)
+ : "memory");
+ GUEST_DONE();
+}
+
+static void pin_to_one_cpu(void)
+{
+ cpu_set_t allowed, pinned;
+ int cpu;
+
+ kvm_sched_getaffinity(0, sizeof(allowed), &allowed);
+ for (cpu = 0; cpu < CPU_SETSIZE; cpu++) {
+ if (CPU_ISSET(cpu, &allowed))
+ break;
+ }
+ TEST_ASSERT(cpu < CPU_SETSIZE, "no CPU available");
+ CPU_ZERO(&pinned);
+ CPU_SET(cpu, &pinned);
+ kvm_sched_setaffinity(0, sizeof(pinned), &pinned);
+}
+
+static void run_to(struct kvm_vcpu *vcpu, u64 expected)
+{
+ struct ucall uc;
+ u64 cmd;
+
+ vcpu_run(vcpu);
+ cmd = get_ucall(vcpu, &uc);
+ if (cmd == UCALL_ABORT)
+ REPORT_GUEST_ASSERT(uc);
+ TEST_ASSERT_EQ(cmd, expected);
+ if (expected == UCALL_SYNC)
+ TEST_ASSERT_EQ(uc.args[1], 1);
+}
+
+int main(void)
+{
+ unsigned char expected[RISCV_MAX_VLENB] __aligned(64) = {};
+ struct kvm_vm *vm;
+ struct kvm_vcpu *vcpu;
+ struct kvm_one_reg reg;
+ unsigned char *mapping;
+ unsigned long i, vlenb;
+ u64 vlenb_id, v0_id, size;
+ long page_size;
+ bool changed = false;
+ int ret, saved_errno;
+
+ pin_to_one_cpu();
+ vm = vm_create_with_one_vcpu(&vcpu, guest_code);
+ TEST_REQUIRE(__vcpu_has_isa_ext(vcpu, KVM_RISCV_ISA_EXT_V));
+ vm_init_vector_tables(vm);
+ vcpu_init_vector_tables(vcpu);
+ vlenb_id = KVM_REG_RISCV | KVM_REG_RISCV_VECTOR |
+ KVM_REG_SIZE_ULONG | KVM_REG_RISCV_VECTOR_CSR_REG(vlenb);
+ ret = __vcpu_get_reg(vcpu, vlenb_id, &vlenb);
+ TEST_ASSERT(!ret, "GET vlenb failed: %d", ret);
+ TEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB &&
+ !(vlenb & (vlenb - 1)), "bad vlenb: %lu", vlenb);
+ size = (u64)__builtin_ctzl(vlenb) << KVM_REG_SIZE_SHIFT;
+ v0_id = KVM_REG_RISCV | KVM_REG_RISCV_VECTOR | size |
+ KVM_REG_RISCV_VECTOR_REG(0);
+
+ page_size = sysconf(_SC_PAGESIZE);
+ TEST_ASSERT(page_size > 0 && vlenb <= 2 * page_size, "bad page size");
+ mapping = mmap(NULL, 2 * page_size, PROT_READ | PROT_WRITE,
+ MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+ TEST_ASSERT(mapping != MAP_FAILED, "mmap failed: %d", errno);
+ memset(mapping, NEW_PATTERN, page_size);
+ ret = mprotect(mapping + page_size, page_size, PROT_NONE);
+ TEST_ASSERT(!ret, "mprotect failed: %d", errno);
+ reg.id = v0_id;
+ reg.addr = (u64)(mapping + page_size - vlenb / 2);
+
+ run_to(vcpu, UCALL_SYNC);
+ ret = __vcpu_ioctl(vcpu, KVM_SET_ONE_REG, ®);
+ saved_errno = errno;
+ TEST_ASSERT(ret == -1 && saved_errno == EFAULT,
+ "SET should fault: ret=%d errno=%d", ret, saved_errno);
+ ret = __vcpu_get_reg(vcpu, v0_id, expected);
+ TEST_ASSERT(!ret, "GET v0 after failed SET: %d", ret);
+
+ /* Compare with memory, without assuming an exact partial-copy length. */
+ run_to(vcpu, UCALL_DONE);
+ sync_global_from_guest(vm, observed);
+ for (i = 0; i < vlenb; i++)
+ changed |= expected[i] != OLD_PATTERN;
+ TEST_ASSERT(changed, "failed copy did not exercise a memory update");
+ for (i = 0; i < vlenb; i++)
+ TEST_ASSERT(observed[i] == expected[i],
+ "partial fault byte %lu: guest=%#x memory=%#x",
+ i, observed[i], expected[i]);
+
+ pr_info("Vector SET_ONE_REG partial fault test passed (vlenb=%lu)\n",
+ vlenb);
+ kvm_vm_free(vm);
+ ret = munmap(mapping, 2 * page_size);
+ TEST_ASSERT(!ret, "munmap failed: %d", errno);
+ return 0;
+}
diff --git a/tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c b/tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c
new file mode 100644
index 000000000000..c7b057b27cd0
--- /dev/null
+++ b/tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c
@@ -0,0 +1,127 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Validate that a stale per-hart Vector owner from a destroyed vCPU cannot
+ * be treated as the owner of a different vCPU.
+ */
+#include <stdio.h>
+
+#include <asm/ptrace.h>
+
+#include "kvm_syscalls.h"
+#include "kvm_util.h"
+#include "processor.h"
+#include "ucall_common.h"
+
+#define OLD_PATTERN 0x5d
+
+static unsigned char old_source[RISCV_MAX_VLENB] __aligned(64);
+static unsigned char new_observed[RISCV_MAX_VLENB] __aligned(64);
+
+static void vector_load(const void *source, unsigned long vlenb)
+{
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vle8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(source), "r"(vlenb)
+ : "memory");
+}
+
+static void vector_store(void *destination, unsigned long vlenb)
+{
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vse8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(destination), "r"(vlenb)
+ : "memory");
+}
+
+static void old_guest(void)
+{
+ unsigned long i, vlenb;
+
+ csr_set(CSR_SSTATUS, SR_VS_INITIAL);
+ vlenb = csr_read(CSR_VLENB);
+ GUEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB);
+ for (i = 0; i < vlenb; i++)
+ old_source[i] = OLD_PATTERN;
+ vector_load(old_source, vlenb);
+ GUEST_DONE();
+}
+
+static void new_guest(void)
+{
+ unsigned long i, vlenb;
+
+ csr_set(CSR_SSTATUS, SR_VS_INITIAL);
+ vlenb = csr_read(CSR_VLENB);
+ GUEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB);
+ vector_store(new_observed, vlenb);
+ for (i = 0; i < vlenb; i++)
+ GUEST_ASSERT_EQ(new_observed[i], 0);
+ GUEST_DONE();
+}
+
+static void pin_to_one_cpu(void)
+{
+ cpu_set_t allowed, pinned;
+ int cpu;
+
+ kvm_sched_getaffinity(0, sizeof(allowed), &allowed);
+ for (cpu = 0; cpu < CPU_SETSIZE; cpu++) {
+ if (CPU_ISSET(cpu, &allowed))
+ break;
+ }
+ TEST_ASSERT(cpu < CPU_SETSIZE, "no CPU available");
+
+ CPU_ZERO(&pinned);
+ CPU_SET(cpu, &pinned);
+ kvm_sched_setaffinity(0, sizeof(pinned), &pinned);
+}
+
+static void run_to_done(struct kvm_vcpu *vcpu)
+{
+ struct ucall uc;
+ u64 cmd;
+
+ vcpu_run(vcpu);
+ cmd = get_ucall(vcpu, &uc);
+ if (cmd == UCALL_ABORT)
+ REPORT_GUEST_ASSERT(uc);
+ TEST_ASSERT_EQ(cmd, UCALL_DONE);
+}
+
+static struct kvm_vm *create_vector_vm(struct kvm_vcpu **vcpu,
+ void (*guest_code)(void))
+{
+ struct kvm_vm *vm;
+
+ vm = vm_create_with_one_vcpu(vcpu, guest_code);
+ TEST_REQUIRE(__vcpu_has_isa_ext(*vcpu, KVM_RISCV_ISA_EXT_V));
+ vm_init_vector_tables(vm);
+ vcpu_init_vector_tables(*vcpu);
+ return vm;
+}
+
+int main(void)
+{
+ struct kvm_vm *old_vm, *new_vm;
+ struct kvm_vcpu *old_vcpu, *new_vcpu;
+
+ pin_to_one_cpu();
+ old_vm = create_vector_vm(&old_vcpu, old_guest);
+ run_to_done(old_vcpu);
+ kvm_vm_free(old_vm);
+
+ new_vm = create_vector_vm(&new_vcpu, new_guest);
+ run_to_done(new_vcpu);
+ kvm_vm_free(new_vm);
+
+ pr_info("Vector owner lifetime test passed\n");
+ return 0;
+}
diff --git a/tools/testing/selftests/kvm/riscv/vector_owner_onereg_test.c b/tools/testing/selftests/kvm/riscv/vector_owner_onereg_test.c
new file mode 100644
index 000000000000..a1bd83e4808b
--- /dev/null
+++ b/tools/testing/selftests/kvm/riscv/vector_owner_onereg_test.c
@@ -0,0 +1,191 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Validate RISC-V KVM Vector KVM_GET/SET_ONE_REG interactions while the
+ * vCPU repeatedly runs on one physical CPU.
+ */
+#include <stdio.h>
+#include <string.h>
+
+#include <asm/ptrace.h>
+
+#include "kvm_syscalls.h"
+#include "kvm_util.h"
+#include "processor.h"
+#include "ucall_common.h"
+
+#define GUEST_PATTERN 0x31
+#define SET_PATTERN 0xa7
+
+static unsigned char guest_source[RISCV_MAX_VLENB] __aligned(64);
+static unsigned char guest_after_get[RISCV_MAX_VLENB] __aligned(64);
+static unsigned char guest_after_set[RISCV_MAX_VLENB] __aligned(64);
+
+static void guest_vector_load(const void *source, unsigned long vlenb)
+{
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vle8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(source), "r"(vlenb)
+ : "memory");
+}
+
+static void guest_vector_store(void *destination, unsigned long vlenb)
+{
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vse8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(destination), "r"(vlenb)
+ : "memory");
+}
+
+static void guest_expect_pattern(const unsigned char *data,
+ unsigned long vlenb, unsigned char pattern)
+{
+ unsigned long i;
+
+ for (i = 0; i < vlenb; i++)
+ GUEST_ASSERT_EQ(data[i], pattern);
+}
+
+static void guest_code(void)
+{
+ unsigned long i, vlenb;
+
+ csr_set(CSR_SSTATUS, SR_VS_INITIAL);
+ vlenb = csr_read(CSR_VLENB);
+ GUEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB);
+
+ for (i = 0; i < vlenb; i++)
+ guest_source[i] = GUEST_PATTERN;
+
+ guest_vector_load(guest_source, vlenb);
+ GUEST_SYNC(1);
+
+ guest_vector_store(guest_after_get, vlenb);
+ guest_expect_pattern(guest_after_get, vlenb, GUEST_PATTERN);
+ GUEST_SYNC(2);
+
+ guest_vector_store(guest_after_set, vlenb);
+ guest_expect_pattern(guest_after_set, vlenb, SET_PATTERN);
+ GUEST_DONE();
+}
+
+static void pin_to_one_cpu(void)
+{
+ cpu_set_t allowed, pinned;
+ int cpu;
+
+ kvm_sched_getaffinity(0, sizeof(allowed), &allowed);
+ for (cpu = 0; cpu < CPU_SETSIZE; cpu++) {
+ if (CPU_ISSET(cpu, &allowed))
+ break;
+ }
+ TEST_ASSERT(cpu < CPU_SETSIZE, "no CPU available");
+
+ CPU_ZERO(&pinned);
+ CPU_SET(cpu, &pinned);
+ kvm_sched_setaffinity(0, sizeof(pinned), &pinned);
+}
+
+static void run_to_sync(struct kvm_vcpu *vcpu, unsigned long stage)
+{
+ struct ucall uc;
+ u64 cmd;
+
+ vcpu_run(vcpu);
+ cmd = get_ucall(vcpu, &uc);
+ if (cmd == UCALL_ABORT)
+ REPORT_GUEST_ASSERT(uc);
+ TEST_ASSERT_EQ(cmd, UCALL_SYNC);
+ TEST_ASSERT_EQ(uc.args[1], stage);
+}
+
+static void run_to_done(struct kvm_vcpu *vcpu)
+{
+ struct ucall uc;
+ u64 cmd;
+
+ vcpu_run(vcpu);
+ cmd = get_ucall(vcpu, &uc);
+ if (cmd == UCALL_ABORT)
+ REPORT_GUEST_ASSERT(uc);
+ TEST_ASSERT_EQ(cmd, UCALL_DONE);
+}
+
+static int set_one_reg_buffer(struct kvm_vcpu *vcpu, u64 id, void *buffer)
+{
+ struct kvm_one_reg reg = {
+ .id = id,
+ .addr = (u64)buffer,
+ };
+
+ return __vcpu_ioctl(vcpu, KVM_SET_ONE_REG, ®);
+}
+
+static void expect_pattern(const unsigned char *data, unsigned long length,
+ unsigned char pattern, const char *where)
+{
+ unsigned long i;
+
+ for (i = 0; i < length; i++)
+ TEST_ASSERT(data[i] == pattern,
+ "%s byte %lu: got %#x, expected %#x",
+ where, i, data[i], pattern);
+}
+
+int main(void)
+{
+ unsigned char get_buffer[RISCV_MAX_VLENB] __aligned(64) = {};
+ unsigned char set_buffer[RISCV_MAX_VLENB] __aligned(64);
+ struct kvm_vm *vm;
+ struct kvm_vcpu *vcpu;
+ unsigned long vlenb;
+ u64 vlenb_id, v0_id, size;
+ int ret;
+
+ pin_to_one_cpu();
+ vm = vm_create_with_one_vcpu(&vcpu, guest_code);
+ TEST_REQUIRE(__vcpu_has_isa_ext(vcpu, KVM_RISCV_ISA_EXT_V));
+
+ vm_init_vector_tables(vm);
+ vcpu_init_vector_tables(vcpu);
+
+ vlenb_id = KVM_REG_RISCV | KVM_REG_RISCV_VECTOR |
+ KVM_REG_SIZE_ULONG | KVM_REG_RISCV_VECTOR_CSR_REG(vlenb);
+ ret = __vcpu_get_reg(vcpu, vlenb_id, &vlenb);
+ TEST_ASSERT(!ret, "KVM_GET_ONE_REG(vlenb) failed: %d", ret);
+ TEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB &&
+ !(vlenb & (vlenb - 1)), "unsupported vlenb: %lu", vlenb);
+
+ size = (u64)__builtin_ctzl(vlenb) << KVM_REG_SIZE_SHIFT;
+ v0_id = KVM_REG_RISCV | KVM_REG_RISCV_VECTOR | size |
+ KVM_REG_RISCV_VECTOR_REG(0);
+
+ run_to_sync(vcpu, 1);
+ ret = __vcpu_get_reg(vcpu, v0_id, get_buffer);
+ TEST_ASSERT(!ret, "KVM_GET_ONE_REG(v0) failed: %d", ret);
+ expect_pattern(get_buffer, vlenb, GUEST_PATTERN, "GET v0");
+
+ run_to_sync(vcpu, 2);
+ memset(set_buffer, SET_PATTERN, vlenb);
+ ret = set_one_reg_buffer(vcpu, v0_id, set_buffer);
+ TEST_ASSERT(!ret, "KVM_SET_ONE_REG(v0) failed: %d", ret);
+
+ run_to_done(vcpu);
+ sync_global_from_guest(vm, guest_after_get);
+ sync_global_from_guest(vm, guest_after_set);
+ expect_pattern(guest_after_get, vlenb, GUEST_PATTERN,
+ "guest after GET");
+ expect_pattern(guest_after_set, vlenb, SET_PATTERN,
+ "guest after SET");
+
+ pr_info("Vector ONE_REG owner test passed (vlenb=%lu)\n", vlenb);
+ kvm_vm_free(vm);
+ return 0;
+}
--
2.43.0
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [RFC PATCH 3/3] KVM: riscv: selftests: Add Vector owner regression tests
@ 2026-10-27 13:13 ` slack up
0 siblings, 0 replies; 15+ messages in thread
From: slack up @ 2026-10-27 13:13 UTC (permalink / raw)
To: Anup Patel, Atish Patra, Palmer Dabbelt, kvm-riscv, linux-riscv
Cc: Paul Walmsley, Albert Ou, Alexandre Ghiti, Paolo Bonzini,
Sean Christopherson, Shuah Khan, kvm, linux-kernel,
linux-kselftest, Xiaochun Yu
From: Xiaochun Yu <yxcnew@outlook.com>
Add focused tests for the correctness boundaries of a resident guest
Vector owner.
The lifetime test destroys a vCPU with resident Vector data before
creating a new vCPU and verifies that the new vCPU cannot inherit it. The
ONE_REG test checks that GET preserves guest data and SET becomes visible
on the next entry. The fault test crosses into a protected page to verify
that a partially completed SET_ONE_REG copy cannot leave stale hardware
state visible to the guest.
Pin each test to one host CPU so the same-hart owner path is deterministic
when the optional optimization is enabled. Size the Vector buffers for
the architectural maximum VLEN so the tests are not limited to the VLEN
implemented by the development platform.
Development note: OpenAI Codex assisted with source analysis, selftest
drafting, test orchestration and changelog preparation.
Signed-off-by: Xiaochun Yu <yxcnew@outlook.com>
---
tools/testing/selftests/kvm/Makefile.kvm | 3 +
.../kvm/riscv/vector_owner_fault_test.c | 147 ++++++++++++++
.../kvm/riscv/vector_owner_lifetime_test.c | 127 ++++++++++++
.../kvm/riscv/vector_owner_onereg_test.c | 191 ++++++++++++++++++
4 files changed, 468 insertions(+)
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c
create mode 100644 tools/testing/selftests/kvm/riscv/vector_owner_onereg_test.c
diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 3cb35a6963d5..50a1f247ebc3 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -236,6 +236,9 @@ TEST_GEN_PROGS_s390 += s390/irq_injection
TEST_GEN_PROGS_riscv = $(TEST_GEN_PROGS_COMMON)
TEST_GEN_PROGS_riscv += riscv/sbi_pmu_test
TEST_GEN_PROGS_riscv += riscv/ebreak_test
+TEST_GEN_PROGS_riscv += riscv/vector_owner_lifetime_test
+TEST_GEN_PROGS_riscv += riscv/vector_owner_onereg_test
+TEST_GEN_PROGS_riscv += riscv/vector_owner_fault_test
TEST_GEN_PROGS_riscv += access_tracking_perf_test
TEST_GEN_PROGS_riscv += arch_timer
TEST_GEN_PROGS_riscv += coalesced_io_test
diff --git a/tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c b/tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c
new file mode 100644
index 000000000000..a25dd21db77f
--- /dev/null
+++ b/tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c
@@ -0,0 +1,147 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Check that failed Vector SET_ONE_REG copies cannot leave a stale owner. */
+#include <errno.h>
+#include <stdio.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <unistd.h>
+
+#include <asm/ptrace.h>
+
+#include "kvm_syscalls.h"
+#include "kvm_util.h"
+#include "processor.h"
+#include "ucall_common.h"
+
+#define OLD_PATTERN 0x31
+#define NEW_PATTERN 0xa7
+
+static unsigned char source[RISCV_MAX_VLENB] __aligned(64);
+static unsigned char observed[RISCV_MAX_VLENB] __aligned(64);
+
+static void guest_code(void)
+{
+ unsigned long i, vlenb;
+
+ csr_set(CSR_SSTATUS, SR_VS_INITIAL);
+ vlenb = csr_read(CSR_VLENB);
+ GUEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB);
+ for (i = 0; i < vlenb; i++)
+ source[i] = OLD_PATTERN;
+
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vle8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(source), "r"(vlenb)
+ : "memory");
+ GUEST_SYNC(1);
+
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vse8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(observed), "r"(vlenb)
+ : "memory");
+ GUEST_DONE();
+}
+
+static void pin_to_one_cpu(void)
+{
+ cpu_set_t allowed, pinned;
+ int cpu;
+
+ kvm_sched_getaffinity(0, sizeof(allowed), &allowed);
+ for (cpu = 0; cpu < CPU_SETSIZE; cpu++) {
+ if (CPU_ISSET(cpu, &allowed))
+ break;
+ }
+ TEST_ASSERT(cpu < CPU_SETSIZE, "no CPU available");
+ CPU_ZERO(&pinned);
+ CPU_SET(cpu, &pinned);
+ kvm_sched_setaffinity(0, sizeof(pinned), &pinned);
+}
+
+static void run_to(struct kvm_vcpu *vcpu, u64 expected)
+{
+ struct ucall uc;
+ u64 cmd;
+
+ vcpu_run(vcpu);
+ cmd = get_ucall(vcpu, &uc);
+ if (cmd == UCALL_ABORT)
+ REPORT_GUEST_ASSERT(uc);
+ TEST_ASSERT_EQ(cmd, expected);
+ if (expected == UCALL_SYNC)
+ TEST_ASSERT_EQ(uc.args[1], 1);
+}
+
+int main(void)
+{
+ unsigned char expected[RISCV_MAX_VLENB] __aligned(64) = {};
+ struct kvm_vm *vm;
+ struct kvm_vcpu *vcpu;
+ struct kvm_one_reg reg;
+ unsigned char *mapping;
+ unsigned long i, vlenb;
+ u64 vlenb_id, v0_id, size;
+ long page_size;
+ bool changed = false;
+ int ret, saved_errno;
+
+ pin_to_one_cpu();
+ vm = vm_create_with_one_vcpu(&vcpu, guest_code);
+ TEST_REQUIRE(__vcpu_has_isa_ext(vcpu, KVM_RISCV_ISA_EXT_V));
+ vm_init_vector_tables(vm);
+ vcpu_init_vector_tables(vcpu);
+ vlenb_id = KVM_REG_RISCV | KVM_REG_RISCV_VECTOR |
+ KVM_REG_SIZE_ULONG | KVM_REG_RISCV_VECTOR_CSR_REG(vlenb);
+ ret = __vcpu_get_reg(vcpu, vlenb_id, &vlenb);
+ TEST_ASSERT(!ret, "GET vlenb failed: %d", ret);
+ TEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB &&
+ !(vlenb & (vlenb - 1)), "bad vlenb: %lu", vlenb);
+ size = (u64)__builtin_ctzl(vlenb) << KVM_REG_SIZE_SHIFT;
+ v0_id = KVM_REG_RISCV | KVM_REG_RISCV_VECTOR | size |
+ KVM_REG_RISCV_VECTOR_REG(0);
+
+ page_size = sysconf(_SC_PAGESIZE);
+ TEST_ASSERT(page_size > 0 && vlenb <= 2 * page_size, "bad page size");
+ mapping = mmap(NULL, 2 * page_size, PROT_READ | PROT_WRITE,
+ MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+ TEST_ASSERT(mapping != MAP_FAILED, "mmap failed: %d", errno);
+ memset(mapping, NEW_PATTERN, page_size);
+ ret = mprotect(mapping + page_size, page_size, PROT_NONE);
+ TEST_ASSERT(!ret, "mprotect failed: %d", errno);
+ reg.id = v0_id;
+ reg.addr = (u64)(mapping + page_size - vlenb / 2);
+
+ run_to(vcpu, UCALL_SYNC);
+ ret = __vcpu_ioctl(vcpu, KVM_SET_ONE_REG, ®);
+ saved_errno = errno;
+ TEST_ASSERT(ret == -1 && saved_errno == EFAULT,
+ "SET should fault: ret=%d errno=%d", ret, saved_errno);
+ ret = __vcpu_get_reg(vcpu, v0_id, expected);
+ TEST_ASSERT(!ret, "GET v0 after failed SET: %d", ret);
+
+ /* Compare with memory, without assuming an exact partial-copy length. */
+ run_to(vcpu, UCALL_DONE);
+ sync_global_from_guest(vm, observed);
+ for (i = 0; i < vlenb; i++)
+ changed |= expected[i] != OLD_PATTERN;
+ TEST_ASSERT(changed, "failed copy did not exercise a memory update");
+ for (i = 0; i < vlenb; i++)
+ TEST_ASSERT(observed[i] == expected[i],
+ "partial fault byte %lu: guest=%#x memory=%#x",
+ i, observed[i], expected[i]);
+
+ pr_info("Vector SET_ONE_REG partial fault test passed (vlenb=%lu)\n",
+ vlenb);
+ kvm_vm_free(vm);
+ ret = munmap(mapping, 2 * page_size);
+ TEST_ASSERT(!ret, "munmap failed: %d", errno);
+ return 0;
+}
diff --git a/tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c b/tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c
new file mode 100644
index 000000000000..c7b057b27cd0
--- /dev/null
+++ b/tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c
@@ -0,0 +1,127 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Validate that a stale per-hart Vector owner from a destroyed vCPU cannot
+ * be treated as the owner of a different vCPU.
+ */
+#include <stdio.h>
+
+#include <asm/ptrace.h>
+
+#include "kvm_syscalls.h"
+#include "kvm_util.h"
+#include "processor.h"
+#include "ucall_common.h"
+
+#define OLD_PATTERN 0x5d
+
+static unsigned char old_source[RISCV_MAX_VLENB] __aligned(64);
+static unsigned char new_observed[RISCV_MAX_VLENB] __aligned(64);
+
+static void vector_load(const void *source, unsigned long vlenb)
+{
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vle8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(source), "r"(vlenb)
+ : "memory");
+}
+
+static void vector_store(void *destination, unsigned long vlenb)
+{
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vse8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(destination), "r"(vlenb)
+ : "memory");
+}
+
+static void old_guest(void)
+{
+ unsigned long i, vlenb;
+
+ csr_set(CSR_SSTATUS, SR_VS_INITIAL);
+ vlenb = csr_read(CSR_VLENB);
+ GUEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB);
+ for (i = 0; i < vlenb; i++)
+ old_source[i] = OLD_PATTERN;
+ vector_load(old_source, vlenb);
+ GUEST_DONE();
+}
+
+static void new_guest(void)
+{
+ unsigned long i, vlenb;
+
+ csr_set(CSR_SSTATUS, SR_VS_INITIAL);
+ vlenb = csr_read(CSR_VLENB);
+ GUEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB);
+ vector_store(new_observed, vlenb);
+ for (i = 0; i < vlenb; i++)
+ GUEST_ASSERT_EQ(new_observed[i], 0);
+ GUEST_DONE();
+}
+
+static void pin_to_one_cpu(void)
+{
+ cpu_set_t allowed, pinned;
+ int cpu;
+
+ kvm_sched_getaffinity(0, sizeof(allowed), &allowed);
+ for (cpu = 0; cpu < CPU_SETSIZE; cpu++) {
+ if (CPU_ISSET(cpu, &allowed))
+ break;
+ }
+ TEST_ASSERT(cpu < CPU_SETSIZE, "no CPU available");
+
+ CPU_ZERO(&pinned);
+ CPU_SET(cpu, &pinned);
+ kvm_sched_setaffinity(0, sizeof(pinned), &pinned);
+}
+
+static void run_to_done(struct kvm_vcpu *vcpu)
+{
+ struct ucall uc;
+ u64 cmd;
+
+ vcpu_run(vcpu);
+ cmd = get_ucall(vcpu, &uc);
+ if (cmd == UCALL_ABORT)
+ REPORT_GUEST_ASSERT(uc);
+ TEST_ASSERT_EQ(cmd, UCALL_DONE);
+}
+
+static struct kvm_vm *create_vector_vm(struct kvm_vcpu **vcpu,
+ void (*guest_code)(void))
+{
+ struct kvm_vm *vm;
+
+ vm = vm_create_with_one_vcpu(vcpu, guest_code);
+ TEST_REQUIRE(__vcpu_has_isa_ext(*vcpu, KVM_RISCV_ISA_EXT_V));
+ vm_init_vector_tables(vm);
+ vcpu_init_vector_tables(*vcpu);
+ return vm;
+}
+
+int main(void)
+{
+ struct kvm_vm *old_vm, *new_vm;
+ struct kvm_vcpu *old_vcpu, *new_vcpu;
+
+ pin_to_one_cpu();
+ old_vm = create_vector_vm(&old_vcpu, old_guest);
+ run_to_done(old_vcpu);
+ kvm_vm_free(old_vm);
+
+ new_vm = create_vector_vm(&new_vcpu, new_guest);
+ run_to_done(new_vcpu);
+ kvm_vm_free(new_vm);
+
+ pr_info("Vector owner lifetime test passed\n");
+ return 0;
+}
diff --git a/tools/testing/selftests/kvm/riscv/vector_owner_onereg_test.c b/tools/testing/selftests/kvm/riscv/vector_owner_onereg_test.c
new file mode 100644
index 000000000000..a1bd83e4808b
--- /dev/null
+++ b/tools/testing/selftests/kvm/riscv/vector_owner_onereg_test.c
@@ -0,0 +1,191 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Validate RISC-V KVM Vector KVM_GET/SET_ONE_REG interactions while the
+ * vCPU repeatedly runs on one physical CPU.
+ */
+#include <stdio.h>
+#include <string.h>
+
+#include <asm/ptrace.h>
+
+#include "kvm_syscalls.h"
+#include "kvm_util.h"
+#include "processor.h"
+#include "ucall_common.h"
+
+#define GUEST_PATTERN 0x31
+#define SET_PATTERN 0xa7
+
+static unsigned char guest_source[RISCV_MAX_VLENB] __aligned(64);
+static unsigned char guest_after_get[RISCV_MAX_VLENB] __aligned(64);
+static unsigned char guest_after_set[RISCV_MAX_VLENB] __aligned(64);
+
+static void guest_vector_load(const void *source, unsigned long vlenb)
+{
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vle8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(source), "r"(vlenb)
+ : "memory");
+}
+
+static void guest_vector_store(void *destination, unsigned long vlenb)
+{
+ asm volatile(".option push\n"
+ ".option arch, +v\n"
+ "vsetvli zero, %1, e8, m1, ta, ma\n"
+ "vse8.v v0, (%0)\n"
+ ".option pop\n"
+ :
+ : "r"(destination), "r"(vlenb)
+ : "memory");
+}
+
+static void guest_expect_pattern(const unsigned char *data,
+ unsigned long vlenb, unsigned char pattern)
+{
+ unsigned long i;
+
+ for (i = 0; i < vlenb; i++)
+ GUEST_ASSERT_EQ(data[i], pattern);
+}
+
+static void guest_code(void)
+{
+ unsigned long i, vlenb;
+
+ csr_set(CSR_SSTATUS, SR_VS_INITIAL);
+ vlenb = csr_read(CSR_VLENB);
+ GUEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB);
+
+ for (i = 0; i < vlenb; i++)
+ guest_source[i] = GUEST_PATTERN;
+
+ guest_vector_load(guest_source, vlenb);
+ GUEST_SYNC(1);
+
+ guest_vector_store(guest_after_get, vlenb);
+ guest_expect_pattern(guest_after_get, vlenb, GUEST_PATTERN);
+ GUEST_SYNC(2);
+
+ guest_vector_store(guest_after_set, vlenb);
+ guest_expect_pattern(guest_after_set, vlenb, SET_PATTERN);
+ GUEST_DONE();
+}
+
+static void pin_to_one_cpu(void)
+{
+ cpu_set_t allowed, pinned;
+ int cpu;
+
+ kvm_sched_getaffinity(0, sizeof(allowed), &allowed);
+ for (cpu = 0; cpu < CPU_SETSIZE; cpu++) {
+ if (CPU_ISSET(cpu, &allowed))
+ break;
+ }
+ TEST_ASSERT(cpu < CPU_SETSIZE, "no CPU available");
+
+ CPU_ZERO(&pinned);
+ CPU_SET(cpu, &pinned);
+ kvm_sched_setaffinity(0, sizeof(pinned), &pinned);
+}
+
+static void run_to_sync(struct kvm_vcpu *vcpu, unsigned long stage)
+{
+ struct ucall uc;
+ u64 cmd;
+
+ vcpu_run(vcpu);
+ cmd = get_ucall(vcpu, &uc);
+ if (cmd == UCALL_ABORT)
+ REPORT_GUEST_ASSERT(uc);
+ TEST_ASSERT_EQ(cmd, UCALL_SYNC);
+ TEST_ASSERT_EQ(uc.args[1], stage);
+}
+
+static void run_to_done(struct kvm_vcpu *vcpu)
+{
+ struct ucall uc;
+ u64 cmd;
+
+ vcpu_run(vcpu);
+ cmd = get_ucall(vcpu, &uc);
+ if (cmd == UCALL_ABORT)
+ REPORT_GUEST_ASSERT(uc);
+ TEST_ASSERT_EQ(cmd, UCALL_DONE);
+}
+
+static int set_one_reg_buffer(struct kvm_vcpu *vcpu, u64 id, void *buffer)
+{
+ struct kvm_one_reg reg = {
+ .id = id,
+ .addr = (u64)buffer,
+ };
+
+ return __vcpu_ioctl(vcpu, KVM_SET_ONE_REG, ®);
+}
+
+static void expect_pattern(const unsigned char *data, unsigned long length,
+ unsigned char pattern, const char *where)
+{
+ unsigned long i;
+
+ for (i = 0; i < length; i++)
+ TEST_ASSERT(data[i] == pattern,
+ "%s byte %lu: got %#x, expected %#x",
+ where, i, data[i], pattern);
+}
+
+int main(void)
+{
+ unsigned char get_buffer[RISCV_MAX_VLENB] __aligned(64) = {};
+ unsigned char set_buffer[RISCV_MAX_VLENB] __aligned(64);
+ struct kvm_vm *vm;
+ struct kvm_vcpu *vcpu;
+ unsigned long vlenb;
+ u64 vlenb_id, v0_id, size;
+ int ret;
+
+ pin_to_one_cpu();
+ vm = vm_create_with_one_vcpu(&vcpu, guest_code);
+ TEST_REQUIRE(__vcpu_has_isa_ext(vcpu, KVM_RISCV_ISA_EXT_V));
+
+ vm_init_vector_tables(vm);
+ vcpu_init_vector_tables(vcpu);
+
+ vlenb_id = KVM_REG_RISCV | KVM_REG_RISCV_VECTOR |
+ KVM_REG_SIZE_ULONG | KVM_REG_RISCV_VECTOR_CSR_REG(vlenb);
+ ret = __vcpu_get_reg(vcpu, vlenb_id, &vlenb);
+ TEST_ASSERT(!ret, "KVM_GET_ONE_REG(vlenb) failed: %d", ret);
+ TEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB &&
+ !(vlenb & (vlenb - 1)), "unsupported vlenb: %lu", vlenb);
+
+ size = (u64)__builtin_ctzl(vlenb) << KVM_REG_SIZE_SHIFT;
+ v0_id = KVM_REG_RISCV | KVM_REG_RISCV_VECTOR | size |
+ KVM_REG_RISCV_VECTOR_REG(0);
+
+ run_to_sync(vcpu, 1);
+ ret = __vcpu_get_reg(vcpu, v0_id, get_buffer);
+ TEST_ASSERT(!ret, "KVM_GET_ONE_REG(v0) failed: %d", ret);
+ expect_pattern(get_buffer, vlenb, GUEST_PATTERN, "GET v0");
+
+ run_to_sync(vcpu, 2);
+ memset(set_buffer, SET_PATTERN, vlenb);
+ ret = set_one_reg_buffer(vcpu, v0_id, set_buffer);
+ TEST_ASSERT(!ret, "KVM_SET_ONE_REG(v0) failed: %d", ret);
+
+ run_to_done(vcpu);
+ sync_global_from_guest(vm, guest_after_get);
+ sync_global_from_guest(vm, guest_after_set);
+ expect_pattern(guest_after_get, vlenb, GUEST_PATTERN,
+ "guest after GET");
+ expect_pattern(guest_after_set, vlenb, SET_PATTERN,
+ "guest after SET");
+
+ pr_info("Vector ONE_REG owner test passed (vlenb=%lu)\n", vlenb);
+ kvm_vm_free(vm);
+ return 0;
+}
--
2.43.0
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [RFC PATCH 3/3] KVM: riscv: selftests: Add Vector owner regression tests
2026-10-27 13:13 ` slack up
(?)
(?)
@ 2026-10-27 14:03 ` sashiko-bot
-1 siblings, 0 replies; 15+ messages in thread
From: sashiko-bot @ 2026-10-27 14:03 UTC (permalink / raw)
To: slack up; +Cc: kvm
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Medium] KVM: riscv: selftests: prevent C calls from clobbering v0 in guest code
- [Medium] KVM: riscv: selftests: split Vector owner tests reimplement kvm_util helpers
- [Medium] KVM: riscv: selftests: vector_owner_lifetime_test clears owner before new_vm
--
commit cd1b458fe31c51e7e8df73a0ffb2811bb58b58bb
Author: Xiaochun Yu <yxcnew@outlook.com>
KVM: riscv: selftests: Add Vector owner regression tests
This commit adds three RISC-V KVM selftests covering guest Vector owner
lifetime across vCPU destruction, KVM_GET_ONE_REG and KVM_SET_ONE_REG state
synchronization, and partial user copy faults during KVM_SET_ONE_REG.
> diff --git a/tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c b/tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c
> new file mode 100644
> index 0000000000000..a25dd21db77f9
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/riscv/vector_owner_fault_test.c
> @@ -0,0 +1,147 @@
[ ... ]
> +static void guest_code(void)
> +{
> + unsigned long i, vlenb;
> +
> + csr_set(CSR_SSTATUS, SR_VS_INITIAL);
> + vlenb = csr_read(CSR_VLENB);
> + GUEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB);
> + for (i = 0; i < vlenb; i++)
> + source[i] = OLD_PATTERN;
> +
> + asm volatile(".option push\n"
> + ".option arch, +v\n"
> + "vsetvli zero, %1, e8, m1, ta, ma\n"
> + "vle8.v v0, (%0)\n"
> + ".option pop\n"
> + :
> + : "r"(source), "r"(vlenb)
> + : "memory");
> + GUEST_SYNC(1);
> +
> + asm volatile(".option push\n"
> + ".option arch, +v\n"
> + "vsetvli zero, %1, e8, m1, ta, ma\n"
> + "vse8.v v0, (%0)\n"
> + ".option pop\n"
> + :
> + : "r"(observed), "r"(vlenb)
> + : "memory");
> + GUEST_DONE();
> +}
[Severity: Medium]
Can C function calls and compiler auto-vectorization clobber v0 while
sstatus.VS is enabled in guest_code()?
When building the RISC-V KVM selftests with a toolchain targeting RVV (such
as a native GCC or Clang configured for rv64gcv/rva23, or with
EXTRA_CFLAGS=-march=rv64gcv), Makefile.kvm compiles guest code and
lib/string_override.c with -O2 -fno-builtin-memset and without
-fno-tree-vectorize or -fno-vectorize.
In guest_code() here, as well as guest_code() in
tools/testing/selftests/kvm/riscv/vector_owner_onereg_test.c and old_guest()
in tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c, the guest
sets SR_VS_INITIAL, loads a pattern into the caller-saved register v0 (in an
inline asm block without vector register clobbers), and then calls
GUEST_SYNC() or GUEST_DONE() while sstatus.VS is still active:
GUEST_SYNC() / GUEST_DONE()
-> ucall()
-> ucall_alloc()
-> memset(uc->args, 0, sizeof(uc->args))
If the compiler vectorizes memset() or the surrounding C loops/helpers, it is
free to use caller-saved v0 before the sbi_ecall traps to the host. That
overwrites v0 with zeros before KVM saves or inspects the register, causing
false test failures in vector_owner_fault_test and vector_owner_onereg_test,
and zeroing old_vcpu's v0 before exit in vector_owner_lifetime_test so a
leaked resident v0 would match the zero pattern expected by new_guest().
Should the guest helpers keep sstatus.VS disabled across C calls (or perform
the vector load/store and ECALL inside a single assembly block) so the
compiler cannot touch v0 while guest Vector state is live?
> +static void pin_to_one_cpu(void)
> +{
> + cpu_set_t allowed, pinned;
> + int cpu;
> +
> + kvm_sched_getaffinity(0, sizeof(allowed), &allowed);
> + for (cpu = 0; cpu < CPU_SETSIZE; cpu++) {
> + if (CPU_ISSET(cpu, &allowed))
> + break;
> + }
> + TEST_ASSERT(cpu < CPU_SETSIZE, "no CPU available");
> + CPU_ZERO(&pinned);
> + CPU_SET(cpu, &pinned);
> + kvm_sched_setaffinity(0, sizeof(pinned), &pinned);
> +}
[ ... ]
> +int main(void)
> +{
[ ... ]
> + pin_to_one_cpu();
> + vm = vm_create_with_one_vcpu(&vcpu, guest_code);
> + TEST_REQUIRE(__vcpu_has_isa_ext(vcpu, KVM_RISCV_ISA_EXT_V));
> + vm_init_vector_tables(vm);
> + vcpu_init_vector_tables(vcpu);
> + vlenb_id = KVM_REG_RISCV | KVM_REG_RISCV_VECTOR |
> + KVM_REG_SIZE_ULONG | KVM_REG_RISCV_VECTOR_CSR_REG(vlenb);
> + ret = __vcpu_get_reg(vcpu, vlenb_id, &vlenb);
> + TEST_ASSERT(!ret, "GET vlenb failed: %d", ret);
> + TEST_ASSERT(vlenb && vlenb <= RISCV_MAX_VLENB &&
> + !(vlenb & (vlenb - 1)), "bad vlenb: %lu", vlenb);
> + size = (u64)__builtin_ctzl(vlenb) << KVM_REG_SIZE_SHIFT;
> + v0_id = KVM_REG_RISCV | KVM_REG_RISCV_VECTOR | size |
> + KVM_REG_RISCV_VECTOR_REG(0);
> +
> + page_size = sysconf(_SC_PAGESIZE);
> + TEST_ASSERT(page_size > 0 && vlenb <= 2 * page_size, "bad page size");
> + mapping = mmap(NULL, 2 * page_size, PROT_READ | PROT_WRITE,
> + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
> + TEST_ASSERT(mapping != MAP_FAILED, "mmap failed: %d", errno);
[ ... ]
> + kvm_vm_free(vm);
> + ret = munmap(mapping, 2 * page_size);
> + TEST_ASSERT(!ret, "munmap failed: %d", errno);
> + return 0;
> +}
[Severity: Medium]
Could these three tests use existing kvm_util/kvm_syscalls helpers and be
consolidated into a single Vector owner test binary?
Specifically:
1. pin_to_one_cpu() in vector_owner_fault_test.c,
vector_owner_lifetime_test.c, and vector_owner_onereg_test.c reimplements
pin_self_to_any_cpu() (or pin_self_to_cpu()) from
tools/testing/selftests/kvm/include/kvm_util.h.
2. main() in vector_owner_fault_test.c hand-rolls mmap() and munmap() with
TEST_ASSERT() instead of using kvm_mmap() and kvm_munmap() from
tools/testing/selftests/kvm/include/kvm_syscalls.h, and both
vector_owner_fault_test.c and vector_owner_onereg_test.c call
__vcpu_get_reg() with a manual TEST_ASSERT() for the ULONG-sized vlenb
register instead of vcpu_get_reg().
3. Splitting the lifetime, ONE_REG, and fault sub-cases across three separate
binaries in tools/testing/selftests/kvm/Makefile.kvm duplicates the Vector
VM/vCPU initialization, vlenb and v0 register ID calculation, guest
vle8.v/vse8.v wrappers, and ucall run helpers across all three files.
> diff --git a/tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c b/tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c
> new file mode 100644
> index 0000000000000..c7b057b27cd0e
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c
> @@ -0,0 +1,127 @@
[ ... ]
> +int main(void)
> +{
> + struct kvm_vm *old_vm, *new_vm;
> + struct kvm_vcpu *old_vcpu, *new_vcpu;
> +
> + pin_to_one_cpu();
> + old_vm = create_vector_vm(&old_vcpu, old_guest);
> + run_to_done(old_vcpu);
> + kvm_vm_free(old_vm);
> +
> + new_vm = create_vector_vm(&new_vcpu, new_guest);
> + run_to_done(new_vcpu);
> + kvm_vm_free(new_vm);
[Severity: Medium]
Does freeing old_vm while no other VM is open clear the per-CPU Vector owner
before new_vm is ever created?
In main() in tools/testing/selftests/kvm/riscv/vector_owner_lifetime_test.c,
kvm_vm_free(old_vm) closes the only open VM fd, which drops kvm_usage_count
to zero:
kvm_destroy_vm()
-> kvm_disable_virtualization()
-> cpuhp_remove_state(CPUHP_AP_KVM_ONLINE)
-> kvm_offline_cpu()
-> kvm_arch_disable_virtualization_cpu()
-> kvm_riscv_csr_cleanup()
-> kvm_riscv_clear_former_vcpu()
-> kvm_riscv_vcpu_vector_owner_invalidate()
Because kvm_riscv_vcpu_vector_owner_invalidate() zeroes the per-CPU
kvm_riscv_vector_owner structure when the last VM is destroyed, the resident
owner from old_vcpu is already cleared before new_vm is created. When
new_vcpu runs, kvm_riscv_vcpu_vector_owner_load() in
arch/riscv/kvm/vcpu_vector.c always hits the !owner->identity branch:
} else if (!owner->identity) {
reason = vcpu->arch.last_exit_cpu == cpu ?
KVM_RISCV_VECTOR_OWNER_EMPTY :
KVM_RISCV_VECTOR_OWNER_MIGRATION;
} else if (owner->identity != (unsigned long)vcpu) {
reason = KVM_RISCV_VECTOR_OWNER_OTHER_VCPU;
} else if (owner->cookie != vcpu->arch.vector_owner_cookie) {
reason = KVM_RISCV_VECTOR_OWNER_COOKIE;
} else if (owner->generation != vcpu->arch.vector_state_generation) {
reason = KVM_RISCV_VECTOR_OWNER_GENERATION;
So the test never exercises the stale-vCPU identity or cookie checks (and
even if a second VM is kept alive so kvm_usage_count does not drop to zero
and new_vcpu reuses old_vcpu's struct kvm_vcpu address, old_vcpu leaves
owner->generation == 2 while new_vcpu starts its first KVM_RUN with
vector_state_generation == 1).
Should the test keep a VM alive across old_vcpu destruction (or recreate the
vCPU in a way that preserves the per-CPU owner state and matches generation)
so that the stale-owner checks in kvm_riscv_vcpu_vector_owner_load() are
actually reached?
--
Sashiko AI review · https://sashiko.dev/#/patchset/OSQPR01MB63265385597FE472C4209353C2602@OSQPR01MB6326.apcprd01.prod.exchangelabs.com?part=3
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC PATCH 1/3] riscv: vector: Add a guest owner invalidation hook
2026-10-27 13:13 ` slack up
(?)
(?)
@ 2026-10-27 14:03 ` sashiko-bot
-1 siblings, 0 replies; 15+ messages in thread
From: sashiko-bot @ 2026-10-27 14:03 UTC (permalink / raw)
To: slack up; +Cc: kvm
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] riscv: vector: RCU and noinstr violation in riscv_v_vstate_discard()
- [High] riscv: vector: missed owner invalidation in riscv_v_start_kernel_context()
--
commit 1bfcfc81beacec1c831b6f2d3e0f036424e54b70
Author: Xiaochun Yu <yxcnew@outlook.com>
riscv: vector: Add a guest owner invalidation hook
This commit introduces an RCU-protected invalidation callback guarded by a
static key so generic RISC-V Vector paths can notify KVM before modifying
the hardware Vector register file. It adds invalidation hooks to task
Vector state save, restore, and discard paths, kernel-mode Vector entry,
and context switch resume for preemptible kernel Vector users.
> diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h
> index 1f8dcae3af517..967dd70f4ea4a 100644
> --- a/arch/riscv/include/asm/vector.h
> +++ b/arch/riscv/include/asm/vector.h
[ ... ]
> @@ -303,6 +323,7 @@ static inline void __riscv_v_vstate_discard(void)
> static inline void riscv_v_vstate_discard(struct pt_regs *regs)
> {
> if (riscv_v_vstate_query(regs)) {
> + riscv_v_owner_invalidate(RISCV_V_INVALIDATE_TASK_DISCARD);
> __riscv_v_vstate_discard();
> __riscv_v_vstate_dirty(regs);
> }
[Severity: High]
Is it safe to call riscv_v_owner_invalidate() from riscv_v_vstate_discard()
during syscall entry before RCU is watching?
In arch/riscv/kernel/traps.c, do_trap_ecall_u() is a __trap_section
(noinstr) handler that calls riscv_v_vstate_discard() before entering the
kernel context:
arch/riscv/kernel/traps.c:do_trap_ecall_u() {
if (user_mode(regs)) {
...
riscv_v_vstate_discard(regs);
if (likely(syscall_enter_from_user_mode_randomize_stack(regs, &syscall))) {
The transition out of the user-mode RCU extended quiescent state
(CT_STATE_USER) via enter_from_user_mode() -> user_exit_irqoff() only
happens inside syscall_enter_from_user_mode_randomize_stack().
When a userspace task with active Vector state issues a syscall while
riscv_v_owner_invalidate_key is enabled, riscv_v_vstate_discard() calls
into the instrumentable __riscv_v_owner_invalidate() function, which calls
rcu_read_lock(), rcu_dereference(), the registered callback, and
rcu_read_unlock() while the CPU is still in CT_STATE_USER.
On kernels with CONFIG_CONTEXT_TRACKING_USER=y (such as CONFIG_NO_HZ_FULL=y
or CONFIG_CONTEXT_TRACKING_FORCE=y), RCU is not watching at this point.
Wouldn't this violate noinstr constraints, trigger the rcu_read_lock()
lockdep warning for being used while idle, and allow synchronize_rcu() in
kvm_riscv_unregister_vctx_invalidate() to finish while invalidate_callback()
is still executing?
> @@ -393,6 +416,13 @@ static inline void __switch_to_vector(struct task_struct *prev,
> }
>
> if (riscv_preempt_v_started(next)) {
> + /*
> + * A preemptible kernel Vector user can be scheduled back after a
> + * vCPU published a guest owner on this hart. Drop that owner before
> + * the kernel context resumes or restores and overwrites the register
> + * file.
> + */
> + riscv_v_owner_invalidate(RISCV_V_INVALIDATE_KERNEL_VECTOR);
> if (next->thread.riscv_v_flags & RISCV_PREEMPT_V_IN_SCHEDULE) {
> next->thread.riscv_v_flags &= ~RISCV_PREEMPT_V_IN_SCHEDULE;
> riscv_v_enable();
[ ... ]
> diff --git a/arch/riscv/kernel/kernel_mode_vector.c b/arch/riscv/kernel/kernel_mode_vector.c
> index 77e98b504485e..9e87bdf020285 100644
> --- a/arch/riscv/kernel/kernel_mode_vector.c
> +++ b/arch/riscv/kernel/kernel_mode_vector.c
[ ... ]
> @@ -106,6 +142,8 @@ static void __riscv_flush_vector_context(void)
> {
> void (*vcpu_flush_v_callback)(void);
>
> + riscv_v_owner_invalidate(RISCV_V_INVALIDATE_KERNEL_VECTOR);
> +
> if (riscv_v_flags() & RISCV_V_VCPU_CTX) {
> rcu_read_lock();
> vcpu_flush_v_callback = rcu_dereference(kvm_flush_vector_ctx_callback);
[Severity: High]
Can a preemption or migration between put_cpu_vector_context() and
riscv_v_start(RISCV_PREEMPT_V) in riscv_v_start_kernel_context() miss a
required guest owner invalidation?
With CONFIG_RISCV_ISA_V_PREEMPTIVE=y, kernel_vector_begin() sets up
preemptible kernel Vector execution via riscv_v_start_kernel_context():
arch/riscv/kernel/kernel_mode_vector.c:riscv_v_start_kernel_context() {
...
/* Transfer the ownership of V from user to kernel, then save */
get_cpu_vector_context();
__riscv_flush_vector_context();
put_cpu_vector_context();
...
riscv_v_start(RISCV_PREEMPT_V);
return 0;
}
__riscv_flush_vector_context() calls riscv_v_owner_invalidate() on the
current hart while preemption is disabled, and then put_cpu_vector_context()
re-enables preemption and bottom halves before riscv_v_start(RISCV_PREEMPT_V)
sets RISCV_PREEMPT_V in current->thread.riscv_v_flags.
If the task is preempted in that window before RISCV_PREEMPT_V is set, a
vCPU thread can run on the same hart and publish guest Vector ownership, or
the task can migrate to another hart that already has a resident guest
Vector owner.
When the task is scheduled back in, __switch_to_vector() checks
riscv_preempt_v_started(next), which is still false because RISCV_PREEMPT_V
has not been set yet, so __switch_to_vector() skips
riscv_v_owner_invalidate(). The task then sets RISCV_PREEMPT_V, returns to
kernel_vector_begin(), calls riscv_v_enable(), and overwrites the hardware
Vector registers while the hart's guest Vector owner remains valid.
--
Sashiko AI review · https://sashiko.dev/#/patchset/OSQPR01MB63265385597FE472C4209353C2602@OSQPR01MB6326.apcprd01.prod.exchangelabs.com?part=1
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [RFC PATCH 2/3] RISC-V: KVM: Preserve guest Vector state across vCPU put/load
2026-10-27 13:13 ` slack up
(?)
(?)
@ 2026-10-27 14:11 ` sashiko-bot
-1 siblings, 0 replies; 15+ messages in thread
From: sashiko-bot @ 2026-10-27 14:11 UTC (permalink / raw)
To: slack up; +Cc: kvm
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Critical] RISC-V: KVM: missing vector owner invalidation on preempt_v start race
- [High] RISC-V: KVM: vector owner invalidation runs before enter_from_user_mode()
- [Low] RISC-V: KVM: missing kernel-parameters.txt entry for vector_owner_fastpath
--
commit f4e86411e0473662833880eb7f87be358e15d73e
Author: Xiaochun Yu <yxcnew@outlook.com>
RISC-V: KVM: Preserve guest Vector state across vCPU put/load
This commit introduces per-hart Vector owner tracking in RISC-V KVM so that a
vCPU reloaded on the same hart can restore only Vector control CSRs when its
Vector data registers remain resident in hardware. The optimization is gated
behind a read-only vector_owner_fastpath module parameter and hooks into the
generic RISC-V Vector context invalidation callback.
> diff --git a/arch/riscv/kvm/vcpu_vector.c b/arch/riscv/kvm/vcpu_vector.c
> index fefc2857f753a..c197d3d7b539d 100644
> --- a/arch/riscv/kvm/vcpu_vector.c
> +++ b/arch/riscv/kvm/vcpu_vector.c
> @@ -17,7 +19,50 @@
> #include <asm/kvm_vcpu_vector.h>
> #include <asm/vector.h>
>
> +#include "trace.h"
> +
> #ifdef CONFIG_RISCV_ISA_V
[ ... ]
> +static DEFINE_PER_CPU(struct kvm_riscv_vector_owner,
> + kvm_riscv_vector_owner);
> +static atomic64_t kvm_riscv_vector_owner_cookie = ATOMIC64_INIT(0);
> +static bool kvm_riscv_vector_owner_fastpath;
> +module_param_named(vector_owner_fastpath, kvm_riscv_vector_owner_fastpath,
> + bool, 0444);
> +MODULE_PARM_DESC(vector_owner_fastpath,
> + "Preserve matched guest Vector registers across vCPU put/load");
[Severity: Low]
Since vector_owner_fastpath is a read-only module parameter checked during
initialization in kvm_riscv_v_init() (and passed on the kernel command line
as kvm.vector_owner_fastpath when KVM is built-in), should this parameter
also be documented in Documentation/admin-guide/kernel-parameters.txt?
> +
> +bool kvm_riscv_vcpu_vector_owner_enabled(void)
> +{
> + return READ_ONCE(kvm_riscv_vector_owner_fastpath);
> +}
[ ... ]
> +bool kvm_riscv_vcpu_vector_owner_load(struct kvm_vcpu *vcpu, int cpu)
> +{
> + struct kvm_riscv_vector_owner *owner;
> + unsigned long guest_vs = vcpu->arch.guest_context.sstatus & SR_VS;
> + unsigned long host_vs = task_pt_regs(current)->status & SR_VS;
> + enum kvm_riscv_vector_owner_reason reason;
> + bool eligible = false;
> +
> + if (!kvm_riscv_vcpu_vector_owner_enabled())
> + return false;
> +
> + owner = this_cpu_ptr(&kvm_riscv_vector_owner);
> + if (guest_vs == SR_VS_OFF) {
> + reason = KVM_RISCV_VECTOR_OWNER_GUEST_OFF;
> + } else if (host_vs != SR_VS_OFF) {
> + reason = KVM_RISCV_VECTOR_OWNER_HOST_ON;
> + } else if (!owner->identity) {
> + reason = vcpu->arch.last_exit_cpu == cpu ?
> + KVM_RISCV_VECTOR_OWNER_EMPTY :
> + KVM_RISCV_VECTOR_OWNER_MIGRATION;
> + } else if (owner->identity != (unsigned long)vcpu) {
> + reason = KVM_RISCV_VECTOR_OWNER_OTHER_VCPU;
> + } else if (owner->cookie != vcpu->arch.vector_owner_cookie) {
> + reason = KVM_RISCV_VECTOR_OWNER_COOKIE;
> + } else if (owner->generation != vcpu->arch.vector_state_generation) {
> + reason = KVM_RISCV_VECTOR_OWNER_GENERATION;
> + } else {
> + reason = KVM_RISCV_VECTOR_OWNER_HIT;
> + eligible = true;
> + }
[Severity: Critical]
Can a preemption window in riscv_v_start_kernel_context() allow a kernel
Vector user to clobber the hardware Vector registers without invalidating
kvm_riscv_vector_owner, causing kvm_riscv_vcpu_vector_owner_load() to
falsely report KVM_RISCV_VECTOR_OWNER_HIT?
With CONFIG_RISCV_ISA_V_PREEMPTIVE=y, kernel_vector_begin() calls
riscv_v_start_kernel_context() in arch/riscv/kernel/kernel_mode_vector.c:
/* Transfer the ownership of V from user to kernel, then save */
get_cpu_vector_context();
__riscv_flush_vector_context();
put_cpu_vector_context();
/*
* A voluntary context switch caused by put_cpu_vector_context() can
* raise the NEED_RESTORE flag if preempt_v starts too early due to a
* failed risv_v_is_on() check.
*
* This causes the next context_nesting_end pollute the v-reg from
* the stale context memory in kernel-mode vector.
*/
riscv_v_start(RISCV_PREEMPT_V);
__riscv_flush_vector_context() calls
riscv_v_owner_invalidate(RISCV_V_INVALIDATE_KERNEL_VECTOR) on the initial
hart, and then put_cpu_vector_context() clears RISCV_KERNEL_MODE_V and
re-enables bottom halves or preemption before RISCV_PREEMPT_V is set on
current->thread.riscv_v_flags.
If the task is preempted at put_cpu_vector_context() before
riscv_v_start(RISCV_PREEMPT_V) runs:
1. A vCPU thread can run on this hart and publish itself in
kvm_riscv_vector_owner via kvm_arch_vcpu_put() ->
kvm_riscv_vcpu_vector_owner_put(), or the preempted task can migrate to
another hart that already has a published kvm_riscv_vector_owner.
2. When the task is scheduled back in, __switch_to_vector() in
arch/riscv/include/asm/vector.h checks riscv_preempt_v_started(next):
if (riscv_preempt_v_started(next)) {
riscv_v_owner_invalidate(RISCV_V_INVALIDATE_KERNEL_VECTOR);
...
} else {
riscv_v_vstate_set_restore(next, task_pt_regs(next));
}
Because RISCV_PREEMPT_V is not yet set, __switch_to_vector() skips
riscv_v_owner_invalidate(RISCV_V_INVALIDATE_KERNEL_VECTOR).
3. The task then executes riscv_v_start(RISCV_PREEMPT_V), returns to
kernel_vector_begin(), enables Vector, and overwrites v0..v31 with host
kernel data while kvm_riscv_vector_owner on that hart still matches the
vCPU.
4. When that vCPU reloads on the same hart,
kvm_riscv_vcpu_vector_owner_load() sees a matching owner, cookie, and
generation and requests RISCV_V_VCPU_CSR_RESTORE.
kvm_riscv_vcpu_enter_exit() in arch/riscv/kvm/vcpu.c then restores only
the Vector CSRs via __kvm_riscv_vector_restore_csrs() and enters the
guest with host kernel Vector register contents in v0..v31.
> +
> + trace_kvm_riscv_vector_owner(vcpu->vcpu_id, cpu, 0, eligible,
> + reason, host_vs, guest_vs,
> + vcpu->arch.vector_state_generation);
[ ... ]
> +void kvm_riscv_vcpu_vector_owner_invalidate(enum riscv_v_invalidate_reason reason)
> +{
> + struct kvm_riscv_vector_owner *owner;
> + struct kvm_vcpu *vcpu;
> + enum kvm_riscv_vector_owner_reason trace_reason;
> + unsigned long host_vs;
> + u32 flags;
[ ... ]
> + preempt_disable();
> + flags = riscv_v_flags();
> + if (flags & RISCV_V_VCPU_CSR_RESTORE) {
> + vcpu = *this_cpu_ptr(kvm_get_running_vcpus());
> + if (!WARN_ON_ONCE(!vcpu)) {
> + flags &= ~RISCV_V_VCPU_CSR_RESTORE;
> + flags |= RISCV_V_VCPU_NEED_RESTORE;
> + riscv_v_flags_set(flags);
> + host_vs = task_pt_regs(current)->status & SR_VS;
> + trace_kvm_riscv_vector_owner(vcpu->vcpu_id,
> + smp_processor_id(), 2, false,
> + trace_reason, host_vs,
> + vcpu->arch.guest_context.sstatus & SR_VS,
> + vcpu->arch.vector_state_generation);
> + }
> + }
> +
> + owner = this_cpu_ptr(&kvm_riscv_vector_owner);
> + if (!owner->identity)
> + goto out;
> +
> + host_vs = task_pt_regs(current)->status & SR_VS;
> + trace_kvm_riscv_vector_owner(owner->vcpu_id, smp_processor_id(), 2,
> + false, trace_reason, host_vs,
> + owner->guest_vs, owner->generation);
> + owner->identity = 0;
> + owner->cookie = 0;
> + owner->generation = 0;
> + owner->vcpu_id = 0;
> + owner->guest_vs = 0;
> +
> +out:
> + preempt_enable();
> +}
[Severity: High]
When kvm_riscv_v_init() in arch/riscv/include/asm/kvm_vcpu_vector.h registers
kvm_riscv_vcpu_vector_owner_invalidate(), can this callback and its RCU
dispatch in __riscv_v_owner_invalidate() run from do_trap_ecall_u() before
enter_from_user_mode() transitions RCU and lockdep out of user mode?
In arch/riscv/kernel/traps.c, do_trap_ecall_u() is a noinstr trap handler
that calls riscv_v_vstate_discard(regs) before entering the kernel context:
if (user_mode(regs)) {
long syscall = regs->a7;
regs->epc += 4;
regs->orig_a0 = regs->a0;
regs->a0 = -ENOSYS;
riscv_v_vstate_discard(regs);
if (likely(syscall_enter_from_user_mode_randomize_stack(regs, &syscall))) {
When a user task with active Vector state issues a syscall,
riscv_v_vstate_discard(regs) in arch/riscv/include/asm/vector.h calls
riscv_v_owner_invalidate(RISCV_V_INVALIDATE_TASK_DISCARD) ->
__riscv_v_owner_invalidate() in arch/riscv/kernel/kernel_mode_vector.c:
rcu_read_lock();
invalidate_callback = rcu_dereference(kvm_invalidate_vector_ctx_callback);
if (invalidate_callback)
invalidate_callback(reason);
rcu_read_unlock();
Because enter_from_user_mode() is only invoked later inside
syscall_enter_from_user_mode_randomize_stack(), on configurations with
CONFIG_CONTEXT_TRACKING_USER=y (such as NO_HZ_FULL), RCU is still in
CT_STATE_USER (!rcu_is_watching()) and lockdep_hardirqs_off() has not run
yet. Calling rcu_read_lock(), rcu_dereference(), preempt_disable(), and
trace_kvm_riscv_vector_owner() here will trigger RCU and tracepoint lockdep
warnings and leave the callback invocation unprotected by synchronize_rcu()
during kvm_riscv_v_exit().
--
Sashiko AI review · https://sashiko.dev/#/patchset/OSQPR01MB63265385597FE472C4209353C2602@OSQPR01MB6326.apcprd01.prod.exchangelabs.com?part=2
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2026-10-27 14:11 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-10-27 13:13 [RFC PATCH 0/3] RISC-V: KVM: Preserve resident guest Vector state slack up
2026-10-27 13:13 ` slack up
2026-10-27 13:13 ` slack up
2026-10-27 13:13 ` [RFC PATCH 1/3] riscv: vector: Add a guest owner invalidation hook slack up
2026-10-27 13:13 ` slack up
2026-10-27 13:13 ` slack up
2026-10-27 14:03 ` sashiko-bot
2026-10-27 13:13 ` [RFC PATCH 2/3] RISC-V: KVM: Preserve guest Vector state across vCPU put/load slack up
2026-10-27 13:13 ` slack up
2026-10-27 13:13 ` slack up
2026-10-27 14:11 ` sashiko-bot
2026-10-27 13:13 ` [RFC PATCH 3/3] KVM: riscv: selftests: Add Vector owner regression tests slack up
2026-10-27 13:13 ` slack up
2026-10-27 13:13 ` slack up
2026-10-27 14:03 ` sashiko-bot
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.