[RFC PATCH v2 0/4] KVM: VM Planes host support for VBS

[RFC PATCH v2 0/4] KVM: VM Planes host support for VBS

[Sriram Nambakam]

This RFC adds the KVM host support needed to run Virtualization-Based
Security (VBS) across KVM VM Planes.  The series reserves the VM Plane and
VTL hypercalls, exits plane configuration and activation to userspace,
adds shared VTL-call state, and handles VTL call/return with an in-kernel
plane switch.

This is v2 of the series.  Based on feedback from maintainers, the KVM
host, guest, and QEMU changes are now being posted as three separate,
layered RFCs to make it easier to understand the context of these
changes as they pertain to the KVM host, guest, and QEMU.  This
patchset does not yet include memory protection between planes; that
will be added in a future version.

This series is based on the KVM VM Planes prerequisite patchset obtained
from Joerg Roedel's tree (kvm-planes-v7.1):
https://github.com/joergroedel/linux/tree/kvm-planes-v7.1

The dependent guest-side VBS support is being posted as a separate RFC.
This remains prototype code and is not intended for production use.

The implementation and integration tooling are available at:

  Linux and KVM support:
  https://github.com/safe-tee/linux/tree/vm-planes-layered

  QEMU support:
  https://github.com/safe-tee/qemu/tree/vm-planes-layered

  Build, test, and integration tooling:
  https://github.com/safe-tee/lvbs

Acknowledgments
===============

This work stands on top of, and is indebted to, several prior efforts:

  - Joerg Roedel, whose QEMU and KVM VM Planes work provides the
    infrastructure that the VBS/VSM secure plane relies on.

  - Paolo Bonzini, whose "[RFC PATCH 00/29] KVM: VM planes" introduced
    the VM Plane concept to KVM as a common in-kernel model for AMD VMPLs,
    Intel TDX partitions, Hyper-V VTLs, and Arm CCA planes.
    https://lwn.net/Articles/1016113/

  - James Bottomley and James Morris, for their ongoing VSM-on-KVM work,
    which informed the design and direction of this series.

Feedback on the hypercall ABI, userspace exits, and in-kernel plane-switch
handling is welcome.

Sriram Nambakam (4):
  kvm: uapi: reserve VM-plane and VBS hypercall numbers
  KVM: x86: exit VM-plane config/activate hypercalls to userspace
  KVM: x86: add VTL-call state to struct kvm_vcpu_common
  KVM: x86: handle VBS VTL call/return via in-kernel plane switch

 arch/x86/kvm/x86.c            | 88 ++++++++++++++++++++++++++++++++++-
 include/linux/kvm_host.h      |  5 ++
 include/uapi/linux/kvm_para.h |  4 ++
 3 files changed, 96 insertions(+), 1 deletion(-)


base-commit: ca4f6f8de47d9b9d9c3fc613668ec72579ba111c
-- 
2.55.0

[RFC PATCH v2 1/4] kvm: uapi: reserve VM-plane and VBS hypercall numbers

[Sriram Nambakam]

Reserve the paravirt hypercall numbers used by the KVM VM-planes and VBS
(Virtualization-Based Security) host support:

  13  KVM_HC_VM_PLANES_CONFIG    configure plane memory (serviced by the VMM)
  14  KVM_HC_VM_PLANES_ACTIVATE  create+activate a plane vCPU (serviced by VMM)
  15  KVM_HC_VBS_VTL_CALL        normal plane -> secure plane
  16  KVM_HC_VBS_VTL_RETURN      secure plane -> normal plane

This patch only reserves the numbers; the handlers are added separately.
---
 include/uapi/linux/kvm_para.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/include/uapi/linux/kvm_para.h b/include/uapi/linux/kvm_para.h
index 960c7e93d1a9..08b52d25ca17 100644
--- a/include/uapi/linux/kvm_para.h
+++ b/include/uapi/linux/kvm_para.h
@@ -30,6 +30,10 @@
 #define KVM_HC_SEND_IPI		10
 #define KVM_HC_SCHED_YIELD		11
 #define KVM_HC_MAP_GPA_RANGE		12
+#define KVM_HC_VM_PLANES_CONFIG		13
+#define KVM_HC_VM_PLANES_ACTIVATE	14
+#define KVM_HC_VBS_VTL_CALL		15
+#define KVM_HC_VBS_VTL_RETURN		16
 
 /*
  * hypercalls use architecture specific
-- 
2.55.0

[RFC PATCH v2 2/4] KVM: x86: exit VM-plane config/activate hypercalls to userspace

[Sriram Nambakam]

Handle the guest's plane-setup hypercalls by exiting to the VMM:

  KVM_HC_VM_PLANES_CONFIG    - allocate plane memory
  KVM_HC_VM_PLANES_ACTIVATE  - create and activate the plane vCPU

Both exit via KVM_EXIT_HYPERCALL (gated by KVM_CAP_EXIT_HYPERCALL, hence
added to KVM_EXIT_HYPERCALL_VALID_MASK) and are completed by userspace.
This is what brings a secure plane and its vCPU into existence; the
in-kernel VBS plane-switch handlers are added separately.
---
 arch/x86/kvm/x86.c | 29 ++++++++++++++++++++++++++++-
 1 file changed, 28 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 0ace79c957bb..3b21c72fc9e0 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -119,7 +119,10 @@ u64 __read_mostly efer_reserved_bits = ~((u64)(EFER_SCE | EFER_LME | EFER_LMA));
 static u64 __read_mostly efer_reserved_bits = ~((u64)EFER_SCE);
 #endif
 
-#define KVM_EXIT_HYPERCALL_VALID_MASK (1 << KVM_HC_MAP_GPA_RANGE)
+#define KVM_EXIT_HYPERCALL_VALID_MASK			\
+	((1 << KVM_HC_MAP_GPA_RANGE)		|	\
+	 (1 << KVM_HC_VM_PLANES_CONFIG)		|	\
+	 (1 << KVM_HC_VM_PLANES_ACTIVATE))
 
 #define KVM_CAP_PMU_VALID_MASK KVM_PMU_CAP_DISABLE
 
@@ -10559,6 +10562,30 @@ int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, int cpl,
 		vcpu->arch.complete_userspace_io = complete_hypercall;
 		return 0;
 	}
+	case KVM_HC_VM_PLANES_CONFIG:
+	case KVM_HC_VM_PLANES_ACTIVATE:
+		/*
+		 * VM plane setup: hand off to the VMM, which allocates plane
+		 * memory (CONFIG) and creates + activates the plane vCPU
+		 * (ACTIVATE).  Serviced entirely in userspace.
+		 */
+		if (!user_exit_on_hypercall(vcpu->kvm, nr))
+			break;
+
+		vcpu->run->exit_reason        = KVM_EXIT_HYPERCALL;
+		vcpu->run->hypercall.nr       = nr;
+		vcpu->run->hypercall.ret      = 0;
+		vcpu->run->hypercall.args[0]  = a0;
+		vcpu->run->hypercall.args[1]  = a1;
+		vcpu->run->hypercall.args[2]  = a2;
+		vcpu->run->hypercall.args[3]  = a3;
+		vcpu->run->hypercall.flags    = 0;
+		if (op_64_bit)
+			vcpu->run->hypercall.flags |= KVM_EXIT_HYPERCALL_LONG_MODE;
+
+		WARN_ON_ONCE(vcpu->run->hypercall.flags & KVM_EXIT_HYPERCALL_MBZ);
+		vcpu->arch.complete_userspace_io = complete_hypercall;
+		return 0;
 	default:
 		ret = -KVM_ENOSYS;
 		break;
-- 
2.55.0

[RFC PATCH v2 3/4] KVM: x86: add VTL-call state to struct kvm_vcpu_common

[Sriram Nambakam]

Add the shared state used by the in-kernel VBS VTL call/return handlers:

  vtl_call_ca       - guest-physical address of the shared calling area
  vtl_call_pending  - a call arrived while the secure plane was booting
  vtl_plane_ready   - the secure plane has parked in its VTL return

The fields live in struct kvm_vcpu_common so they are shared across a
logical CPU's plane vCPUs.
---
 include/linux/kvm_host.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index a61524ab0d94..61bb59c50ae6 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -386,6 +386,11 @@ struct kvm_vcpu_common {
 
 	bool plane_switch;
 
+	/* VBS VTL-call state, shared across this CPU's plane vCPUs. */
+	gpa_t vtl_call_ca;
+	bool vtl_call_pending;
+	bool vtl_plane_ready;
+
 	struct kvm_vcpu_arch_common arch;
 };
 
-- 
2.55.0

[RFC PATCH v2 4/4] KVM: x86: handle VBS VTL call/return via in-kernel plane switch

[Sriram Nambakam]

Service the VBS inter-plane hypercalls in-kernel, with no userspace round
trip, by switching planes:

  KVM_HC_VBS_VTL_CALL   - the normal plane (0) calls into the secure plane
                          (1).  a0 is the guest-physical address of the
                          shared calling area; deliver it to the secure
                          plane's pending VTL return (or mark it pending if
                          the secure plane is still booting) and switch.
  KVM_HC_VBS_VTL_RETURN - the secure plane (>0) hands control back to plane
                          0, announcing readiness and delivering any call
                          that arrived while it was booting.

The switch reuses the per-plane scheduling (kvm_vcpu_set_plane_runnable/
stopped + KVM_REQ_PLANE_RESCHED).
---
 arch/x86/kvm/x86.c | 59 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 3b21c72fc9e0..35fbe0776a3e 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -10562,6 +10562,65 @@ int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, int cpl,
 		vcpu->arch.complete_userspace_io = complete_hypercall;
 		return 0;
 	}
+	case KVM_HC_VBS_VTL_CALL: {
+		/*
+		 * Runtime VBS call from the normal plane (0) into the secure
+		 * plane (1), serviced in-kernel by switching planes.  a0 is the
+		 * guest-physical address of the shared calling area.  If the
+		 * secure plane is parked in its VTL return, deliver the GPA as
+		 * that return's value now; otherwise mark it pending so the
+		 * secure plane picks it up on its first return.
+		 */
+		struct kvm_vcpu_common *common = vcpu->common;
+		struct kvm_vcpu *secure;
+
+		if (vcpu->plane_level != 0)
+			break;
+
+		secure = common->vcpus[1];
+		if (!secure)
+			break;
+
+		common->vtl_call_ca = a0;
+		if (common->vtl_plane_ready) {
+			kvm_rax_write_raw(secure, a0);
+			common->vtl_call_pending = false;
+		} else {
+			common->vtl_call_pending = true;
+		}
+
+		kvm_vcpu_set_plane_runnable(secure);
+		kvm_vcpu_set_plane_stopped(vcpu);
+		ret = 0;
+		break;
+	}
+	case KVM_HC_VBS_VTL_RETURN: {
+		/*
+		 * The secure plane (>0) hands control back to plane 0.  On its
+		 * first return it announces readiness; if a call arrived while
+		 * it was still booting, deliver that calling-area GPA now and
+		 * stay in the secure plane.  Otherwise switch back to plane 0.
+		 */
+		struct kvm_vcpu_common *common = vcpu->common;
+
+		if (vcpu->plane_level == 0) {
+			ret = -KVM_EPERM;
+			break;
+		}
+
+		common->vtl_plane_ready = true;
+
+		if (common->vtl_call_pending) {
+			common->vtl_call_pending = false;
+			ret = common->vtl_call_ca;
+			break;
+		}
+
+		kvm_vcpu_set_plane_runnable(common->vcpus[0]);
+		kvm_vcpu_set_plane_stopped(vcpu);
+		ret = 0;
+		break;
+	}
 	case KVM_HC_VM_PLANES_CONFIG:
 	case KVM_HC_VM_PLANES_ACTIVATE:
 		/*
-- 
2.55.0