[PATCH v2 0/4] KVM: s390: Permit ESA mode guests in VSIE

[PATCH v2 0/4] KVM: s390: Permit ESA mode guests in VSIE

[Eric Farman]

Hi Hendrik, Janosch, et al,

As promised here's a quick update to the ESA/VSIE series,
addressing the small comments raised in v1.

v1->v2:
 - [CB] Remove ECB_TE in this scenario
 - [JF] Simplify initialization of newflags
 - [JF] Include "_MASK_" in names of prefix mask definitions
v1: https://lore.kernel.org/linux-s390/20260401020915.1339228-1-farman@linux.ibm.com/

Eric Farman (3):
  KVM: s390: vsie: Allow non-zarch guests
  KVM: s390: vsie: Disable some bits when in ESA mode
  KVM: s390: vsie: Accommodate ESA prefix pages

Hendrik Brueckner (1):
  KVM: s390: Add KVM capability for ESA mode guests

 Documentation/virt/kvm/api.rst         |  8 +++++++
 arch/s390/include/asm/kvm_host.h       |  1 +
 arch/s390/include/asm/kvm_host_types.h |  3 +--
 arch/s390/kvm/kvm-s390.c               |  6 ++++++
 arch/s390/kvm/kvm-s390.h               |  5 ++++-
 arch/s390/kvm/vsie.c                   | 29 ++++++++++++++++++++++----
 include/uapi/linux/kvm.h               |  1 +
 7 files changed, 46 insertions(+), 7 deletions(-)

-- 
2.51.0

[PATCH v2 1/4] KVM: s390: vsie: Allow non-zarch guests

[Eric Farman]

Linux/KVM runs in z/Architecture-only mode. Although z/Architecture
is built upon a long history of hardware refinements, any other
CPU mode is not permitted.

Allow a userspace to explicitly enable the use of ESA mode for
nested guests, otherwise usage will be rejected.

Reviewed-by: Janosch Frank <frankja@linux.ibm.com>
Signed-off-by: Eric Farman <farman@linux.ibm.com>
---
 arch/s390/include/asm/kvm_host.h | 1 +
 arch/s390/kvm/vsie.c             | 8 +++++---
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
index 3039c88daa63..8a4f4a39f7a2 100644
--- a/arch/s390/include/asm/kvm_host.h
+++ b/arch/s390/include/asm/kvm_host.h
@@ -656,6 +656,7 @@ struct kvm_arch {
 	int user_stsi;
 	int user_instr0;
 	int user_operexec;
+	int allow_vsie_esamode;
 	struct s390_io_adapter *adapters[MAX_S390_IO_ADAPTERS];
 	wait_queue_head_t ipte_wq;
 	int ipte_lock_count;
diff --git a/arch/s390/kvm/vsie.c b/arch/s390/kvm/vsie.c
index 72895dddc39a..eee1b72e14df 100644
--- a/arch/s390/kvm/vsie.c
+++ b/arch/s390/kvm/vsie.c
@@ -125,8 +125,8 @@ static int prepare_cpuflags(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)
 	struct kvm_s390_sie_block *scb_o = vsie_page->scb_o;
 	int newflags, cpuflags = atomic_read(&scb_o->cpuflags);
 
-	/* we don't allow ESA/390 guests */
-	if (!(cpuflags & CPUSTAT_ZARCH))
+	/* we don't allow ESA/390 guests unless explicitly enabled */
+	if (!(cpuflags & CPUSTAT_ZARCH) && !vcpu->kvm->arch.allow_vsie_esamode)
 		return set_validity_icpt(scb_s, 0x0001U);
 
 	if (cpuflags & (CPUSTAT_RRF | CPUSTAT_MCDS))
@@ -135,7 +135,9 @@ static int prepare_cpuflags(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)
 		return set_validity_icpt(scb_s, 0x0007U);
 
 	/* intervention requests will be set later */
-	newflags = CPUSTAT_ZARCH;
+	newflags = 0;
+	if (cpuflags & CPUSTAT_ZARCH)
+		newflags = CPUSTAT_ZARCH;
 	if (cpuflags & CPUSTAT_GED && test_kvm_facility(vcpu->kvm, 8))
 		newflags |= CPUSTAT_GED;
 	if (cpuflags & CPUSTAT_GED2 && test_kvm_facility(vcpu->kvm, 78)) {
-- 
2.51.0

[PATCH v2 2/4] KVM: s390: vsie: Disable some bits when in ESA mode

[Eric Farman]

In the event that a nested guest is put in ESA mode,
ensure that some bits are scrubbed from the shadow SCB.

Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Eric Farman <farman@linux.ibm.com>
---
 arch/s390/kvm/vsie.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/arch/s390/kvm/vsie.c b/arch/s390/kvm/vsie.c
index eee1b72e14df..a3cf1d89d573 100644
--- a/arch/s390/kvm/vsie.c
+++ b/arch/s390/kvm/vsie.c
@@ -387,6 +387,17 @@ static int shadow_crycb(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)
 	return 0;
 }
 
+static void shadow_esa(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)
+{
+	struct kvm_s390_sie_block *scb_s = &vsie_page->scb_s;
+
+	/* Ensure these bits are indeed turned off */
+	scb_s->eca &= ~ECA_VX;
+	scb_s->ecb &= ~(ECB_GS | ECB_TE);
+	scb_s->ecb3 &= ~ECB3_RI;
+	scb_s->ecd &= ~ECD_HOSTREGMGMT;
+}
+
 /* shadow (round up/down) the ibc to avoid validity icpt */
 static void prepare_ibc(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)
 {
@@ -590,6 +601,9 @@ static int shadow_scb(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)
 	scb_s->hpid = HPID_VSIE;
 	scb_s->cpnc = scb_o->cpnc;
 
+	if (!(atomic_read(&scb_s->cpuflags) & CPUSTAT_ZARCH))
+		shadow_esa(vcpu, vsie_page);
+
 	prepare_ibc(vcpu, vsie_page);
 	rc = shadow_crycb(vcpu, vsie_page);
 out:
-- 
2.51.0

[PATCH v2 3/4] KVM: s390: vsie: Accommodate ESA prefix pages

[Eric Farman]

The prefix page address occupies a different number
of bits for z/Architecture versus ESA mode. Adjust the
definition to cover both, and permit an ESA mode
address within the nested codepath.

Signed-off-by: Eric Farman <farman@linux.ibm.com>
---
 arch/s390/include/asm/kvm_host_types.h | 3 +--
 arch/s390/kvm/kvm-s390.h               | 5 ++++-
 arch/s390/kvm/vsie.c                   | 7 ++++++-
 3 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/arch/s390/include/asm/kvm_host_types.h b/arch/s390/include/asm/kvm_host_types.h
index 1394d3fb648f..3f50942bdfe6 100644
--- a/arch/s390/include/asm/kvm_host_types.h
+++ b/arch/s390/include/asm/kvm_host_types.h
@@ -137,8 +137,7 @@ struct mcck_volatile_info {
 struct kvm_s390_sie_block {
 	atomic_t cpuflags;		/* 0x0000 */
 	__u32 : 1;			/* 0x0004 */
-	__u32 prefix : 18;
-	__u32 : 1;
+	__u32 prefix : 19;
 	__u32 ibc : 12;
 	__u8	reserved08[4];		/* 0x0008 */
 #define PROG_IN_SIE (1<<0)
diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h
index bf1d7798c1af..dc0573b7aa4b 100644
--- a/arch/s390/kvm/kvm-s390.h
+++ b/arch/s390/kvm/kvm-s390.h
@@ -122,7 +122,9 @@ static inline int kvm_is_ucontrol(struct kvm *kvm)
 #endif
 }
 
-#define GUEST_PREFIX_SHIFT 13
+#define GUEST_PREFIX_SHIFT 12
+#define GUEST_PREFIX_MASK_ZARCH 0x7fffe
+#define GUEST_PREFIX_MASK_ESA   0x7ffff
 static inline u32 kvm_s390_get_prefix(struct kvm_vcpu *vcpu)
 {
 	return vcpu->arch.sie_block->prefix << GUEST_PREFIX_SHIFT;
@@ -133,6 +135,7 @@ static inline void kvm_s390_set_prefix(struct kvm_vcpu *vcpu, u32 prefix)
 	VCPU_EVENT(vcpu, 3, "set prefix of cpu %03u to 0x%x", vcpu->vcpu_id,
 		   prefix);
 	vcpu->arch.sie_block->prefix = prefix >> GUEST_PREFIX_SHIFT;
+	vcpu->arch.sie_block->prefix &= GUEST_PREFIX_MASK_ZARCH;
 	kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu);
 	kvm_make_request(KVM_REQ_REFRESH_GUEST_PREFIX, vcpu);
 }
diff --git a/arch/s390/kvm/vsie.c b/arch/s390/kvm/vsie.c
index a3cf1d89d573..e5a23f1c9749 100644
--- a/arch/s390/kvm/vsie.c
+++ b/arch/s390/kvm/vsie.c
@@ -479,7 +479,7 @@ static int shadow_scb(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)
 	struct kvm_s390_sie_block *scb_s = &vsie_page->scb_s;
 	/* READ_ONCE does not work on bitfields - use a temporary variable */
 	const uint32_t __new_prefix = scb_o->prefix;
-	const uint32_t new_prefix = READ_ONCE(__new_prefix);
+	uint32_t new_prefix = READ_ONCE(__new_prefix);
 	const bool wants_tx = READ_ONCE(scb_o->ecb) & ECB_TE;
 	bool had_tx = scb_s->ecb & ECB_TE;
 	unsigned long new_mso = 0;
@@ -527,6 +527,11 @@ static int shadow_scb(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)
 
 	scb_s->icpua = scb_o->icpua;
 
+	if (!(atomic_read(&scb_s->cpuflags) & CPUSTAT_ZARCH))
+		new_prefix &= GUEST_PREFIX_MASK_ESA;
+	else
+		new_prefix &= GUEST_PREFIX_MASK_ZARCH;
+
 	if (!(atomic_read(&scb_s->cpuflags) & CPUSTAT_SM))
 		new_mso = READ_ONCE(scb_o->mso) & 0xfffffffffff00000UL;
 	/* if the hva of the prefix changes, we have to remap the prefix */
-- 
2.51.0

[PATCH v2 4/4] KVM: s390: Add KVM capability for ESA mode guests

[Eric Farman]

From: Hendrik Brueckner <brueckner@linux.ibm.com>

Now that all the bits are properly addressed, provide a mechanism
for testing ESA mode guests in nested configurations.

Signed-off-by: Hendrik Brueckner <brueckner@linux.ibm.com>
[farman@us.ibm.com: Updated commit message]
Reviewed-by: Janosch Frank <frankja@linux.ibm.com>
Signed-off-by: Eric Farman <farman@linux.ibm.com>
---
 Documentation/virt/kvm/api.rst | 8 ++++++++
 arch/s390/kvm/kvm-s390.c       | 6 ++++++
 include/uapi/linux/kvm.h       | 1 +
 3 files changed, 15 insertions(+)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 032516783e96..feabfee0f714 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -9436,6 +9436,14 @@ KVM exits with the register state of either the L1 or L2 guest
 depending on which executed at the time of an exit. Userspace must
 take care to differentiate between these cases.
 
+8.47 KVM_CAP_S390_VSIE_ESAMODE
+------------------------------
+
+:Architectures: s390
+
+The presence of this capability indicates that the nested KVM guest can
+start in ESA mode.
+
 9. Known KVM API problems
 =========================
 
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index d7838334a338..3856af54b6fe 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -629,6 +629,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_IRQFD_RESAMPLE:
 	case KVM_CAP_S390_USER_OPEREXEC:
 	case KVM_CAP_S390_KEYOP:
+	case KVM_CAP_S390_VSIE_ESAMODE:
 		r = 1;
 		break;
 	case KVM_CAP_SET_GUEST_DEBUG2:
@@ -926,6 +927,11 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
 		icpt_operexc_on_all_vcpus(kvm);
 		r = 0;
 		break;
+	case KVM_CAP_S390_VSIE_ESAMODE:
+		VM_EVENT(kvm, 3, "%s", "ENABLE: CAP_S390_VSIE_ESAMODE");
+		kvm->arch.allow_vsie_esamode = 1;
+		r = 0;
+		break;
 	default:
 		r = -EINVAL;
 		break;
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 80364d4dbebb..9710184d883d 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -989,6 +989,7 @@ struct kvm_enable_cap {
 #define KVM_CAP_ARM_SEA_TO_USER 245
 #define KVM_CAP_S390_USER_OPEREXEC 246
 #define KVM_CAP_S390_KEYOP 247
+#define KVM_CAP_S390_VSIE_ESAMODE 248
 
 struct kvm_irq_routing_irqchip {
 	__u32 irqchip;
-- 
2.51.0

Re: [PATCH v2 1/4] KVM: s390: vsie: Allow non-zarch guests

[Christian Borntraeger]

Am 01.04.26 um 17:12 schrieb Eric Farman:
> Linux/KVM runs in z/Architecture-only mode. Although z/Architecture
> is built upon a long history of hardware refinements, any other
> CPU mode is not permitted.
> 
> Allow a userspace to explicitly enable the use of ESA mode for
> nested guests, otherwise usage will be rejected.
> 
> Reviewed-by: Janosch Frank <frankja@linux.ibm.com>
> Signed-off-by: Eric Farman <farman@linux.ibm.com>

Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>

Re: [PATCH v2 1/4] KVM: s390: vsie: Allow non-zarch guests

[Hendrik Brueckner]

On Wed, Apr 01, 2026 at 05:12:18PM +0200, Eric Farman wrote:
> Linux/KVM runs in z/Architecture-only mode. Although z/Architecture
> is built upon a long history of hardware refinements, any other
> CPU mode is not permitted.
> 
> Allow a userspace to explicitly enable the use of ESA mode for
> nested guests, otherwise usage will be rejected.
> 
> Reviewed-by: Janosch Frank <frankja@linux.ibm.com>
> Signed-off-by: Eric Farman <farman@linux.ibm.com>

Reviewed-by: Hendrik Brueckner <brueckner@linux.ibm.com>

Re: [PATCH v2 2/4] KVM: s390: vsie: Disable some bits when in ESA mode

[Hendrik Brueckner]

On Wed, Apr 01, 2026 at 05:12:19PM +0200, Eric Farman wrote:
> In the event that a nested guest is put in ESA mode,
> ensure that some bits are scrubbed from the shadow SCB.
> 
> Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
> Signed-off-by: Eric Farman <farman@linux.ibm.com>

Reviewed-by: Hendrik Brueckner <brueckner@linux.ibm.com> 

Re: [PATCH v2 3/4] KVM: s390: vsie: Accommodate ESA prefix pages

[Hendrik Brueckner]

On Wed, Apr 01, 2026 at 05:12:20PM +0200, Eric Farman wrote:
> The prefix page address occupies a different number
> of bits for z/Architecture versus ESA mode. Adjust the
> definition to cover both, and permit an ESA mode
> address within the nested codepath.
> 
> Signed-off-by: Eric Farman <farman@linux.ibm.com>

Reviewed-by: Hendrik Brueckner <brueckner@linux.ibm.com>