[PATCH 0/8] KVM: s390: fixes and cleanup

[PATCH 0/8] KVM: s390: fixes and cleanup

[Christian Borntraeger]

Gleb, Paolo,

here are some fixes and cleanups for KVM/s390.

The first two patches
"KVM: s390: move kvm_guest_enter,exit closer to sie"
and
"KVM: s390: fix pfmf non-quiescing control handling"

should go into 3.11. Everything else looks more like 3.12.
Please apply.

Christian


Dominik Dingel (2):
  KVM: s390: move kvm_guest_enter,exit closer to sie
  KVM: s390: Make KVM_HVA_ERR_BAD usable on s390

Heiko Carstens (1):
  KVM: s390: fix pfmf non-quiescing control handling

Martin Schwidefsky (2):
  KVM: s390: allow sie enablement for multi-threaded programs
  KVM: s390: fix task size check

Michael Mueller (1):
  KVM: s390: declare virtual HW facilities

Thomas Huth (2):
  KVM: s390: Fix sparse warnings in priv.c
  KVM: s390: Add helper function for setting condition code

 arch/s390/include/asm/kvm_host.h    |   8 ++
 arch/s390/include/asm/mmu.h         |   2 -
 arch/s390/include/asm/mmu_context.h |  19 +---
 arch/s390/include/asm/pgtable.h     |  11 +++
 arch/s390/include/asm/processor.h   |   2 +
 arch/s390/kvm/kvm-s390.c            |  44 ++++++---
 arch/s390/kvm/kvm-s390.h            |  10 ++
 arch/s390/kvm/priv.c                |  36 +++----
 arch/s390/mm/pgtable.c              | 183 +++++++++++++++++++++++-------------
 include/linux/kvm_host.h            |   8 ++
 10 files changed, 202 insertions(+), 121 deletions(-)

-- 
1.8.3.1

Re: [PATCH 0/8] KVM: s390: fixes and cleanup

[Paolo Bonzini]

Il 26/07/2013 15:03, Christian Borntraeger ha scritto:
> Gleb, Paolo,
> 
> here are some fixes and cleanups for KVM/s390.
> 
> The first two patches
> "KVM: s390: move kvm_guest_enter,exit closer to sie"
> and
> "KVM: s390: fix pfmf non-quiescing control handling"
> 
> should go into 3.11. Everything else looks more like 3.12.
> Please apply.
> 
> Christian
> 
> 
> Dominik Dingel (2):
>   KVM: s390: move kvm_guest_enter,exit closer to sie
>   KVM: s390: Make KVM_HVA_ERR_BAD usable on s390
> 
> Heiko Carstens (1):
>   KVM: s390: fix pfmf non-quiescing control handling
> 
> Martin Schwidefsky (2):
>   KVM: s390: allow sie enablement for multi-threaded programs
>   KVM: s390: fix task size check
> 
> Michael Mueller (1):
>   KVM: s390: declare virtual HW facilities
> 
> Thomas Huth (2):
>   KVM: s390: Fix sparse warnings in priv.c
>   KVM: s390: Add helper function for setting condition code
> 
>  arch/s390/include/asm/kvm_host.h    |   8 ++
>  arch/s390/include/asm/mmu.h         |   2 -
>  arch/s390/include/asm/mmu_context.h |  19 +---
>  arch/s390/include/asm/pgtable.h     |  11 +++
>  arch/s390/include/asm/processor.h   |   2 +
>  arch/s390/kvm/kvm-s390.c            |  44 ++++++---
>  arch/s390/kvm/kvm-s390.h            |  10 ++
>  arch/s390/kvm/priv.c                |  36 +++----
>  arch/s390/mm/pgtable.c              | 183 +++++++++++++++++++++++-------------
>  include/linux/kvm_host.h            |   8 ++
>  10 files changed, 202 insertions(+), 121 deletions(-)
> 

Applied all (1-2 to master, 3-8 to next/queue), thanks.

Paolo

[PATCH 0/8] KVM: s390: fixes and cleanup

[Christian Borntraeger]

Paolo, Gleb,

here is a bunch of patch for kvm on s390.
The first 4 patches restructure the code to protect most of 
vcpu_run with the kvm->srcu lock. The old code was structured
in way that adding the lock was more complicated than necessary,
therefore, the rework.

The last 4 patches deal with instruction intercept handling.

Patches are against kvm.git queue. Please apply.


Thomas Huth (8):
  KVM: s390: Remove dead "rerun vcpu" code
  KVM: s390: Split up __vcpu_run into three parts
  KVM: s390: Push run loop into __vcpu_run
  KVM: s390: Lock kvm->srcu at the appropriate places
  KVM: s390: Allow NULL parameter for kvm_s390_get_regs_rre
  KVM: s390: Helper for converting real addresses to absolute
  KVM: s390: Implement TEST BLOCK
  KVM: s390: Intercept SCK instruction

 arch/s390/include/asm/kvm_host.h |  1 -
 arch/s390/kvm/diag.c             |  4 +-
 arch/s390/kvm/gaccess.h          | 21 ++++++----
 arch/s390/kvm/intercept.c        |  6 ---
 arch/s390/kvm/interrupt.c        |  3 ++
 arch/s390/kvm/kvm-s390.c         | 91 ++++++++++++++++++++++++++--------------
 arch/s390/kvm/kvm-s390.h         |  9 ++--
 arch/s390/kvm/priv.c             | 61 +++++++++++++++++++++++++++
 8 files changed, 144 insertions(+), 52 deletions(-)

-- 
1.8.3.1

[PATCH 1/8] KVM: s390: Remove dead "rerun vcpu" code

[Christian Borntraeger]

From: Thomas Huth <thuth@linux.vnet.ibm.com>

The need for SIE_INTERCEPT_RERUNVCPU has been removed long ago already,
with the following commit:
	f7850c92884b40915001e332a0a33ed4f10158e8
	[S390] remove kvm mmu reload on s390
Since the remainders are dead code, they are now removed by this patch.

Signed-off-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 arch/s390/include/asm/kvm_host.h | 1 -
 arch/s390/kvm/intercept.c        | 6 ------
 arch/s390/kvm/kvm-s390.c         | 4 ----
 arch/s390/kvm/kvm-s390.h         | 3 +--
 4 files changed, 1 insertion(+), 13 deletions(-)

diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
index e87ecaa..6a0e27b 100644
--- a/arch/s390/include/asm/kvm_host.h
+++ b/arch/s390/include/asm/kvm_host.h
@@ -220,7 +220,6 @@ struct kvm_s390_interrupt_info {
 /* for local_interrupt.action_flags */
 #define ACTION_STORE_ON_STOP		(1<<0)
 #define ACTION_STOP_ON_STOP		(1<<1)
-#define ACTION_RELOADVCPU_ON_STOP	(1<<2)
 
 struct kvm_s390_local_interrupt {
 	spinlock_t lock;
diff --git a/arch/s390/kvm/intercept.c b/arch/s390/kvm/intercept.c
index 5ee56e5..5ddbbde 100644
--- a/arch/s390/kvm/intercept.c
+++ b/arch/s390/kvm/intercept.c
@@ -62,12 +62,6 @@ static int handle_stop(struct kvm_vcpu *vcpu)
 
 	trace_kvm_s390_stop_request(vcpu->arch.local_int.action_bits);
 
-	if (vcpu->arch.local_int.action_bits & ACTION_RELOADVCPU_ON_STOP) {
-		vcpu->arch.local_int.action_bits &= ~ACTION_RELOADVCPU_ON_STOP;
-		rc = SIE_INTERCEPT_RERUNVCPU;
-		vcpu->run->exit_reason = KVM_EXIT_INTR;
-	}
-
 	if (vcpu->arch.local_int.action_bits & ACTION_STOP_ON_STOP) {
 		atomic_set_mask(CPUSTAT_STOPPED,
 				&vcpu->arch.sie_block->cpuflags);
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index 776dafe..e3e7ff7 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -749,7 +749,6 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
 	int rc;
 	sigset_t sigsaved;
 
-rerun_vcpu:
 	if (vcpu->sigset_active)
 		sigprocmask(SIG_SETMASK, &vcpu->sigset, &sigsaved);
 
@@ -793,9 +792,6 @@ rerun_vcpu:
 			rc = kvm_handle_sie_intercept(vcpu);
 	} while (!signal_pending(current) && !rc);
 
-	if (rc == SIE_INTERCEPT_RERUNVCPU)
-		goto rerun_vcpu;
-
 	if (signal_pending(current) && !rc) {
 		kvm_run->exit_reason = KVM_EXIT_INTR;
 		rc = -EINTR;
diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h
index dc99f1c..a307a85 100644
--- a/arch/s390/kvm/kvm-s390.h
+++ b/arch/s390/kvm/kvm-s390.h
@@ -28,8 +28,7 @@ typedef int (*intercept_handler_t)(struct kvm_vcpu *vcpu);
 extern unsigned long *vfacilities;
 
 /* negativ values are error codes, positive values for internal conditions */
-#define SIE_INTERCEPT_RERUNVCPU		(1<<0)
-#define SIE_INTERCEPT_UCONTROL		(1<<1)
+#define SIE_INTERCEPT_UCONTROL		(1<<0)
 int kvm_handle_sie_intercept(struct kvm_vcpu *vcpu);
 
 #define VM_EVENT(d_kvm, d_loglevel, d_string, d_args...)\
-- 
1.8.3.1

[PATCH 2/8] KVM: s390: Split up __vcpu_run into three parts

[Christian Borntraeger]

From: Thomas Huth <thuth@linux.vnet.ibm.com>

In preparation for the following patch (which will change the indentation
of __vcpu_run quite a bit), this patch puts most of the code from __vcpu_run
into separate functions. The first function handles the code that runs
before the SIE instruction and the other one handles the code that runs
afterwards.

Signed-off-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 arch/s390/kvm/kvm-s390.c | 54 +++++++++++++++++++++++++++++++++---------------
 1 file changed, 37 insertions(+), 17 deletions(-)

diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index e3e7ff7..69c7592 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -689,9 +689,9 @@ static int kvm_s390_handle_requests(struct kvm_vcpu *vcpu)
 	return 0;
 }
 
-static int __vcpu_run(struct kvm_vcpu *vcpu)
+static int vcpu_pre_run(struct kvm_vcpu *vcpu)
 {
-	int rc;
+	int rc, cpuflags;
 
 	memcpy(&vcpu->arch.sie_block->gg14, &vcpu->run->s.regs.gprs[14], 16);
 
@@ -709,28 +709,24 @@ static int __vcpu_run(struct kvm_vcpu *vcpu)
 		return rc;
 
 	vcpu->arch.sie_block->icptcode = 0;
-	VCPU_EVENT(vcpu, 6, "entering sie flags %x",
-		   atomic_read(&vcpu->arch.sie_block->cpuflags));
-	trace_kvm_s390_sie_enter(vcpu,
-				 atomic_read(&vcpu->arch.sie_block->cpuflags));
+	cpuflags = atomic_read(&vcpu->arch.sie_block->cpuflags);
+	VCPU_EVENT(vcpu, 6, "entering sie flags %x", cpuflags);
+	trace_kvm_s390_sie_enter(vcpu, cpuflags);
 
-	/*
-	 * As PF_VCPU will be used in fault handler, between guest_enter
-	 * and guest_exit should be no uaccess.
-	 */
-	preempt_disable();
-	kvm_guest_enter();
-	preempt_enable();
-	rc = sie64a(vcpu->arch.sie_block, vcpu->run->s.regs.gprs);
-	kvm_guest_exit();
+	return 0;
+}
+
+static int vcpu_post_run(struct kvm_vcpu *vcpu, int exit_reason)
+{
+	int rc;
 
 	VCPU_EVENT(vcpu, 6, "exit sie icptcode %d",
 		   vcpu->arch.sie_block->icptcode);
 	trace_kvm_s390_sie_exit(vcpu, vcpu->arch.sie_block->icptcode);
 
-	if (rc > 0)
+	if (exit_reason >= 0) {
 		rc = 0;
-	if (rc < 0) {
+	} else {
 		if (kvm_is_ucontrol(vcpu->kvm)) {
 			rc = SIE_INTERCEPT_UCONTROL;
 		} else {
@@ -741,6 +737,30 @@ static int __vcpu_run(struct kvm_vcpu *vcpu)
 	}
 
 	memcpy(&vcpu->run->s.regs.gprs[14], &vcpu->arch.sie_block->gg14, 16);
+
+	return rc;
+}
+
+static int __vcpu_run(struct kvm_vcpu *vcpu)
+{
+	int rc, exit_reason;
+
+	rc = vcpu_pre_run(vcpu);
+	if (rc)
+		return rc;
+
+	/*
+	 * As PF_VCPU will be used in fault handler, between guest_enter
+	 * and guest_exit should be no uaccess.
+	 */
+	preempt_disable();
+	kvm_guest_enter();
+	preempt_enable();
+	exit_reason = sie64a(vcpu->arch.sie_block, vcpu->run->s.regs.gprs);
+	kvm_guest_exit();
+
+	rc = vcpu_post_run(vcpu, exit_reason);
+
 	return rc;
 }
 
-- 
1.8.3.1

[PATCH 3/8] KVM: s390: Push run loop into __vcpu_run

[Christian Borntraeger]

From: Thomas Huth <thuth@linux.vnet.ibm.com>

Moved the do-while loop from kvm_arch_vcpu_ioctl_run into __vcpu_run
and the calling of kvm_handle_sie_intercept() into vcpu_post_run()
(so we can add the srcu locks in a proper way in the next patch).

Signed-off-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 arch/s390/kvm/kvm-s390.c | 49 ++++++++++++++++++++++++------------------------
 1 file changed, 25 insertions(+), 24 deletions(-)

diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index 69c7592..8eec7ab 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -738,6 +738,13 @@ static int vcpu_post_run(struct kvm_vcpu *vcpu, int exit_reason)
 
 	memcpy(&vcpu->run->s.regs.gprs[14], &vcpu->arch.sie_block->gg14, 16);
 
+	if (rc == 0) {
+		if (kvm_is_ucontrol(vcpu->kvm))
+			rc = -EOPNOTSUPP;
+		else
+			rc = kvm_handle_sie_intercept(vcpu);
+	}
+
 	return rc;
 }
 
@@ -745,21 +752,24 @@ static int __vcpu_run(struct kvm_vcpu *vcpu)
 {
 	int rc, exit_reason;
 
-	rc = vcpu_pre_run(vcpu);
-	if (rc)
-		return rc;
-
-	/*
-	 * As PF_VCPU will be used in fault handler, between guest_enter
-	 * and guest_exit should be no uaccess.
-	 */
-	preempt_disable();
-	kvm_guest_enter();
-	preempt_enable();
-	exit_reason = sie64a(vcpu->arch.sie_block, vcpu->run->s.regs.gprs);
-	kvm_guest_exit();
+	do {
+		rc = vcpu_pre_run(vcpu);
+		if (rc)
+			break;
 
-	rc = vcpu_post_run(vcpu, exit_reason);
+		/*
+		 * As PF_VCPU will be used in fault handler, between
+		 * guest_enter and guest_exit should be no uaccess.
+		 */
+		preempt_disable();
+		kvm_guest_enter();
+		preempt_enable();
+		exit_reason = sie64a(vcpu->arch.sie_block,
+				     vcpu->run->s.regs.gprs);
+		kvm_guest_exit();
+
+		rc = vcpu_post_run(vcpu, exit_reason);
+	} while (!signal_pending(current) && !rc);
 
 	return rc;
 }
@@ -801,16 +811,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
 	}
 
 	might_fault();
-
-	do {
-		rc = __vcpu_run(vcpu);
-		if (rc)
-			break;
-		if (kvm_is_ucontrol(vcpu->kvm))
-			rc = -EOPNOTSUPP;
-		else
-			rc = kvm_handle_sie_intercept(vcpu);
-	} while (!signal_pending(current) && !rc);
+	rc = __vcpu_run(vcpu);
 
 	if (signal_pending(current) && !rc) {
 		kvm_run->exit_reason = KVM_EXIT_INTR;
-- 
1.8.3.1

[PATCH 4/8] KVM: s390: Lock kvm->srcu at the appropriate places

[Christian Borntraeger]

From: Thomas Huth <thuth@linux.vnet.ibm.com>

The kvm->srcu lock has to be held while accessing the memory of
guests and during certain other actions. This patch now adds
the locks to the __vcpu_run function so that all affected code
is protected now (and additionally to the KVM_S390_STORE_STATUS
ioctl, which can be called out-of-band and needs a separate lock).

Signed-off-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 arch/s390/kvm/diag.c      |  4 +---
 arch/s390/kvm/interrupt.c |  3 +++
 arch/s390/kvm/kvm-s390.c  | 12 ++++++++++++
 3 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/arch/s390/kvm/diag.c b/arch/s390/kvm/diag.c
index 3a74d8a..78d967f 100644
--- a/arch/s390/kvm/diag.c
+++ b/arch/s390/kvm/diag.c
@@ -107,14 +107,13 @@ static int __diag_ipl_functions(struct kvm_vcpu *vcpu)
 
 static int __diag_virtio_hypercall(struct kvm_vcpu *vcpu)
 {
-	int ret, idx;
+	int ret;
 
 	/* No virtio-ccw notification? Get out quickly. */
 	if (!vcpu->kvm->arch.css_support ||
 	    (vcpu->run->s.regs.gprs[1] != KVM_S390_VIRTIO_CCW_NOTIFY))
 		return -EOPNOTSUPP;
 
-	idx = srcu_read_lock(&vcpu->kvm->srcu);
 	/*
 	 * The layout is as follows:
 	 * - gpr 2 contains the subchannel id (passed as addr)
@@ -125,7 +124,6 @@ static int __diag_virtio_hypercall(struct kvm_vcpu *vcpu)
 				      vcpu->run->s.regs.gprs[2],
 				      8, &vcpu->run->s.regs.gprs[3],
 				      vcpu->run->s.regs.gprs[4]);
-	srcu_read_unlock(&vcpu->kvm->srcu, idx);
 
 	/*
 	 * Return cookie in gpr 2, but don't overwrite the register if the
diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
index 7f35cb3..e7323cd 100644
--- a/arch/s390/kvm/interrupt.c
+++ b/arch/s390/kvm/interrupt.c
@@ -436,6 +436,7 @@ int kvm_s390_handle_wait(struct kvm_vcpu *vcpu)
 	hrtimer_start(&vcpu->arch.ckc_timer, ktime_set (0, sltime) , HRTIMER_MODE_REL);
 	VCPU_EVENT(vcpu, 5, "enabled wait via clock comparator: %llx ns", sltime);
 no_timer:
+	srcu_read_unlock(&vcpu->kvm->srcu, vcpu->srcu_idx);
 	spin_lock(&vcpu->arch.local_int.float_int->lock);
 	spin_lock_bh(&vcpu->arch.local_int.lock);
 	add_wait_queue(&vcpu->wq, &wait);
@@ -455,6 +456,8 @@ no_timer:
 	remove_wait_queue(&vcpu->wq, &wait);
 	spin_unlock_bh(&vcpu->arch.local_int.lock);
 	spin_unlock(&vcpu->arch.local_int.float_int->lock);
+	vcpu->srcu_idx = srcu_read_lock(&vcpu->kvm->srcu);
+
 	hrtimer_try_to_cancel(&vcpu->arch.ckc_timer);
 	return 0;
 }
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index 8eec7ab..1e4e7b9 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -752,11 +752,18 @@ static int __vcpu_run(struct kvm_vcpu *vcpu)
 {
 	int rc, exit_reason;
 
+	/*
+	 * We try to hold kvm->srcu during most of vcpu_run (except when run-
+	 * ning the guest), so that memslots (and other stuff) are protected
+	 */
+	vcpu->srcu_idx = srcu_read_lock(&vcpu->kvm->srcu);
+
 	do {
 		rc = vcpu_pre_run(vcpu);
 		if (rc)
 			break;
 
+		srcu_read_unlock(&vcpu->kvm->srcu, vcpu->srcu_idx);
 		/*
 		 * As PF_VCPU will be used in fault handler, between
 		 * guest_enter and guest_exit should be no uaccess.
@@ -767,10 +774,12 @@ static int __vcpu_run(struct kvm_vcpu *vcpu)
 		exit_reason = sie64a(vcpu->arch.sie_block,
 				     vcpu->run->s.regs.gprs);
 		kvm_guest_exit();
+		vcpu->srcu_idx = srcu_read_lock(&vcpu->kvm->srcu);
 
 		rc = vcpu_post_run(vcpu, exit_reason);
 	} while (!signal_pending(current) && !rc);
 
+	srcu_read_unlock(&vcpu->kvm->srcu, vcpu->srcu_idx);
 	return rc;
 }
 
@@ -968,6 +977,7 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
 {
 	struct kvm_vcpu *vcpu = filp->private_data;
 	void __user *argp = (void __user *)arg;
+	int idx;
 	long r;
 
 	switch (ioctl) {
@@ -981,7 +991,9 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
 		break;
 	}
 	case KVM_S390_STORE_STATUS:
+		idx = srcu_read_lock(&vcpu->kvm->srcu);
 		r = kvm_s390_vcpu_store_status(vcpu, arg);
+		srcu_read_unlock(&vcpu->kvm->srcu, idx);
 		break;
 	case KVM_S390_SET_INITIAL_PSW: {
 		psw_t psw;
-- 
1.8.3.1

[PATCH 5/8] KVM: s390: Allow NULL parameter for kvm_s390_get_regs_rre

[Christian Borntraeger]

From: Thomas Huth <thuth@linux.vnet.ibm.com>

We're not always interested in both registers that are specified
for an RRE instruction. So allow NULL as parameter, too, to indicate
that we do not need the corresponding value.

Signed-off-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Reviewed-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 arch/s390/kvm/kvm-s390.h | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h
index a307a85..b44912a 100644
--- a/arch/s390/kvm/kvm-s390.h
+++ b/arch/s390/kvm/kvm-s390.h
@@ -90,8 +90,10 @@ static inline void kvm_s390_get_base_disp_sse(struct kvm_vcpu *vcpu,
 
 static inline void kvm_s390_get_regs_rre(struct kvm_vcpu *vcpu, int *r1, int *r2)
 {
-	*r1 = (vcpu->arch.sie_block->ipb & 0x00f00000) >> 20;
-	*r2 = (vcpu->arch.sie_block->ipb & 0x000f0000) >> 16;
+	if (r1)
+		*r1 = (vcpu->arch.sie_block->ipb & 0x00f00000) >> 20;
+	if (r2)
+		*r2 = (vcpu->arch.sie_block->ipb & 0x000f0000) >> 16;
 }
 
 static inline u64 kvm_s390_get_base_disp_rsy(struct kvm_vcpu *vcpu)
-- 
1.8.3.1

[PATCH 6/8] KVM: s390: Helper for converting real addresses to absolute

[Christian Borntraeger]

From: Thomas Huth <thuth@linux.vnet.ibm.com>

Added a separate helper function that translates guest real addresses
to guest absolute addresses by applying the prefix of the guest CPU.

Signed-off-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 arch/s390/kvm/gaccess.h | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)

diff --git a/arch/s390/kvm/gaccess.h b/arch/s390/kvm/gaccess.h
index 99d789e..374a439 100644
--- a/arch/s390/kvm/gaccess.h
+++ b/arch/s390/kvm/gaccess.h
@@ -18,20 +18,27 @@
 #include <asm/uaccess.h>
 #include "kvm-s390.h"
 
+/* Convert real to absolute address by applying the prefix of the CPU */
+static inline unsigned long kvm_s390_real_to_abs(struct kvm_vcpu *vcpu,
+						 unsigned long gaddr)
+{
+	unsigned long prefix  = vcpu->arch.sie_block->prefix;
+	if (gaddr < 2 * PAGE_SIZE)
+		gaddr += prefix;
+	else if (gaddr >= prefix && gaddr < prefix + 2 * PAGE_SIZE)
+		gaddr -= prefix;
+	return gaddr;
+}
+
 static inline void __user *__gptr_to_uptr(struct kvm_vcpu *vcpu,
 					  void __user *gptr,
 					  int prefixing)
 {
-	unsigned long prefix  = vcpu->arch.sie_block->prefix;
 	unsigned long gaddr = (unsigned long) gptr;
 	unsigned long uaddr;
 
-	if (prefixing) {
-		if (gaddr < 2 * PAGE_SIZE)
-			gaddr += prefix;
-		else if ((gaddr >= prefix) && (gaddr < prefix + 2 * PAGE_SIZE))
-			gaddr -= prefix;
-	}
+	if (prefixing)
+		gaddr = kvm_s390_real_to_abs(vcpu, gaddr);
 	uaddr = gmap_fault(gaddr, vcpu->arch.gmap);
 	if (IS_ERR_VALUE(uaddr))
 		uaddr = -EFAULT;
-- 
1.8.3.1

[PATCH 7/8] KVM: s390: Implement TEST BLOCK

[Christian Borntraeger]

From: Thomas Huth <thuth@linux.vnet.ibm.com>

This patch provides a simple version for the mandatory TEST BLOCK
instruction interception, so that guests that use this instruction
do not crash anymore.

Signed-off-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Reviewed-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 arch/s390/kvm/priv.c | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/arch/s390/kvm/priv.c b/arch/s390/kvm/priv.c
index 59200ee..6f95994 100644
--- a/arch/s390/kvm/priv.c
+++ b/arch/s390/kvm/priv.c
@@ -128,6 +128,33 @@ static int handle_skey(struct kvm_vcpu *vcpu)
 	return 0;
 }
 
+static int handle_test_block(struct kvm_vcpu *vcpu)
+{
+	unsigned long hva;
+	gpa_t addr;
+	int reg2;
+
+	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
+		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
+
+	kvm_s390_get_regs_rre(vcpu, NULL, &reg2);
+	addr = vcpu->run->s.regs.gprs[reg2] & PAGE_MASK;
+	addr = kvm_s390_real_to_abs(vcpu, addr);
+
+	hva = gfn_to_hva(vcpu->kvm, gpa_to_gfn(addr));
+	if (kvm_is_error_hva(hva))
+		return kvm_s390_inject_program_int(vcpu, PGM_ADDRESSING);
+	/*
+	 * We don't expect errors on modern systems, and do not care
+	 * about storage keys (yet), so let's just clear the page.
+	 */
+	if (clear_user((void __user *)hva, PAGE_SIZE) != 0)
+		return -EFAULT;
+	kvm_s390_set_psw_cc(vcpu, 0);
+	vcpu->run->s.regs.gprs[0] = 0;
+	return 0;
+}
+
 static int handle_tpi(struct kvm_vcpu *vcpu)
 {
 	struct kvm_s390_interrupt_info *inti;
@@ -444,6 +471,7 @@ static const intercept_handler_t b2_handlers[256] = {
 	[0x29] = handle_skey,
 	[0x2a] = handle_skey,
 	[0x2b] = handle_skey,
+	[0x2c] = handle_test_block,
 	[0x30] = handle_io_inst,
 	[0x31] = handle_io_inst,
 	[0x32] = handle_io_inst,
-- 
1.8.3.1

[PATCH 8/8] KVM: s390: Intercept SCK instruction

[Christian Borntraeger]

From: Thomas Huth <thuth@linux.vnet.ibm.com>

Interception of the SET CLOCK instruction is mandatory, so this patch
provides a simple handler for this instruction (by setting up the
"epoch" field in the sie_block).

Signed-off-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Acked-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 arch/s390/kvm/priv.c | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/arch/s390/kvm/priv.c b/arch/s390/kvm/priv.c
index 6f95994..2440602 100644
--- a/arch/s390/kvm/priv.c
+++ b/arch/s390/kvm/priv.c
@@ -30,6 +30,38 @@
 #include "kvm-s390.h"
 #include "trace.h"
 
+/* Handle SCK (SET CLOCK) interception */
+static int handle_set_clock(struct kvm_vcpu *vcpu)
+{
+	struct kvm_vcpu *cpup;
+	s64 hostclk, val;
+	u64 op2;
+	int i;
+
+	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
+		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
+
+	op2 = kvm_s390_get_base_disp_s(vcpu);
+	if (op2 & 7)	/* Operand must be on a doubleword boundary */
+		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
+	if (get_guest(vcpu, val, (u64 __user *) op2))
+		return kvm_s390_inject_program_int(vcpu, PGM_ADDRESSING);
+
+	if (store_tod_clock(&hostclk)) {
+		kvm_s390_set_psw_cc(vcpu, 3);
+		return 0;
+	}
+	val = (val - hostclk) & ~0x3fUL;
+
+	mutex_lock(&vcpu->kvm->lock);
+	kvm_for_each_vcpu(i, cpup, vcpu->kvm)
+		cpup->arch.sie_block->epoch = val;
+	mutex_unlock(&vcpu->kvm->lock);
+
+	kvm_s390_set_psw_cc(vcpu, 0);
+	return 0;
+}
+
 static int handle_set_prefix(struct kvm_vcpu *vcpu)
 {
 	u64 operand2;
@@ -465,6 +497,7 @@ out_exception:
 
 static const intercept_handler_t b2_handlers[256] = {
 	[0x02] = handle_stidp,
+	[0x04] = handle_set_clock,
 	[0x10] = handle_set_prefix,
 	[0x11] = handle_store_prefix,
 	[0x12] = handle_store_cpu_address,
-- 
1.8.3.1

Re: [PATCH 2/8] KVM: s390: Split up __vcpu_run into three parts

[Paolo Bonzini]

Il 12/09/2013 10:33, Christian Borntraeger ha scritto:
>  
>  	vcpu->arch.sie_block->icptcode = 0;
> -	VCPU_EVENT(vcpu, 6, "entering sie flags %x",
> -		   atomic_read(&vcpu->arch.sie_block->cpuflags));
> -	trace_kvm_s390_sie_enter(vcpu,
> -				 atomic_read(&vcpu->arch.sie_block->cpuflags));
> +	cpuflags = atomic_read(&vcpu->arch.sie_block->cpuflags);
> +	VCPU_EVENT(vcpu, 6, "entering sie flags %x", cpuflags);
> +	trace_kvm_s390_sie_enter(vcpu, cpuflags);

I don't have this VCPU_EVENT in kvm/queue, it is after preempt_enable:

        vcpu->arch.sie_block->icptcode = 0;
        preempt_disable();
        kvm_guest_enter();
        preempt_enable();
        VCPU_EVENT(vcpu, 6, "entering sie flags %x",
                   atomic_read(&vcpu->arch.sie_block->cpuflags));
        trace_kvm_s390_sie_enter(vcpu,
                                 atomic_read(&vcpu->arch.sie_block->cpuflags));
        rc = sie64a(vcpu->arch.sie_block, vcpu->run->s.regs.gprs);

Patch missing?  Perhaps you should send a pull request instead as soon as
3.12-rc1 comes out.

Paolo

Re: [PATCH 2/8] KVM: s390: Split up __vcpu_run into three parts

[Christian Borntraeger]

On 12/09/13 10:59, Paolo Bonzini wrote:
> Il 12/09/2013 10:33, Christian Borntraeger ha scritto:
>>  
>>  	vcpu->arch.sie_block->icptcode = 0;
>> -	VCPU_EVENT(vcpu, 6, "entering sie flags %x",
>> -		   atomic_read(&vcpu->arch.sie_block->cpuflags));
>> -	trace_kvm_s390_sie_enter(vcpu,
>> -				 atomic_read(&vcpu->arch.sie_block->cpuflags));
>> +	cpuflags = atomic_read(&vcpu->arch.sie_block->cpuflags);
>> +	VCPU_EVENT(vcpu, 6, "entering sie flags %x", cpuflags);
>> +	trace_kvm_s390_sie_enter(vcpu, cpuflags);
> 
> I don't have this VCPU_EVENT in kvm/queue, it is after preempt_enable:

Huh? This was moved with

commit 2b29a9fdcb92bfc6b6f4c412d71505869de61a56
Author: Dominik Dingel <dingel@linux.vnet.ibm.com>
Date:   Fri Jul 26 15:04:00 2013 +0200

    KVM: s390: move kvm_guest_enter,exit closer to sie


which is clearly in kvm/qeue:

https://git.kernel.org/cgit/virt/kvm/kvm.git/tree/arch/s390/kvm/kvm-s390.c?h=queue


--- snip---
	vcpu->arch.sie_block->icptcode = 0;
	VCPU_EVENT(vcpu, 6, "entering sie flags %x",
		   atomic_read(&vcpu->arch.sie_block->cpuflags));
	trace_kvm_s390_sie_enter(vcpu,
				 atomic_read(&vcpu->arch.sie_block->cpuflags));

	/*
	 * As PF_VCPU will be used in fault handler, between guest_enter
	 * and guest_exit should be no uaccess.
	 */
	preempt_disable();
	kvm_guest_enter();
	preempt_enable();
	rc = sie64a(vcpu->arch.sie_block, vcpu->run->s.regs.gprs);
	kvm_guest_exit();

	VCPU_EVENT(vcpu, 6, "exit sie icptcode %d",
		   vcpu->arch.sie_block->icptcode);
	trace_kvm_s390_sie_exit(vcpu, vcpu->arch.sie_block->icptcode);
--- snip---


> 
>         vcpu->arch.sie_block->icptcode = 0;
>         preempt_disable();
>         kvm_guest_enter();
>         preempt_enable();
>         VCPU_EVENT(vcpu, 6, "entering sie flags %x",
>                    atomic_read(&vcpu->arch.sie_block->cpuflags));
>         trace_kvm_s390_sie_enter(vcpu,
>                                  atomic_read(&vcpu->arch.sie_block->cpuflags));
>         rc = sie64a(vcpu->arch.sie_block, vcpu->run->s.regs.gprs);
> 
> Patch missing?  Perhaps you should send a pull request instead as soon as
> 3.12-rc1 comes out.

For future patches, yes. I will go out for a kernel.org account, but it will certainly
take some time.

Christian

Re: [PATCH 2/8] KVM: s390: Split up __vcpu_run into three parts

[Paolo Bonzini]

Il 12/09/2013 11:09, Christian Borntraeger ha scritto:
>> > I don't have this VCPU_EVENT in kvm/queue, it is after preempt_enable:
> Huh? This was moved with
> 
> commit 2b29a9fdcb92bfc6b6f4c412d71505869de61a56
> Author: Dominik Dingel <dingel@linux.vnet.ibm.com>
> Date:   Fri Jul 26 15:04:00 2013 +0200
> 
>     KVM: s390: move kvm_guest_enter,exit closer to sie

Nevermind...

Paolo

Re: [PATCH 0/8] KVM: s390: fixes and cleanup

[Paolo Bonzini]

Il 12/09/2013 10:33, Christian Borntraeger ha scritto:
> Paolo, Gleb,
> 
> here is a bunch of patch for kvm on s390.
> The first 4 patches restructure the code to protect most of 
> vcpu_run with the kvm->srcu lock. The old code was structured
> in way that adding the lock was more complicated than necessary,
> therefore, the rework.
> 
> The last 4 patches deal with instruction intercept handling.
> 
> Patches are against kvm.git queue. Please apply.
> 
> 
> Thomas Huth (8):
>   KVM: s390: Remove dead "rerun vcpu" code
>   KVM: s390: Split up __vcpu_run into three parts
>   KVM: s390: Push run loop into __vcpu_run
>   KVM: s390: Lock kvm->srcu at the appropriate places
>   KVM: s390: Allow NULL parameter for kvm_s390_get_regs_rre
>   KVM: s390: Helper for converting real addresses to absolute
>   KVM: s390: Implement TEST BLOCK
>   KVM: s390: Intercept SCK instruction
> 
>  arch/s390/include/asm/kvm_host.h |  1 -
>  arch/s390/kvm/diag.c             |  4 +-
>  arch/s390/kvm/gaccess.h          | 21 ++++++----
>  arch/s390/kvm/intercept.c        |  6 ---
>  arch/s390/kvm/interrupt.c        |  3 ++
>  arch/s390/kvm/kvm-s390.c         | 91 ++++++++++++++++++++++++++--------------
>  arch/s390/kvm/kvm-s390.h         |  9 ++--
>  arch/s390/kvm/priv.c             | 61 +++++++++++++++++++++++++++
>  8 files changed, 144 insertions(+), 52 deletions(-)
> 

Applied to kvm/queue for 3.13, thanks.

Paolo