[PATCH/RFC 0/9] Next bunch of KVM/s390x changes for next

[PATCH/RFC 0/9] Next bunch of KVM/s390x changes for next

[Christian Borntraeger]

Paolo, Marcelo, Alex,

this is my current patch queue that I want to send for the
next pull request.
As it contains 3 API extensions
- KVM: s390: introduce post handlers for STSI
- KVM: s390: Create ioctl for Getting/Setting guest storage keys
- KVM: s390: Add MEMOP ioctls for reading/writing guest memory

some time for review is warranted.


Alexander Yarygin (4):
  KVM: s390: Fix low-address protection for real addresses
  KVM: s390: Guest's memory access functions get access registers
  KVM: s390: Optimize paths where get_vcpu_asce() is invoked
  KVM: s390: Add access register mode

Dominik Dingel (1):
  KVM: s390: cleanup jump lables in kvm_arch_init_vm

Ekaterina Tumanova (1):
  KVM: s390: introduce post handlers for STSI

Geert Uytterhoeven (1):
  KVM: s390: Spelling s/intance/instance/

Jason J. Herne (1):
  KVM: s390: Create ioctl for Getting/Setting guest storage keys

Thomas Huth (1):
  KVM: s390: Add MEMOP ioctls for reading/writing guest memory

 Documentation/virtual/kvm/api.txt | 132 +++++++++++++++++
 arch/s390/include/asm/kvm_host.h  |   1 +
 arch/s390/kvm/diag.c              |   4 +-
 arch/s390/kvm/gaccess.c           | 294 +++++++++++++++++++++++++++++++-------
 arch/s390/kvm/gaccess.h           |  21 +--
 arch/s390/kvm/intercept.c         |   4 +-
 arch/s390/kvm/kvm-s390.c          | 217 ++++++++++++++++++++++++++--
 arch/s390/kvm/kvm-s390.h          |  28 +++-
 arch/s390/kvm/priv.c              |  93 +++++++-----
 arch/s390/kvm/sigp.c              |   4 +-
 include/uapi/linux/kvm.h          |  46 ++++++
 11 files changed, 730 insertions(+), 114 deletions(-)

-- 
2.3.0

[PATCH/RFC 1/9] KVM: s390: Spelling s/intance/instance/

[Christian Borntraeger]

From: Geert Uytterhoeven <geert+renesas@glider.be>

Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
Message-Id: <1425932832-6244-1-git-send-email-geert+renesas@glider.be>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 arch/s390/kvm/kvm-s390.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h
index fda3f31..83f32a1 100644
--- a/arch/s390/kvm/kvm-s390.h
+++ b/arch/s390/kvm/kvm-s390.h
@@ -125,7 +125,7 @@ static inline void kvm_s390_set_psw_cc(struct kvm_vcpu *vcpu, unsigned long cc)
 	vcpu->arch.sie_block->gpsw.mask |= cc << 44;
 }
 
-/* test availability of facility in a kvm intance */
+/* test availability of facility in a kvm instance */
 static inline int test_kvm_facility(struct kvm *kvm, unsigned long nr)
 {
 	return __test_facility(nr, kvm->arch.model.fac->mask) &&
-- 
2.3.0

[PATCH/RFC 2/9] KVM: s390: cleanup jump lables in kvm_arch_init_vm

[Christian Borntraeger]

From: Dominik Dingel <dingel@linux.vnet.ibm.com>

As all cleanup functions can handle their respective NULL case
there is no need to have more than one error jump label.

Signed-off-by: Dominik Dingel <dingel@linux.vnet.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 arch/s390/kvm/kvm-s390.c | 14 +++++---------
 1 file changed, 5 insertions(+), 9 deletions(-)

diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index 02e03c8..4075acb 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -897,7 +897,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 
 	kvm->arch.dbf = debug_register(debug_name, 8, 2, 8 * sizeof(long));
 	if (!kvm->arch.dbf)
-		goto out_nodbf;
+		goto out_err;
 
 	/*
 	 * The architectural maximum amount of facilities is 16 kbit. To store
@@ -909,7 +909,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	kvm->arch.model.fac =
 		(struct kvm_s390_fac *) get_zeroed_page(GFP_KERNEL | GFP_DMA);
 	if (!kvm->arch.model.fac)
-		goto out_nofac;
+		goto out_err;
 
 	/* Populate the facility mask initially. */
 	memcpy(kvm->arch.model.fac->mask, S390_lowcore.stfle_fac_list,
@@ -929,7 +929,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	kvm->arch.model.ibc = sclp_get_ibc() & 0x0fff;
 
 	if (kvm_s390_crypto_init(kvm) < 0)
-		goto out_crypto;
+		goto out_err;
 
 	spin_lock_init(&kvm->arch.float_int.lock);
 	INIT_LIST_HEAD(&kvm->arch.float_int.list);
@@ -944,7 +944,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	} else {
 		kvm->arch.gmap = gmap_alloc(current->mm, (1UL << 44) - 1);
 		if (!kvm->arch.gmap)
-			goto out_nogmap;
+			goto out_err;
 		kvm->arch.gmap->private = kvm;
 		kvm->arch.gmap->pfault_enabled = 0;
 	}
@@ -957,15 +957,11 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	spin_lock_init(&kvm->arch.start_stop_lock);
 
 	return 0;
-out_nogmap:
+out_err:
 	kfree(kvm->arch.crypto.crycb);
-out_crypto:
 	free_page((unsigned long)kvm->arch.model.fac);
-out_nofac:
 	debug_unregister(kvm->arch.dbf);
-out_nodbf:
 	free_page((unsigned long)(kvm->arch.sca));
-out_err:
 	return rc;
 }
 
-- 
2.3.0

Re: [PATCH/RFC 2/9] KVM: s390: cleanup jump lables in kvm_arch_init_vm

[Cornelia Huck]

On Mon, 16 Mar 2015 09:51:38 +0100
Christian Borntraeger <borntraeger@de.ibm.com> wrote:

> From: Dominik Dingel <dingel@linux.vnet.ibm.com>
> 
> As all cleanup functions can handle their respective NULL case
> there is no need to have more than one error jump label.
> 
> Signed-off-by: Dominik Dingel <dingel@linux.vnet.ibm.com>
> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
> ---
>  arch/s390/kvm/kvm-s390.c | 14 +++++---------
>  1 file changed, 5 insertions(+), 9 deletions(-)

Acked-by: Cornelia Huck <cornelia.huck@de.ibm.com>

[PATCH/RFC 3/9] KVM: s390: Fix low-address protection for real addresses

[Christian Borntraeger]

From: Alexander Yarygin <yarygin@linux.vnet.ibm.com>

The kvm_s390_check_low_addr_protection() function is used only with real
addresses. According to the POP (the "Low-Address Protection"
paragraph in chapter 3), if the effective address is real or absolute,
the low-address protection procedure should rise PROTECTION exception
only when the low-address protection is enabled in the control register
0 and the address is low.
This patch removes ASCE checks from the function and renames it to
better reflect its behavior.

Cc: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Alexander Yarygin <yarygin@linux.vnet.ibm.com>
Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Reviewed-by: David Hildenbrand <dahi@linux.vnet.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 arch/s390/kvm/gaccess.c | 11 ++++++-----
 arch/s390/kvm/gaccess.h |  2 +-
 arch/s390/kvm/priv.c    |  4 ++--
 3 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/arch/s390/kvm/gaccess.c b/arch/s390/kvm/gaccess.c
index 633fe9b..c230904 100644
--- a/arch/s390/kvm/gaccess.c
+++ b/arch/s390/kvm/gaccess.c
@@ -697,28 +697,29 @@ int guest_translate_address(struct kvm_vcpu *vcpu, unsigned long gva,
 }
 
 /**
- * kvm_s390_check_low_addr_protection - check for low-address protection
- * @ga: Guest address
+ * kvm_s390_check_low_addr_prot_real - check for low-address protection
+ * @gra: Guest real address
  *
  * Checks whether an address is subject to low-address protection and set
  * up vcpu->arch.pgm accordingly if necessary.
  *
  * Return: 0 if no protection exception, or PGM_PROTECTION if protected.
  */
-int kvm_s390_check_low_addr_protection(struct kvm_vcpu *vcpu, unsigned long ga)
+int kvm_s390_check_low_addr_prot_real(struct kvm_vcpu *vcpu, unsigned long gra)
 {
 	struct kvm_s390_pgm_info *pgm = &vcpu->arch.pgm;
 	psw_t *psw = &vcpu->arch.sie_block->gpsw;
 	struct trans_exc_code_bits *tec_bits;
+	union ctlreg0 ctlreg0 = {.val = vcpu->arch.sie_block->gcr[0]};
 
-	if (!is_low_address(ga) || !low_address_protection_enabled(vcpu))
+	if (!ctlreg0.lap || !is_low_address(gra))
 		return 0;
 
 	memset(pgm, 0, sizeof(*pgm));
 	tec_bits = (struct trans_exc_code_bits *)&pgm->trans_exc_code;
 	tec_bits->fsi = FSI_STORE;
 	tec_bits->as = psw_bits(*psw).as;
-	tec_bits->addr = ga >> PAGE_SHIFT;
+	tec_bits->addr = gra >> PAGE_SHIFT;
 	pgm->code = PGM_PROTECTION;
 
 	return pgm->code;
diff --git a/arch/s390/kvm/gaccess.h b/arch/s390/kvm/gaccess.h
index 0149cf1..20de77e 100644
--- a/arch/s390/kvm/gaccess.h
+++ b/arch/s390/kvm/gaccess.h
@@ -330,6 +330,6 @@ int read_guest_real(struct kvm_vcpu *vcpu, unsigned long gra, void *data,
 void ipte_lock(struct kvm_vcpu *vcpu);
 void ipte_unlock(struct kvm_vcpu *vcpu);
 int ipte_lock_held(struct kvm_vcpu *vcpu);
-int kvm_s390_check_low_addr_protection(struct kvm_vcpu *vcpu, unsigned long ga);
+int kvm_s390_check_low_addr_prot_real(struct kvm_vcpu *vcpu, unsigned long gra);
 
 #endif /* __KVM_S390_GACCESS_H */
diff --git a/arch/s390/kvm/priv.c b/arch/s390/kvm/priv.c
index b982fbc..5f26425 100644
--- a/arch/s390/kvm/priv.c
+++ b/arch/s390/kvm/priv.c
@@ -207,7 +207,7 @@ static int handle_test_block(struct kvm_vcpu *vcpu)
 	kvm_s390_get_regs_rre(vcpu, NULL, &reg2);
 	addr = vcpu->run->s.regs.gprs[reg2] & PAGE_MASK;
 	addr = kvm_s390_logical_to_effective(vcpu, addr);
-	if (kvm_s390_check_low_addr_protection(vcpu, addr))
+	if (kvm_s390_check_low_addr_prot_real(vcpu, addr))
 		return kvm_s390_inject_prog_irq(vcpu, &vcpu->arch.pgm);
 	addr = kvm_s390_real_to_abs(vcpu, addr);
 
@@ -680,7 +680,7 @@ static int handle_pfmf(struct kvm_vcpu *vcpu)
 	}
 
 	if (vcpu->run->s.regs.gprs[reg1] & PFMF_CF) {
-		if (kvm_s390_check_low_addr_protection(vcpu, start))
+		if (kvm_s390_check_low_addr_prot_real(vcpu, start))
 			return kvm_s390_inject_prog_irq(vcpu, &vcpu->arch.pgm);
 	}
 
-- 
2.3.0

Re: [PATCH/RFC 3/9] KVM: s390: Fix low-address protection for real addresses

[Cornelia Huck]

On Mon, 16 Mar 2015 09:51:39 +0100
Christian Borntraeger <borntraeger@de.ibm.com> wrote:

> From: Alexander Yarygin <yarygin@linux.vnet.ibm.com>
> 
> The kvm_s390_check_low_addr_protection() function is used only with real
> addresses. According to the POP (the "Low-Address Protection"
> paragraph in chapter 3), if the effective address is real or absolute,
> the low-address protection procedure should rise PROTECTION exception

s/rise/raise a/

> only when the low-address protection is enabled in the control register
> 0 and the address is low.
> This patch removes ASCE checks from the function and renames it to
> better reflect its behavior.
> 
> Cc: Thomas Huth <thuth@linux.vnet.ibm.com>
> Signed-off-by: Alexander Yarygin <yarygin@linux.vnet.ibm.com>
> Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
> Reviewed-by: David Hildenbrand <dahi@linux.vnet.ibm.com>
> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
> ---
>  arch/s390/kvm/gaccess.c | 11 ++++++-----
>  arch/s390/kvm/gaccess.h |  2 +-
>  arch/s390/kvm/priv.c    |  4 ++--
>  3 files changed, 9 insertions(+), 8 deletions(-)

Acked-by: Cornelia Huck <cornelia.huck@de.ibm.com>

[PATCH/RFC 4/9] KVM: s390: Add MEMOP ioctls for reading/writing guest memory

[Christian Borntraeger]

From: Thomas Huth <thuth@linux.vnet.ibm.com>

On s390, we've got to make sure to hold the IPTE lock while accessing
logical memory. So let's add an ioctl for reading and writing logical
memory to provide this feature for userspace, too.
The maximum transfer size of this call is limited to 64kB to prevent
that the guest can trigger huge copy_from/to_user transfers. QEMU
currently only requests up to one or two pages so far, so 16*4kB seems
to be a reasonable limit here.

Signed-off-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 Documentation/virtual/kvm/api.txt | 46 ++++++++++++++++++++++++
 arch/s390/kvm/gaccess.c           | 22 ++++++++++++
 arch/s390/kvm/gaccess.h           |  2 ++
 arch/s390/kvm/kvm-s390.c          | 74 +++++++++++++++++++++++++++++++++++++++
 include/uapi/linux/kvm.h          | 21 +++++++++++
 5 files changed, 165 insertions(+)

diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index ee47998e..f03178d 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2716,6 +2716,52 @@ The fields in each entry are defined as follows:
    eax, ebx, ecx, edx: the values returned by the cpuid instruction for
          this function/index combination
 
+4.89 KVM_S390_MEM_OP
+
+Capability: KVM_CAP_S390_MEM_OP
+Architectures: s390
+Type: vcpu ioctl
+Parameters: struct kvm_s390_mem_op (in)
+Returns: = 0 on success,
+         < 0 on generic error (e.g. -EFAULT or -ENOMEM),
+         > 0 if an exception occurred while walking the page tables
+
+Read or write data from/to the logical (virtual) memory of a VPCU.
+
+Parameters are specified via the following structure:
+
+struct kvm_s390_mem_op {
+	__u64 gaddr;		/* the guest address */
+	__u64 flags;		/* arch specific flags */
+	__u32 size;		/* amount of bytes */
+	__u32 op;		/* type of operation */
+	__u64 buf;		/* buffer in userspace */
+	__u8 ar;		/* the access register number */
+	__u8 reserved[31];	/* should be set to 0 */
+};
+
+The type of operation is specified in the "op" field. It is either
+KVM_S390_MEMOP_LOGICAL_READ for reading from logical memory space or
+KVM_S390_MEMOP_LOGICAL_WRITE for writing to logical memory space. The
+KVM_S390_MEMOP_F_CHECK_ONLY flag can be set in the "flags" field to check
+whether the corresponding memory access would create an access exception
+(without touching the data in the memory at the destination). In case an
+access exception occurred while walking the MMU tables of the guest, the
+ioctl returns a positive error number to indicate the type of exception.
+This exception is also raised directly at the corresponding VCPU if the
+flag KVM_S390_MEMOP_F_INJECT_EXCEPTION is set in the "flags" field.
+
+The start address of the memory region has to be specified in the "gaddr"
+field, and the length of the region in the "size" field. "buf" is the buffer
+supplied by the userspace application where the read data should be written
+to for KVM_S390_MEMOP_LOGICAL_READ, or where the data that should be written
+is stored for a KVM_S390_MEMOP_LOGICAL_WRITE. "buf" is unused and can be NULL
+when KVM_S390_MEMOP_F_CHECK_ONLY is specified. "ar" designates the access
+register number to be used.
+
+The "reserved" field is meant for future extensions. It is not used by
+KVM with the currently defined set of flags.
+
 5. The kvm_run structure
 ------------------------
 
diff --git a/arch/s390/kvm/gaccess.c b/arch/s390/kvm/gaccess.c
index c230904..04a7c67 100644
--- a/arch/s390/kvm/gaccess.c
+++ b/arch/s390/kvm/gaccess.c
@@ -697,6 +697,28 @@ int guest_translate_address(struct kvm_vcpu *vcpu, unsigned long gva,
 }
 
 /**
+ * check_gva_range - test a range of guest virtual addresses for accessibility
+ */
+int check_gva_range(struct kvm_vcpu *vcpu, unsigned long gva,
+		    unsigned long length, int is_write)
+{
+	unsigned long gpa;
+	unsigned long currlen;
+	int rc = 0;
+
+	ipte_lock(vcpu);
+	while (length > 0 && !rc) {
+		currlen = min(length, PAGE_SIZE - (gva % PAGE_SIZE));
+		rc = guest_translate_address(vcpu, gva, &gpa, is_write);
+		gva += currlen;
+		length -= currlen;
+	}
+	ipte_unlock(vcpu);
+
+	return rc;
+}
+
+/**
  * kvm_s390_check_low_addr_prot_real - check for low-address protection
  * @gra: Guest real address
  *
diff --git a/arch/s390/kvm/gaccess.h b/arch/s390/kvm/gaccess.h
index 20de77e..d0de110 100644
--- a/arch/s390/kvm/gaccess.h
+++ b/arch/s390/kvm/gaccess.h
@@ -157,6 +157,8 @@ int read_guest_lc(struct kvm_vcpu *vcpu, unsigned long gra, void *data,
 
 int guest_translate_address(struct kvm_vcpu *vcpu, unsigned long gva,
 			    unsigned long *gpa, int write);
+int check_gva_range(struct kvm_vcpu *vcpu, unsigned long gva,
+		    unsigned long length, int is_write);
 
 int access_guest(struct kvm_vcpu *vcpu, unsigned long ga, void *data,
 		 unsigned long len, int write);
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index 4075acb..92855e2 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -25,6 +25,7 @@
 #include <linux/random.h>
 #include <linux/slab.h>
 #include <linux/timer.h>
+#include <linux/vmalloc.h>
 #include <asm/asm-offsets.h>
 #include <asm/lowcore.h>
 #include <asm/pgtable.h>
@@ -38,6 +39,8 @@
 #include "trace.h"
 #include "trace-s390.h"
 
+#define MEM_OP_MAX_SIZE 65536	/* Maximum transfer size for KVM_S390_MEM_OP */
+
 #define VCPU_STAT(x) offsetof(struct kvm_vcpu, stat.x), KVM_STAT_VCPU
 
 struct kvm_stats_debugfs_item debugfs_entries[] = {
@@ -177,6 +180,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_S390_USER_SIGP:
 		r = 1;
 		break;
+	case KVM_CAP_S390_MEM_OP:
+		r = MEM_OP_MAX_SIZE;
+		break;
 	case KVM_CAP_NR_VCPUS:
 	case KVM_CAP_MAX_VCPUS:
 		r = KVM_MAX_VCPUS;
@@ -2185,6 +2191,65 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
 	return r;
 }
 
+static long kvm_s390_guest_mem_op(struct kvm_vcpu *vcpu,
+				  struct kvm_s390_mem_op *mop)
+{
+	void __user *uaddr = (void __user *)mop->buf;
+	void *tmpbuf = NULL;
+	int r, srcu_idx;
+	const u64 supported_flags = KVM_S390_MEMOP_F_INJECT_EXCEPTION
+				    | KVM_S390_MEMOP_F_CHECK_ONLY;
+
+	if (mop->flags & ~supported_flags)
+		return -EINVAL;
+
+	if (mop->size > MEM_OP_MAX_SIZE)
+		return -E2BIG;
+
+	if (!(mop->flags & KVM_S390_MEMOP_F_CHECK_ONLY)) {
+		tmpbuf = vmalloc(mop->size);
+		if (!tmpbuf)
+			return -ENOMEM;
+	}
+
+	srcu_idx = srcu_read_lock(&vcpu->kvm->srcu);
+
+	switch (mop->op) {
+	case KVM_S390_MEMOP_LOGICAL_READ:
+		if (mop->flags & KVM_S390_MEMOP_F_CHECK_ONLY) {
+			r = check_gva_range(vcpu, mop->gaddr, mop->size, false);
+			break;
+		}
+		r = read_guest(vcpu, mop->gaddr, tmpbuf, mop->size);
+		if (r == 0) {
+			if (copy_to_user(uaddr, tmpbuf, mop->size))
+				r = -EFAULT;
+		}
+		break;
+	case KVM_S390_MEMOP_LOGICAL_WRITE:
+		if (mop->flags & KVM_S390_MEMOP_F_CHECK_ONLY) {
+			r = check_gva_range(vcpu, mop->gaddr, mop->size, true);
+			break;
+		}
+		if (copy_from_user(tmpbuf, uaddr, mop->size)) {
+			r = -EFAULT;
+			break;
+		}
+		r = write_guest(vcpu, mop->gaddr, tmpbuf, mop->size);
+		break;
+	default:
+		r = -EINVAL;
+	}
+
+	srcu_read_unlock(&vcpu->kvm->srcu, srcu_idx);
+
+	if (r > 0 && (mop->flags & KVM_S390_MEMOP_F_INJECT_EXCEPTION) != 0)
+		kvm_s390_inject_prog_irq(vcpu, &vcpu->arch.pgm);
+
+	vfree(tmpbuf);
+	return r;
+}
+
 long kvm_arch_vcpu_ioctl(struct file *filp,
 			 unsigned int ioctl, unsigned long arg)
 {
@@ -2284,6 +2349,15 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
 		r = kvm_vcpu_ioctl_enable_cap(vcpu, &cap);
 		break;
 	}
+	case KVM_S390_MEM_OP: {
+		struct kvm_s390_mem_op mem_op;
+
+		if (copy_from_user(&mem_op, argp, sizeof(mem_op)) == 0)
+			r = kvm_s390_guest_mem_op(vcpu, &mem_op);
+		else
+			r = -EFAULT;
+		break;
+	}
 	default:
 		r = -ENOTTY;
 	}
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 82634a4..74bbd9a 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -365,6 +365,24 @@ struct kvm_translation {
 	__u8  pad[5];
 };
 
+/* for KVM_S390_MEM_OP */
+struct kvm_s390_mem_op {
+	/* in */
+	__u64 gaddr;		/* the guest address */
+	__u64 flags;		/* arch specific flags */
+	__u32 size;		/* amount of bytes */
+	__u32 op;		/* type of operation */
+	__u64 buf;		/* buffer in userspace */
+	__u8 ar;		/* the access register number */
+	__u8 reserved[31];	/* should be set to 0 */
+};
+/* types for kvm_s390_mem_op->op */
+#define KVM_S390_MEMOP_LOGICAL_READ	0
+#define KVM_S390_MEMOP_LOGICAL_WRITE	1
+/* flags for kvm_s390_mem_op->flags */
+#define KVM_S390_MEMOP_F_CHECK_ONLY		(1ULL << 0)
+#define KVM_S390_MEMOP_F_INJECT_EXCEPTION	(1ULL << 1)
+
 /* for KVM_INTERRUPT */
 struct kvm_interrupt {
 	/* in */
@@ -761,6 +779,7 @@ struct kvm_ppc_smmu_info {
 #define KVM_CAP_CHECK_EXTENSION_VM 105
 #define KVM_CAP_S390_USER_SIGP 106
 #define KVM_CAP_S390_VECTOR_REGISTERS 107
+#define KVM_CAP_S390_MEM_OP 108
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
@@ -1136,6 +1155,8 @@ struct kvm_s390_ucas_mapping {
 #define KVM_ARM_VCPU_INIT	  _IOW(KVMIO,  0xae, struct kvm_vcpu_init)
 #define KVM_ARM_PREFERRED_TARGET  _IOR(KVMIO,  0xaf, struct kvm_vcpu_init)
 #define KVM_GET_REG_LIST	  _IOWR(KVMIO, 0xb0, struct kvm_reg_list)
+/* Available with KVM_CAP_S390_MEM_OP */
+#define KVM_S390_MEM_OP		  _IOW(KVMIO,  0xb1, struct kvm_s390_mem_op)
 
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3		(1 << 1)
-- 
2.3.0

Re: [PATCH/RFC 4/9] KVM: s390: Add MEMOP ioctls for reading/writing guest memory

[Cornelia Huck]

On Mon, 16 Mar 2015 09:51:40 +0100
Christian Borntraeger <borntraeger@de.ibm.com> wrote:

> From: Thomas Huth <thuth@linux.vnet.ibm.com>
> 
> On s390, we've got to make sure to hold the IPTE lock while accessing
> logical memory. So let's add an ioctl for reading and writing logical
> memory to provide this feature for userspace, too.
> The maximum transfer size of this call is limited to 64kB to prevent
> that the guest can trigger huge copy_from/to_user transfers. QEMU
> currently only requests up to one or two pages so far, so 16*4kB seems
> to be a reasonable limit here.
> 
> Signed-off-by: Thomas Huth <thuth@linux.vnet.ibm.com>
> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
> ---
>  Documentation/virtual/kvm/api.txt | 46 ++++++++++++++++++++++++
>  arch/s390/kvm/gaccess.c           | 22 ++++++++++++
>  arch/s390/kvm/gaccess.h           |  2 ++
>  arch/s390/kvm/kvm-s390.c          | 74 +++++++++++++++++++++++++++++++++++++++
>  include/uapi/linux/kvm.h          | 21 +++++++++++
>  5 files changed, 165 insertions(+)
> 
> diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
> index ee47998e..f03178d 100644
> --- a/Documentation/virtual/kvm/api.txt
> +++ b/Documentation/virtual/kvm/api.txt
> @@ -2716,6 +2716,52 @@ The fields in each entry are defined as follows:
>     eax, ebx, ecx, edx: the values returned by the cpuid instruction for
>           this function/index combination
> 
> +4.89 KVM_S390_MEM_OP
> +
> +Capability: KVM_CAP_S390_MEM_OP
> +Architectures: s390
> +Type: vcpu ioctl
> +Parameters: struct kvm_s390_mem_op (in)
> +Returns: = 0 on success,
> +         < 0 on generic error (e.g. -EFAULT or -ENOMEM),
> +         > 0 if an exception occurred while walking the page tables
> +
> +Read or write data from/to the logical (virtual) memory of a VPCU.
> +
> +Parameters are specified via the following structure:
> +
> +struct kvm_s390_mem_op {
> +	__u64 gaddr;		/* the guest address */
> +	__u64 flags;		/* arch specific flags */

Drop the "arch"? This is a s390-only ioctl :)

> +	__u32 size;		/* amount of bytes */
> +	__u32 op;		/* type of operation */
> +	__u64 buf;		/* buffer in userspace */
> +	__u8 ar;		/* the access register number */

This makes me wonder whether the patches should be reordered to
introduce access register mode first: It seems strange that you can
specify a parameter that is ignored. Same for the STSI patch.

> +	__u8 reserved[31];	/* should be set to 0 */
> +};
> +
> +The type of operation is specified in the "op" field. It is either
> +KVM_S390_MEMOP_LOGICAL_READ for reading from logical memory space or
> +KVM_S390_MEMOP_LOGICAL_WRITE for writing to logical memory space. The
> +KVM_S390_MEMOP_F_CHECK_ONLY flag can be set in the "flags" field to check
> +whether the corresponding memory access would create an access exception
> +(without touching the data in the memory at the destination). In case an
> +access exception occurred while walking the MMU tables of the guest, the
> +ioctl returns a positive error number to indicate the type of exception.
> +This exception is also raised directly at the corresponding VCPU if the
> +flag KVM_S390_MEMOP_F_INJECT_EXCEPTION is set in the "flags" field.
> +
> +The start address of the memory region has to be specified in the "gaddr"
> +field, and the length of the region in the "size" field. "buf" is the buffer
> +supplied by the userspace application where the read data should be written
> +to for KVM_S390_MEMOP_LOGICAL_READ, or where the data that should be written
> +is stored for a KVM_S390_MEMOP_LOGICAL_WRITE. "buf" is unused and can be NULL
> +when KVM_S390_MEMOP_F_CHECK_ONLY is specified. "ar" designates the access
> +register number to be used.
> +
> +The "reserved" field is meant for future extensions. It is not used by
> +KVM with the currently defined set of flags.
> +
>  5. The kvm_run structure
>  ------------------------
> 

Re: [PATCH/RFC 4/9] KVM: s390: Add MEMOP ioctls for reading/writing guest memory

[Christian Borntraeger]

Am 16.03.2015 um 13:16 schrieb Cornelia Huck:
> On Mon, 16 Mar 2015 09:51:40 +0100
> Christian Borntraeger <borntraeger@de.ibm.com> wrote:
> 
>> From: Thomas Huth <thuth@linux.vnet.ibm.com>
>>
>> On s390, we've got to make sure to hold the IPTE lock while accessing
>> logical memory. So let's add an ioctl for reading and writing logical
>> memory to provide this feature for userspace, too.
>> The maximum transfer size of this call is limited to 64kB to prevent
>> that the guest can trigger huge copy_from/to_user transfers. QEMU
>> currently only requests up to one or two pages so far, so 16*4kB seems
>> to be a reasonable limit here.
>>
>> Signed-off-by: Thomas Huth <thuth@linux.vnet.ibm.com>
>> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
>> ---
>>  Documentation/virtual/kvm/api.txt | 46 ++++++++++++++++++++++++
>>  arch/s390/kvm/gaccess.c           | 22 ++++++++++++
>>  arch/s390/kvm/gaccess.h           |  2 ++
>>  arch/s390/kvm/kvm-s390.c          | 74 +++++++++++++++++++++++++++++++++++++++
>>  include/uapi/linux/kvm.h          | 21 +++++++++++
>>  5 files changed, 165 insertions(+)
>>
>> diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
>> index ee47998e..f03178d 100644
>> --- a/Documentation/virtual/kvm/api.txt
>> +++ b/Documentation/virtual/kvm/api.txt
>> @@ -2716,6 +2716,52 @@ The fields in each entry are defined as follows:
>>     eax, ebx, ecx, edx: the values returned by the cpuid instruction for
>>           this function/index combination
>>
>> +4.89 KVM_S390_MEM_OP
>> +
>> +Capability: KVM_CAP_S390_MEM_OP
>> +Architectures: s390
>> +Type: vcpu ioctl
>> +Parameters: struct kvm_s390_mem_op (in)
>> +Returns: = 0 on success,
>> +         < 0 on generic error (e.g. -EFAULT or -ENOMEM),
>> +         > 0 if an exception occurred while walking the page tables
>> +
>> +Read or write data from/to the logical (virtual) memory of a VPCU.
>> +
>> +Parameters are specified via the following structure:
>> +
>> +struct kvm_s390_mem_op {
>> +	__u64 gaddr;		/* the guest address */
>> +	__u64 flags;		/* arch specific flags */
> 
> Drop the "arch"? This is a s390-only ioctl :)

Will fixup.

> 
>> +	__u32 size;		/* amount of bytes */
>> +	__u32 op;		/* type of operation */
>> +	__u64 buf;		/* buffer in userspace */
>> +	__u8 ar;		/* the access register number */
> 
> This makes me wonder whether the patches should be reordered to
> introduce access register mode first: It seems strange that you can
> specify a parameter that is ignored. Same for the STSI patch.

Yes make sense. Will be some ripple when reordering but certainly doable.


> 
>> +	__u8 reserved[31];	/* should be set to 0 */
>> +};
>> +
>> +The type of operation is specified in the "op" field. It is either
>> +KVM_S390_MEMOP_LOGICAL_READ for reading from logical memory space or
>> +KVM_S390_MEMOP_LOGICAL_WRITE for writing to logical memory space. The
>> +KVM_S390_MEMOP_F_CHECK_ONLY flag can be set in the "flags" field to check
>> +whether the corresponding memory access would create an access exception
>> +(without touching the data in the memory at the destination). In case an
>> +access exception occurred while walking the MMU tables of the guest, the
>> +ioctl returns a positive error number to indicate the type of exception.
>> +This exception is also raised directly at the corresponding VCPU if the
>> +flag KVM_S390_MEMOP_F_INJECT_EXCEPTION is set in the "flags" field.
>> +
>> +The start address of the memory region has to be specified in the "gaddr"
>> +field, and the length of the region in the "size" field. "buf" is the buffer
>> +supplied by the userspace application where the read data should be written
>> +to for KVM_S390_MEMOP_LOGICAL_READ, or where the data that should be written
>> +is stored for a KVM_S390_MEMOP_LOGICAL_WRITE. "buf" is unused and can be NULL
>> +when KVM_S390_MEMOP_F_CHECK_ONLY is specified. "ar" designates the access
>> +register number to be used.
>> +
>> +The "reserved" field is meant for future extensions. It is not used by
>> +KVM with the currently defined set of flags.
>> +
>>  5. The kvm_run structure
>>  ------------------------
>>
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH/RFC 4/9] KVM: s390: Add MEMOP ioctls for reading/writing guest memory

[Marcelo Tosatti]

On Mon, Mar 16, 2015 at 09:51:40AM +0100, Christian Borntraeger wrote:
> From: Thomas Huth <thuth@linux.vnet.ibm.com>
> 
> On s390, we've got to make sure to hold the IPTE lock while accessing
> logical memory. So let's add an ioctl for reading and writing logical
> memory to provide this feature for userspace, too.
> The maximum transfer size of this call is limited to 64kB to prevent
> that the guest can trigger huge copy_from/to_user transfers. QEMU
> currently only requests up to one or two pages so far, so 16*4kB seems
> to be a reasonable limit here.
> 
> Signed-off-by: Thomas Huth <thuth@linux.vnet.ibm.com>
> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
> ---
>  Documentation/virtual/kvm/api.txt | 46 ++++++++++++++++++++++++
>  arch/s390/kvm/gaccess.c           | 22 ++++++++++++
>  arch/s390/kvm/gaccess.h           |  2 ++
>  arch/s390/kvm/kvm-s390.c          | 74 +++++++++++++++++++++++++++++++++++++++
>  include/uapi/linux/kvm.h          | 21 +++++++++++
>  5 files changed, 165 insertions(+)
> 
> diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
> index ee47998e..f03178d 100644
> --- a/Documentation/virtual/kvm/api.txt
> +++ b/Documentation/virtual/kvm/api.txt
> @@ -2716,6 +2716,52 @@ The fields in each entry are defined as follows:
>     eax, ebx, ecx, edx: the values returned by the cpuid instruction for
>           this function/index combination
>  
> +4.89 KVM_S390_MEM_OP
> +
> +Capability: KVM_CAP_S390_MEM_OP
> +Architectures: s390
> +Type: vcpu ioctl
> +Parameters: struct kvm_s390_mem_op (in)
> +Returns: = 0 on success,
> +         < 0 on generic error (e.g. -EFAULT or -ENOMEM),
> +         > 0 if an exception occurred while walking the page tables
> +
> +Read or write data from/to the logical (virtual) memory of a VPCU.
> +
> +Parameters are specified via the following structure:
> +
> +struct kvm_s390_mem_op {
> +	__u64 gaddr;		/* the guest address */
> +	__u64 flags;		/* arch specific flags */
> +	__u32 size;		/* amount of bytes */
> +	__u32 op;		/* type of operation */
> +	__u64 buf;		/* buffer in userspace */
> +	__u8 ar;		/* the access register number */
> +	__u8 reserved[31];	/* should be set to 0 */
> +};
> +
> +The type of operation is specified in the "op" field. It is either
> +KVM_S390_MEMOP_LOGICAL_READ for reading from logical memory space or
> +KVM_S390_MEMOP_LOGICAL_WRITE for writing to logical memory space. The
> +KVM_S390_MEMOP_F_CHECK_ONLY flag can be set in the "flags" field to check
> +whether the corresponding memory access would create an access exception
> +(without touching the data in the memory at the destination). In case an
> +access exception occurred while walking the MMU tables of the guest, the
> +ioctl returns a positive error number to indicate the type of exception.
> +This exception is also raised directly at the corresponding VCPU if the
> +flag KVM_S390_MEMOP_F_INJECT_EXCEPTION is set in the "flags" field.
> +
> +The start address of the memory region has to be specified in the "gaddr"
> +field, and the length of the region in the "size" field. "buf" is the buffer
> +supplied by the userspace application where the read data should be written
> +to for KVM_S390_MEMOP_LOGICAL_READ, or where the data that should be written
> +is stored for a KVM_S390_MEMOP_LOGICAL_WRITE. "buf" is unused and can be NULL
> +when KVM_S390_MEMOP_F_CHECK_ONLY is specified. "ar" designates the access
> +register number to be used.
> +
> +The "reserved" field is meant for future extensions. It is not used by
> +KVM with the currently defined set of flags.
> +
>  5. The kvm_run structure
>  ------------------------
>  
> diff --git a/arch/s390/kvm/gaccess.c b/arch/s390/kvm/gaccess.c
> index c230904..04a7c67 100644
> --- a/arch/s390/kvm/gaccess.c
> +++ b/arch/s390/kvm/gaccess.c
> @@ -697,6 +697,28 @@ int guest_translate_address(struct kvm_vcpu *vcpu, unsigned long gva,
>  }
>  
>  /**
> + * check_gva_range - test a range of guest virtual addresses for accessibility
> + */
> +int check_gva_range(struct kvm_vcpu *vcpu, unsigned long gva,
> +		    unsigned long length, int is_write)
> +{
> +	unsigned long gpa;
> +	unsigned long currlen;
> +	int rc = 0;
> +
> +	ipte_lock(vcpu);
> +	while (length > 0 && !rc) {
> +		currlen = min(length, PAGE_SIZE - (gva % PAGE_SIZE));
> +		rc = guest_translate_address(vcpu, gva, &gpa, is_write);
> +		gva += currlen;
> +		length -= currlen;
> +	}
> +	ipte_unlock(vcpu);
> +
> +	return rc;
> +}

What i was wondering is why you can't translate the address
in the kernel and let userspace perform the actual read/write?

[PATCH/RFC 5/9] KVM: s390: introduce post handlers for STSI

[Christian Borntraeger]

From: Ekaterina Tumanova <tumanova@linux.vnet.ibm.com>

The Store System Information (STSI) instruction currently collects all
information it relays to the caller in the kernel. Some information,
however, is only available in user space. An example of this is the
guest name: The kernel always sets "KVMGuest", but user space knows the
actual guest name.

This patch introduces a new exit, KVM_EXIT_S390_STSI, guarded by a
capability that can be enabled by user space if it wants to be able to
insert such data. User space will be provided with the target buffer
and the requested STSI function code.

Reviewed-by: Eric Farman <farman@linux.vnet.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Ekaterina Tumanova <tumanova@linux.vnet.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 Documentation/virtual/kvm/api.txt | 28 ++++++++++++++++++++++++++++
 arch/s390/include/asm/kvm_host.h  |  1 +
 arch/s390/kvm/kvm-s390.c          |  5 +++++
 arch/s390/kvm/priv.c              | 16 +++++++++++++++-
 include/uapi/linux/kvm.h          | 11 +++++++++++
 5 files changed, 60 insertions(+), 1 deletion(-)

diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index f03178d..74d18b6 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -3304,3 +3304,31 @@ Returns: 0 on success, negative value on error
 Allows use of the vector registers introduced with z13 processor, and
 provides for the synchronization between host and user space.  Will
 return -EINVAL if the machine does not support vectors.
+
+7.4 KVM_CAP_S390_USER_STSI
+
+Architectures: s390
+Parameters: none
+
+This capability allows post-handlers for the STSI instruction. After
+initial handling in the kernel, KVM exits to user space with
+KVM_EXIT_S390_STSI to allow user space to insert further data.
+
+Before exiting to userspace, kvm handlers should fill in s390_stsi field of
+vcpu->run:
+struct {
+	__u64 addr;
+	__u8 ar;
+	__u8 reserved;
+	__u8 fc;
+	__u8 sel1;
+	__u16 sel2;
+} s390_stsi;
+
+@addr - guest address of STSI SYSIB
+@fc   - function code
+@sel1 - selector 1
+@sel2 - selector 2
+@ar   - access register number
+
+KVM handlers should exit to userspace with rc = -EREMOTE.
diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
index 347a333..2356a8c 100644
--- a/arch/s390/include/asm/kvm_host.h
+++ b/arch/s390/include/asm/kvm_host.h
@@ -565,6 +565,7 @@ struct kvm_arch{
 	int use_vectors;
 	int user_cpu_state_ctrl;
 	int user_sigp;
+	int user_stsi;
 	struct s390_io_adapter *adapters[MAX_S390_IO_ADAPTERS];
 	wait_queue_head_t ipte_wq;
 	int ipte_lock_count;
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index 92855e2..1f5baf4 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -178,6 +178,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_VM_ATTRIBUTES:
 	case KVM_CAP_MP_STATE:
 	case KVM_CAP_S390_USER_SIGP:
+	case KVM_CAP_S390_USER_STSI:
 		r = 1;
 		break;
 	case KVM_CAP_S390_MEM_OP:
@@ -280,6 +281,10 @@ static int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
 		kvm->arch.use_vectors = MACHINE_HAS_VX;
 		r = MACHINE_HAS_VX ? 0 : -EINVAL;
 		break;
+	case KVM_CAP_S390_USER_STSI:
+		kvm->arch.user_stsi = 1;
+		r = 0;
+		break;
 	default:
 		r = -EINVAL;
 		break;
diff --git a/arch/s390/kvm/priv.c b/arch/s390/kvm/priv.c
index 5f26425..211102c 100644
--- a/arch/s390/kvm/priv.c
+++ b/arch/s390/kvm/priv.c
@@ -488,6 +488,16 @@ static void handle_stsi_3_2_2(struct kvm_vcpu *vcpu, struct sysinfo_3_2_2 *mem)
 	ASCEBC(mem->vm[0].cpi, 16);
 }
 
+static void insert_stsi_usr_data(struct kvm_vcpu *vcpu, u64 addr, u8 fc,
+				 u8 sel1, u16 sel2)
+{
+	vcpu->run->exit_reason = KVM_EXIT_S390_STSI;
+	vcpu->run->s390_stsi.addr = addr;
+	vcpu->run->s390_stsi.fc = fc;
+	vcpu->run->s390_stsi.sel1 = sel1;
+	vcpu->run->s390_stsi.sel2 = sel2;
+}
+
 static int handle_stsi(struct kvm_vcpu *vcpu)
 {
 	int fc = (vcpu->run->s.regs.gprs[0] & 0xf0000000) >> 28;
@@ -547,11 +557,15 @@ static int handle_stsi(struct kvm_vcpu *vcpu)
 		rc = kvm_s390_inject_prog_cond(vcpu, rc);
 		goto out;
 	}
+	if (vcpu->kvm->arch.user_stsi) {
+		insert_stsi_usr_data(vcpu, operand2, fc, sel1, sel2);
+		rc = -EREMOTE;
+	}
 	trace_kvm_s390_handle_stsi(vcpu, fc, sel1, sel2, operand2);
 	free_page(mem);
 	kvm_s390_set_psw_cc(vcpu, 0);
 	vcpu->run->s.regs.gprs[0] = 0;
-	return 0;
+	return rc;
 out_no_data:
 	kvm_s390_set_psw_cc(vcpu, 3);
 out:
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 74bbd9a..279ffd2 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -172,6 +172,7 @@ struct kvm_pit_config {
 #define KVM_EXIT_S390_TSCH        22
 #define KVM_EXIT_EPR              23
 #define KVM_EXIT_SYSTEM_EVENT     24
+#define KVM_EXIT_S390_STSI        25
 
 /* For KVM_EXIT_INTERNAL_ERROR */
 /* Emulate instruction failed. */
@@ -309,6 +310,15 @@ struct kvm_run {
 			__u32 type;
 			__u64 flags;
 		} system_event;
+		/* KVM_EXIT_S390_STSI */
+		struct {
+			__u64 addr;
+			__u8 ar;
+			__u8 reserved;
+			__u8 fc;
+			__u8 sel1;
+			__u16 sel2;
+		} s390_stsi;
 		/* Fix the size of the union. */
 		char padding[256];
 	};
@@ -780,6 +790,7 @@ struct kvm_ppc_smmu_info {
 #define KVM_CAP_S390_USER_SIGP 106
 #define KVM_CAP_S390_VECTOR_REGISTERS 107
 #define KVM_CAP_S390_MEM_OP 108
+#define KVM_CAP_S390_USER_STSI 109
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
-- 
2.3.0

Re: [PATCH/RFC 5/9] KVM: s390: introduce post handlers for STSI

[Paolo Bonzini]

On 16/03/2015 09:51, Christian Borntraeger wrote:
> From: Ekaterina Tumanova <tumanova@linux.vnet.ibm.com>
> 
> The Store System Information (STSI) instruction currently collects all
> information it relays to the caller in the kernel. Some information,
> however, is only available in user space. An example of this is the
> guest name: The kernel always sets "KVMGuest", but user space knows the
> actual guest name.
> 
> This patch introduces a new exit, KVM_EXIT_S390_STSI, guarded by a
> capability that can be enabled by user space if it wants to be able to
> insert such data. User space will be provided with the target buffer
> and the requested STSI function code.
> 
> Reviewed-by: Eric Farman <farman@linux.vnet.ibm.com>
> Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com>
> Signed-off-by: Ekaterina Tumanova <tumanova@linux.vnet.ibm.com>
> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
> ---
>  Documentation/virtual/kvm/api.txt | 28 ++++++++++++++++++++++++++++
>  arch/s390/include/asm/kvm_host.h  |  1 +
>  arch/s390/kvm/kvm-s390.c          |  5 +++++
>  arch/s390/kvm/priv.c              | 16 +++++++++++++++-
>  include/uapi/linux/kvm.h          | 11 +++++++++++
>  5 files changed, 60 insertions(+), 1 deletion(-)

This is okay, but it made me curious about s390 specifics because it's
different from the x86 equivalent.  The x86 info is always 16-byte (4
times 32 bit) and the selectors are few (roughly 30), and it is passed
to the kernel at init time via KVM_SET_CPUID2.  How many selectors are
there on s390, and how big is the info that can be written?

Paolo

> diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
> index f03178d..74d18b6 100644
> --- a/Documentation/virtual/kvm/api.txt
> +++ b/Documentation/virtual/kvm/api.txt
> @@ -3304,3 +3304,31 @@ Returns: 0 on success, negative value on error
>  Allows use of the vector registers introduced with z13 processor, and
>  provides for the synchronization between host and user space.  Will
>  return -EINVAL if the machine does not support vectors.
> +
> +7.4 KVM_CAP_S390_USER_STSI
> +
> +Architectures: s390
> +Parameters: none
> +
> +This capability allows post-handlers for the STSI instruction. After
> +initial handling in the kernel, KVM exits to user space with
> +KVM_EXIT_S390_STSI to allow user space to insert further data.
> +
> +Before exiting to userspace, kvm handlers should fill in s390_stsi field of
> +vcpu->run:
> +struct {
> +	__u64 addr;
> +	__u8 ar;
> +	__u8 reserved;
> +	__u8 fc;
> +	__u8 sel1;
> +	__u16 sel2;
> +} s390_stsi;
> +
> +@addr - guest address of STSI SYSIB
> +@fc   - function code
> +@sel1 - selector 1
> +@sel2 - selector 2
> +@ar   - access register number
> +
> +KVM handlers should exit to userspace with rc = -EREMOTE.
> diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
> index 347a333..2356a8c 100644
> --- a/arch/s390/include/asm/kvm_host.h
> +++ b/arch/s390/include/asm/kvm_host.h
> @@ -565,6 +565,7 @@ struct kvm_arch{
>  	int use_vectors;
>  	int user_cpu_state_ctrl;
>  	int user_sigp;
> +	int user_stsi;
>  	struct s390_io_adapter *adapters[MAX_S390_IO_ADAPTERS];
>  	wait_queue_head_t ipte_wq;
>  	int ipte_lock_count;
> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
> index 92855e2..1f5baf4 100644
> --- a/arch/s390/kvm/kvm-s390.c
> +++ b/arch/s390/kvm/kvm-s390.c
> @@ -178,6 +178,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
>  	case KVM_CAP_VM_ATTRIBUTES:
>  	case KVM_CAP_MP_STATE:
>  	case KVM_CAP_S390_USER_SIGP:
> +	case KVM_CAP_S390_USER_STSI:
>  		r = 1;
>  		break;
>  	case KVM_CAP_S390_MEM_OP:
> @@ -280,6 +281,10 @@ static int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
>  		kvm->arch.use_vectors = MACHINE_HAS_VX;
>  		r = MACHINE_HAS_VX ? 0 : -EINVAL;
>  		break;
> +	case KVM_CAP_S390_USER_STSI:
> +		kvm->arch.user_stsi = 1;
> +		r = 0;
> +		break;
>  	default:
>  		r = -EINVAL;
>  		break;
> diff --git a/arch/s390/kvm/priv.c b/arch/s390/kvm/priv.c
> index 5f26425..211102c 100644
> --- a/arch/s390/kvm/priv.c
> +++ b/arch/s390/kvm/priv.c
> @@ -488,6 +488,16 @@ static void handle_stsi_3_2_2(struct kvm_vcpu *vcpu, struct sysinfo_3_2_2 *mem)
>  	ASCEBC(mem->vm[0].cpi, 16);
>  }
>  
> +static void insert_stsi_usr_data(struct kvm_vcpu *vcpu, u64 addr, u8 fc,
> +				 u8 sel1, u16 sel2)
> +{
> +	vcpu->run->exit_reason = KVM_EXIT_S390_STSI;
> +	vcpu->run->s390_stsi.addr = addr;
> +	vcpu->run->s390_stsi.fc = fc;
> +	vcpu->run->s390_stsi.sel1 = sel1;
> +	vcpu->run->s390_stsi.sel2 = sel2;
> +}
> +
>  static int handle_stsi(struct kvm_vcpu *vcpu)
>  {
>  	int fc = (vcpu->run->s.regs.gprs[0] & 0xf0000000) >> 28;
> @@ -547,11 +557,15 @@ static int handle_stsi(struct kvm_vcpu *vcpu)
>  		rc = kvm_s390_inject_prog_cond(vcpu, rc);
>  		goto out;
>  	}
> +	if (vcpu->kvm->arch.user_stsi) {
> +		insert_stsi_usr_data(vcpu, operand2, fc, sel1, sel2);
> +		rc = -EREMOTE;
> +	}
>  	trace_kvm_s390_handle_stsi(vcpu, fc, sel1, sel2, operand2);
>  	free_page(mem);
>  	kvm_s390_set_psw_cc(vcpu, 0);
>  	vcpu->run->s.regs.gprs[0] = 0;
> -	return 0;
> +	return rc;
>  out_no_data:
>  	kvm_s390_set_psw_cc(vcpu, 3);
>  out:
> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
> index 74bbd9a..279ffd2 100644
> --- a/include/uapi/linux/kvm.h
> +++ b/include/uapi/linux/kvm.h
> @@ -172,6 +172,7 @@ struct kvm_pit_config {
>  #define KVM_EXIT_S390_TSCH        22
>  #define KVM_EXIT_EPR              23
>  #define KVM_EXIT_SYSTEM_EVENT     24
> +#define KVM_EXIT_S390_STSI        25
>  
>  /* For KVM_EXIT_INTERNAL_ERROR */
>  /* Emulate instruction failed. */
> @@ -309,6 +310,15 @@ struct kvm_run {
>  			__u32 type;
>  			__u64 flags;
>  		} system_event;
> +		/* KVM_EXIT_S390_STSI */
> +		struct {
> +			__u64 addr;
> +			__u8 ar;
> +			__u8 reserved;
> +			__u8 fc;
> +			__u8 sel1;
> +			__u16 sel2;
> +		} s390_stsi;
>  		/* Fix the size of the union. */
>  		char padding[256];
>  	};
> @@ -780,6 +790,7 @@ struct kvm_ppc_smmu_info {
>  #define KVM_CAP_S390_USER_SIGP 106
>  #define KVM_CAP_S390_VECTOR_REGISTERS 107
>  #define KVM_CAP_S390_MEM_OP 108
> +#define KVM_CAP_S390_USER_STSI 109
>  
>  #ifdef KVM_CAP_IRQ_ROUTING
>  
> 

Re: [PATCH/RFC 5/9] KVM: s390: introduce post handlers for STSI

[Christian Borntraeger]

Am 16.03.2015 um 12:55 schrieb Paolo Bonzini:
> 
> 
> On 16/03/2015 09:51, Christian Borntraeger wrote:
>> From: Ekaterina Tumanova <tumanova@linux.vnet.ibm.com>
>>
>> The Store System Information (STSI) instruction currently collects all
>> information it relays to the caller in the kernel. Some information,
>> however, is only available in user space. An example of this is the
>> guest name: The kernel always sets "KVMGuest", but user space knows the
>> actual guest name.
>>
>> This patch introduces a new exit, KVM_EXIT_S390_STSI, guarded by a
>> capability that can be enabled by user space if it wants to be able to
>> insert such data. User space will be provided with the target buffer
>> and the requested STSI function code.
>>
>> Reviewed-by: Eric Farman <farman@linux.vnet.ibm.com>
>> Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com>
>> Signed-off-by: Ekaterina Tumanova <tumanova@linux.vnet.ibm.com>
>> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
>> ---
>>  Documentation/virtual/kvm/api.txt | 28 ++++++++++++++++++++++++++++
>>  arch/s390/include/asm/kvm_host.h  |  1 +
>>  arch/s390/kvm/kvm-s390.c          |  5 +++++
>>  arch/s390/kvm/priv.c              | 16 +++++++++++++++-
>>  include/uapi/linux/kvm.h          | 11 +++++++++++
>>  5 files changed, 60 insertions(+), 1 deletion(-)
> 
> This is okay, but it made me curious about s390 specifics because it's
> different from the x86 equivalent.  The x86 info is always 16-byte (4
> times 32 bit) and the selectors are few (roughly 30), and it is passed
> to the kernel at init time via KVM_SET_CPUID2.  How many selectors are
> there on s390, and how big is the info that can be written?

Its up to a page and the content depends on the function code, selector1
and selector2.
See arch/s390/kernel/sysinfo.c for the details or zarchitecture principles of
opration the STSI instruction.


Some time ago we had patches that passed the guest name
and UUID into the kernel - then Alex Graf asked, why we cant do that in 
userspace.
https://lists.nongnu.org/archive/html/qemu-devel/2014-04/msg00094.html

Now, stsi also gives the bare information as provides by upper
layers: e.g. 2.2.2 gives basic LPAR information. When doing that under
KVM or z/VM, KVM or z/VM will pass that information along to the underlying 
LPAR hypervisor and give the result back to the guest.

So I came up with the idea to let KVM do what it needs but also allow
QEMU to override (3.2.2 - this 2nd level hypervisor)

https://lists.nongnu.org/archive/html/qemu-devel/2014-04/msg00286.html

Just passing in uuid and guest name (e.g. via VM attribute) would suffice
for what we need today, but it certainly goes back to where we came from
(minus that we dont have the config device, instead we now have VM attributes)
The overall code would certainly by smaller, but then its kernel only.

Christian



> 
> Paolo
> 
>> diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
>> index f03178d..74d18b6 100644
>> --- a/Documentation/virtual/kvm/api.txt
>> +++ b/Documentation/virtual/kvm/api.txt
>> @@ -3304,3 +3304,31 @@ Returns: 0 on success, negative value on error
>>  Allows use of the vector registers introduced with z13 processor, and
>>  provides for the synchronization between host and user space.  Will
>>  return -EINVAL if the machine does not support vectors.
>> +
>> +7.4 KVM_CAP_S390_USER_STSI
>> +
>> +Architectures: s390
>> +Parameters: none
>> +
>> +This capability allows post-handlers for the STSI instruction. After
>> +initial handling in the kernel, KVM exits to user space with
>> +KVM_EXIT_S390_STSI to allow user space to insert further data.
>> +
>> +Before exiting to userspace, kvm handlers should fill in s390_stsi field of
>> +vcpu->run:
>> +struct {
>> +	__u64 addr;
>> +	__u8 ar;
>> +	__u8 reserved;
>> +	__u8 fc;
>> +	__u8 sel1;
>> +	__u16 sel2;
>> +} s390_stsi;
>> +
>> +@addr - guest address of STSI SYSIB
>> +@fc   - function code
>> +@sel1 - selector 1
>> +@sel2 - selector 2
>> +@ar   - access register number
>> +
>> +KVM handlers should exit to userspace with rc = -EREMOTE.
>> diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
>> index 347a333..2356a8c 100644
>> --- a/arch/s390/include/asm/kvm_host.h
>> +++ b/arch/s390/include/asm/kvm_host.h
>> @@ -565,6 +565,7 @@ struct kvm_arch{
>>  	int use_vectors;
>>  	int user_cpu_state_ctrl;
>>  	int user_sigp;
>> +	int user_stsi;
>>  	struct s390_io_adapter *adapters[MAX_S390_IO_ADAPTERS];
>>  	wait_queue_head_t ipte_wq;
>>  	int ipte_lock_count;
>> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
>> index 92855e2..1f5baf4 100644
>> --- a/arch/s390/kvm/kvm-s390.c
>> +++ b/arch/s390/kvm/kvm-s390.c
>> @@ -178,6 +178,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
>>  	case KVM_CAP_VM_ATTRIBUTES:
>>  	case KVM_CAP_MP_STATE:
>>  	case KVM_CAP_S390_USER_SIGP:
>> +	case KVM_CAP_S390_USER_STSI:
>>  		r = 1;
>>  		break;
>>  	case KVM_CAP_S390_MEM_OP:
>> @@ -280,6 +281,10 @@ static int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
>>  		kvm->arch.use_vectors = MACHINE_HAS_VX;
>>  		r = MACHINE_HAS_VX ? 0 : -EINVAL;
>>  		break;
>> +	case KVM_CAP_S390_USER_STSI:
>> +		kvm->arch.user_stsi = 1;
>> +		r = 0;
>> +		break;
>>  	default:
>>  		r = -EINVAL;
>>  		break;
>> diff --git a/arch/s390/kvm/priv.c b/arch/s390/kvm/priv.c
>> index 5f26425..211102c 100644
>> --- a/arch/s390/kvm/priv.c
>> +++ b/arch/s390/kvm/priv.c
>> @@ -488,6 +488,16 @@ static void handle_stsi_3_2_2(struct kvm_vcpu *vcpu, struct sysinfo_3_2_2 *mem)
>>  	ASCEBC(mem->vm[0].cpi, 16);
>>  }
>>  
>> +static void insert_stsi_usr_data(struct kvm_vcpu *vcpu, u64 addr, u8 fc,
>> +				 u8 sel1, u16 sel2)
>> +{
>> +	vcpu->run->exit_reason = KVM_EXIT_S390_STSI;
>> +	vcpu->run->s390_stsi.addr = addr;
>> +	vcpu->run->s390_stsi.fc = fc;
>> +	vcpu->run->s390_stsi.sel1 = sel1;
>> +	vcpu->run->s390_stsi.sel2 = sel2;
>> +}
>> +
>>  static int handle_stsi(struct kvm_vcpu *vcpu)
>>  {
>>  	int fc = (vcpu->run->s.regs.gprs[0] & 0xf0000000) >> 28;
>> @@ -547,11 +557,15 @@ static int handle_stsi(struct kvm_vcpu *vcpu)
>>  		rc = kvm_s390_inject_prog_cond(vcpu, rc);
>>  		goto out;
>>  	}
>> +	if (vcpu->kvm->arch.user_stsi) {
>> +		insert_stsi_usr_data(vcpu, operand2, fc, sel1, sel2);
>> +		rc = -EREMOTE;
>> +	}
>>  	trace_kvm_s390_handle_stsi(vcpu, fc, sel1, sel2, operand2);
>>  	free_page(mem);
>>  	kvm_s390_set_psw_cc(vcpu, 0);
>>  	vcpu->run->s.regs.gprs[0] = 0;
>> -	return 0;
>> +	return rc;
>>  out_no_data:
>>  	kvm_s390_set_psw_cc(vcpu, 3);
>>  out:
>> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
>> index 74bbd9a..279ffd2 100644
>> --- a/include/uapi/linux/kvm.h
>> +++ b/include/uapi/linux/kvm.h
>> @@ -172,6 +172,7 @@ struct kvm_pit_config {
>>  #define KVM_EXIT_S390_TSCH        22
>>  #define KVM_EXIT_EPR              23
>>  #define KVM_EXIT_SYSTEM_EVENT     24
>> +#define KVM_EXIT_S390_STSI        25
>>  
>>  /* For KVM_EXIT_INTERNAL_ERROR */
>>  /* Emulate instruction failed. */
>> @@ -309,6 +310,15 @@ struct kvm_run {
>>  			__u32 type;
>>  			__u64 flags;
>>  		} system_event;
>> +		/* KVM_EXIT_S390_STSI */
>> +		struct {
>> +			__u64 addr;
>> +			__u8 ar;
>> +			__u8 reserved;
>> +			__u8 fc;
>> +			__u8 sel1;
>> +			__u16 sel2;
>> +		} s390_stsi;
>>  		/* Fix the size of the union. */
>>  		char padding[256];
>>  	};
>> @@ -780,6 +790,7 @@ struct kvm_ppc_smmu_info {
>>  #define KVM_CAP_S390_USER_SIGP 106
>>  #define KVM_CAP_S390_VECTOR_REGISTERS 107
>>  #define KVM_CAP_S390_MEM_OP 108
>> +#define KVM_CAP_S390_USER_STSI 109
>>  
>>  #ifdef KVM_CAP_IRQ_ROUTING
>>  
>>
> 

Re: [PATCH/RFC 5/9] KVM: s390: introduce post handlers for STSI

[Paolo Bonzini]

On 16/03/2015 13:18, Christian Borntraeger wrote:
> Its up to a page and the content depends on the function code, selector1
> and selector2.
> See arch/s390/kernel/sysinfo.c for the details or zarchitecture principles of
> opration the STSI instruction.
> 
> 
> Some time ago we had patches that passed the guest name
> and UUID into the kernel - then Alex Graf asked, why we cant do that in 
> userspace.
> https://lists.nongnu.org/archive/html/qemu-devel/2014-04/msg00094.html
> 
> Now, stsi also gives the bare information as provides by upper
> layers: e.g. 2.2.2 gives basic LPAR information. When doing that under
> KVM or z/VM, KVM or z/VM will pass that information along to the underlying 
> LPAR hypervisor and give the result back to the guest.
> 
> So I came up with the idea to let KVM do what it needs but also allow
> QEMU to override (3.2.2 - this 2nd level hypervisor)

If it's up to a page it definitely makes sense to do it this way.

Thanks,

Paolo

[PATCH/RFC 6/9] KVM: s390: Guest's memory access functions get access registers

[Christian Borntraeger]

From: Alexander Yarygin <yarygin@linux.vnet.ibm.com>

In access register mode, the write_guest() read_guest() and other
functions will invoke the access register translation, which
requires an ar, designated by one of the instruction fields.

Signed-off-by: Alexander Yarygin <yarygin@linux.vnet.ibm.com>
Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 arch/s390/kvm/diag.c      |  4 +--
 arch/s390/kvm/gaccess.c   |  8 ++---
 arch/s390/kvm/gaccess.h   | 16 +++++-----
 arch/s390/kvm/intercept.c |  4 +--
 arch/s390/kvm/kvm-s390.c  | 10 +++---
 arch/s390/kvm/kvm-s390.h  | 26 +++++++++++++---
 arch/s390/kvm/priv.c      | 79 ++++++++++++++++++++++++++++-------------------
 arch/s390/kvm/sigp.c      |  4 +--
 8 files changed, 93 insertions(+), 58 deletions(-)

diff --git a/arch/s390/kvm/diag.c b/arch/s390/kvm/diag.c
index 9254aff..89140dd 100644
--- a/arch/s390/kvm/diag.c
+++ b/arch/s390/kvm/diag.c
@@ -77,7 +77,7 @@ static int __diag_page_ref_service(struct kvm_vcpu *vcpu)
 
 	if (vcpu->run->s.regs.gprs[rx] & 7)
 		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
-	rc = read_guest(vcpu, vcpu->run->s.regs.gprs[rx], &parm, sizeof(parm));
+	rc = read_guest(vcpu, vcpu->run->s.regs.gprs[rx], rx, &parm, sizeof(parm));
 	if (rc)
 		return kvm_s390_inject_prog_cond(vcpu, rc);
 	if (parm.parm_version != 2 || parm.parm_len < 5 || parm.code != 0x258)
@@ -230,7 +230,7 @@ static int __diag_virtio_hypercall(struct kvm_vcpu *vcpu)
 
 int kvm_s390_handle_diag(struct kvm_vcpu *vcpu)
 {
-	int code = kvm_s390_get_base_disp_rs(vcpu) & 0xffff;
+	int code = kvm_s390_get_base_disp_rs(vcpu, NULL) & 0xffff;
 
 	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
 		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
diff --git a/arch/s390/kvm/gaccess.c b/arch/s390/kvm/gaccess.c
index 04a7c67..33b5ae1 100644
--- a/arch/s390/kvm/gaccess.c
+++ b/arch/s390/kvm/gaccess.c
@@ -578,7 +578,7 @@ static int guest_page_range(struct kvm_vcpu *vcpu, unsigned long ga,
 	return 0;
 }
 
-int access_guest(struct kvm_vcpu *vcpu, unsigned long ga, void *data,
+int access_guest(struct kvm_vcpu *vcpu, unsigned long ga, ar_t ar, void *data,
 		 unsigned long len, int write)
 {
 	psw_t *psw = &vcpu->arch.sie_block->gpsw;
@@ -652,7 +652,7 @@ int access_guest_real(struct kvm_vcpu *vcpu, unsigned long gra,
  * Note: The IPTE lock is not taken during this function, so the caller
  * has to take care of this.
  */
-int guest_translate_address(struct kvm_vcpu *vcpu, unsigned long gva,
+int guest_translate_address(struct kvm_vcpu *vcpu, unsigned long gva, ar_t ar,
 			    unsigned long *gpa, int write)
 {
 	struct kvm_s390_pgm_info *pgm = &vcpu->arch.pgm;
@@ -699,7 +699,7 @@ int guest_translate_address(struct kvm_vcpu *vcpu, unsigned long gva,
 /**
  * check_gva_range - test a range of guest virtual addresses for accessibility
  */
-int check_gva_range(struct kvm_vcpu *vcpu, unsigned long gva,
+int check_gva_range(struct kvm_vcpu *vcpu, unsigned long gva, ar_t ar,
 		    unsigned long length, int is_write)
 {
 	unsigned long gpa;
@@ -709,7 +709,7 @@ int check_gva_range(struct kvm_vcpu *vcpu, unsigned long gva,
 	ipte_lock(vcpu);
 	while (length > 0 && !rc) {
 		currlen = min(length, PAGE_SIZE - (gva % PAGE_SIZE));
-		rc = guest_translate_address(vcpu, gva, &gpa, is_write);
+		rc = guest_translate_address(vcpu, gva, ar, &gpa, is_write);
 		gva += currlen;
 		length -= currlen;
 	}
diff --git a/arch/s390/kvm/gaccess.h b/arch/s390/kvm/gaccess.h
index d0de110..68bf4f7 100644
--- a/arch/s390/kvm/gaccess.h
+++ b/arch/s390/kvm/gaccess.h
@@ -156,11 +156,11 @@ int read_guest_lc(struct kvm_vcpu *vcpu, unsigned long gra, void *data,
 }
 
 int guest_translate_address(struct kvm_vcpu *vcpu, unsigned long gva,
-			    unsigned long *gpa, int write);
-int check_gva_range(struct kvm_vcpu *vcpu, unsigned long gva,
+			    ar_t ar, unsigned long *gpa, int write);
+int check_gva_range(struct kvm_vcpu *vcpu, unsigned long gva, ar_t ar,
 		    unsigned long length, int is_write);
 
-int access_guest(struct kvm_vcpu *vcpu, unsigned long ga, void *data,
+int access_guest(struct kvm_vcpu *vcpu, unsigned long ga, ar_t ar, void *data,
 		 unsigned long len, int write);
 
 int access_guest_real(struct kvm_vcpu *vcpu, unsigned long gra,
@@ -170,6 +170,7 @@ int access_guest_real(struct kvm_vcpu *vcpu, unsigned long gra,
  * write_guest - copy data from kernel space to guest space
  * @vcpu: virtual cpu
  * @ga: guest address
+ * @ar: access register
  * @data: source address in kernel space
  * @len: number of bytes to copy
  *
@@ -212,16 +213,17 @@ int access_guest_real(struct kvm_vcpu *vcpu, unsigned long gra,
  *	 if data has been changed in guest space in case of an exception.
  */
 static inline __must_check
-int write_guest(struct kvm_vcpu *vcpu, unsigned long ga, void *data,
+int write_guest(struct kvm_vcpu *vcpu, unsigned long ga, ar_t ar, void *data,
 		unsigned long len)
 {
-	return access_guest(vcpu, ga, data, len, 1);
+	return access_guest(vcpu, ga, ar, data, len, 1);
 }
 
 /**
  * read_guest - copy data from guest space to kernel space
  * @vcpu: virtual cpu
  * @ga: guest address
+ * @ar: access register
  * @data: destination address in kernel space
  * @len: number of bytes to copy
  *
@@ -231,10 +233,10 @@ int write_guest(struct kvm_vcpu *vcpu, unsigned long ga, void *data,
  * data will be copied from guest space to kernel space.
  */
 static inline __must_check
-int read_guest(struct kvm_vcpu *vcpu, unsigned long ga, void *data,
+int read_guest(struct kvm_vcpu *vcpu, unsigned long ga, ar_t ar, void *data,
 	       unsigned long len)
 {
-	return access_guest(vcpu, ga, data, len, 0);
+	return access_guest(vcpu, ga, ar, data, len, 0);
 }
 
 /**
diff --git a/arch/s390/kvm/intercept.c b/arch/s390/kvm/intercept.c
index 08ae10a..9e3779e 100644
--- a/arch/s390/kvm/intercept.c
+++ b/arch/s390/kvm/intercept.c
@@ -320,7 +320,7 @@ static int handle_mvpg_pei(struct kvm_vcpu *vcpu)
 
 	/* Make sure that the source is paged-in */
 	rc = guest_translate_address(vcpu, vcpu->run->s.regs.gprs[reg2],
-				     &srcaddr, 0);
+				     reg2, &srcaddr, 0);
 	if (rc)
 		return kvm_s390_inject_prog_cond(vcpu, rc);
 	rc = kvm_arch_fault_in_page(vcpu, srcaddr, 0);
@@ -329,7 +329,7 @@ static int handle_mvpg_pei(struct kvm_vcpu *vcpu)
 
 	/* Make sure that the destination is paged-in */
 	rc = guest_translate_address(vcpu, vcpu->run->s.regs.gprs[reg1],
-				     &dstaddr, 1);
+				     reg1, &dstaddr, 1);
 	if (rc)
 		return kvm_s390_inject_prog_cond(vcpu, rc);
 	rc = kvm_arch_fault_in_page(vcpu, dstaddr, 1);
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index 1f5baf4..fdfa106 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -1787,7 +1787,7 @@ static int vcpu_post_run_fault_in_sie(struct kvm_vcpu *vcpu)
 	 * to look up the current opcode to get the length of the instruction
 	 * to be able to forward the PSW.
 	 */
-	rc = read_guest(vcpu, psw->addr, &opcode, 1);
+	rc = read_guest(vcpu, psw->addr, 0, &opcode, 1);
 	if (rc)
 		return kvm_s390_inject_prog_cond(vcpu, rc);
 	psw->addr = __rewind_psw(*psw, -insn_length(opcode));
@@ -2222,10 +2222,10 @@ static long kvm_s390_guest_mem_op(struct kvm_vcpu *vcpu,
 	switch (mop->op) {
 	case KVM_S390_MEMOP_LOGICAL_READ:
 		if (mop->flags & KVM_S390_MEMOP_F_CHECK_ONLY) {
-			r = check_gva_range(vcpu, mop->gaddr, mop->size, false);
+			r = check_gva_range(vcpu, mop->gaddr, mop->ar, mop->size, false);
 			break;
 		}
-		r = read_guest(vcpu, mop->gaddr, tmpbuf, mop->size);
+		r = read_guest(vcpu, mop->gaddr, mop->ar, tmpbuf, mop->size);
 		if (r == 0) {
 			if (copy_to_user(uaddr, tmpbuf, mop->size))
 				r = -EFAULT;
@@ -2233,14 +2233,14 @@ static long kvm_s390_guest_mem_op(struct kvm_vcpu *vcpu,
 		break;
 	case KVM_S390_MEMOP_LOGICAL_WRITE:
 		if (mop->flags & KVM_S390_MEMOP_F_CHECK_ONLY) {
-			r = check_gva_range(vcpu, mop->gaddr, mop->size, true);
+			r = check_gva_range(vcpu, mop->gaddr, mop->ar, mop->size, true);
 			break;
 		}
 		if (copy_from_user(tmpbuf, uaddr, mop->size)) {
 			r = -EFAULT;
 			break;
 		}
-		r = write_guest(vcpu, mop->gaddr, tmpbuf, mop->size);
+		r = write_guest(vcpu, mop->gaddr, mop->ar, tmpbuf, mop->size);
 		break;
 	default:
 		r = -EINVAL;
diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h
index 83f32a1..fdc600b 100644
--- a/arch/s390/kvm/kvm-s390.h
+++ b/arch/s390/kvm/kvm-s390.h
@@ -70,16 +70,23 @@ static inline void kvm_s390_set_prefix(struct kvm_vcpu *vcpu, u32 prefix)
 	kvm_make_request(KVM_REQ_MMU_RELOAD, vcpu);
 }
 
-static inline u64 kvm_s390_get_base_disp_s(struct kvm_vcpu *vcpu)
+typedef u8 __bitwise ar_t;
+#define AR_INVAL ((ar_t)-1)
+
+static inline u64 kvm_s390_get_base_disp_s(struct kvm_vcpu *vcpu, ar_t *ar)
 {
 	u32 base2 = vcpu->arch.sie_block->ipb >> 28;
 	u32 disp2 = ((vcpu->arch.sie_block->ipb & 0x0fff0000) >> 16);
 
+	if (ar)
+		*ar = base2;
+
 	return (base2 ? vcpu->run->s.regs.gprs[base2] : 0) + disp2;
 }
 
 static inline void kvm_s390_get_base_disp_sse(struct kvm_vcpu *vcpu,
-					      u64 *address1, u64 *address2)
+					      u64 *address1, u64 *address2,
+					      ar_t *ar_b1, ar_t *ar_b2)
 {
 	u32 base1 = (vcpu->arch.sie_block->ipb & 0xf0000000) >> 28;
 	u32 disp1 = (vcpu->arch.sie_block->ipb & 0x0fff0000) >> 16;
@@ -88,6 +95,11 @@ static inline void kvm_s390_get_base_disp_sse(struct kvm_vcpu *vcpu,
 
 	*address1 = (base1 ? vcpu->run->s.regs.gprs[base1] : 0) + disp1;
 	*address2 = (base2 ? vcpu->run->s.regs.gprs[base2] : 0) + disp2;
+
+	if (ar_b1)
+		*ar_b1 = base1;
+	if (ar_b2)
+		*ar_b2 = base2;
 }
 
 static inline void kvm_s390_get_regs_rre(struct kvm_vcpu *vcpu, int *r1, int *r2)
@@ -98,7 +110,7 @@ static inline void kvm_s390_get_regs_rre(struct kvm_vcpu *vcpu, int *r1, int *r2
 		*r2 = (vcpu->arch.sie_block->ipb & 0x000f0000) >> 16;
 }
 
-static inline u64 kvm_s390_get_base_disp_rsy(struct kvm_vcpu *vcpu)
+static inline u64 kvm_s390_get_base_disp_rsy(struct kvm_vcpu *vcpu, ar_t *ar)
 {
 	u32 base2 = vcpu->arch.sie_block->ipb >> 28;
 	u32 disp2 = ((vcpu->arch.sie_block->ipb & 0x0fff0000) >> 16) +
@@ -107,14 +119,20 @@ static inline u64 kvm_s390_get_base_disp_rsy(struct kvm_vcpu *vcpu)
 	if (disp2 & 0x80000)
 		disp2+=0xfff00000;
 
+	if (ar)
+		*ar = base2;
+
 	return (base2 ? vcpu->run->s.regs.gprs[base2] : 0) + (long)(int)disp2;
 }
 
-static inline u64 kvm_s390_get_base_disp_rs(struct kvm_vcpu *vcpu)
+static inline u64 kvm_s390_get_base_disp_rs(struct kvm_vcpu *vcpu, ar_t *ar)
 {
 	u32 base2 = vcpu->arch.sie_block->ipb >> 28;
 	u32 disp2 = ((vcpu->arch.sie_block->ipb & 0x0fff0000) >> 16);
 
+	if (ar)
+		*ar = base2;
+
 	return (base2 ? vcpu->run->s.regs.gprs[base2] : 0) + disp2;
 }
 
diff --git a/arch/s390/kvm/priv.c b/arch/s390/kvm/priv.c
index 211102c..5e4658d 100644
--- a/arch/s390/kvm/priv.c
+++ b/arch/s390/kvm/priv.c
@@ -36,15 +36,16 @@ static int handle_set_clock(struct kvm_vcpu *vcpu)
 	struct kvm_vcpu *cpup;
 	s64 hostclk, val;
 	int i, rc;
+	ar_t ar;
 	u64 op2;
 
 	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
 		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
 
-	op2 = kvm_s390_get_base_disp_s(vcpu);
+	op2 = kvm_s390_get_base_disp_s(vcpu, &ar);
 	if (op2 & 7)	/* Operand must be on a doubleword boundary */
 		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
-	rc = read_guest(vcpu, op2, &val, sizeof(val));
+	rc = read_guest(vcpu, op2, ar, &val, sizeof(val));
 	if (rc)
 		return kvm_s390_inject_prog_cond(vcpu, rc);
 
@@ -68,20 +69,21 @@ static int handle_set_prefix(struct kvm_vcpu *vcpu)
 	u64 operand2;
 	u32 address;
 	int rc;
+	ar_t ar;
 
 	vcpu->stat.instruction_spx++;
 
 	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
 		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
 
-	operand2 = kvm_s390_get_base_disp_s(vcpu);
+	operand2 = kvm_s390_get_base_disp_s(vcpu, &ar);
 
 	/* must be word boundary */
 	if (operand2 & 3)
 		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
 
 	/* get the value */
-	rc = read_guest(vcpu, operand2, &address, sizeof(address));
+	rc = read_guest(vcpu, operand2, ar, &address, sizeof(address));
 	if (rc)
 		return kvm_s390_inject_prog_cond(vcpu, rc);
 
@@ -107,13 +109,14 @@ static int handle_store_prefix(struct kvm_vcpu *vcpu)
 	u64 operand2;
 	u32 address;
 	int rc;
+	ar_t ar;
 
 	vcpu->stat.instruction_stpx++;
 
 	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
 		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
 
-	operand2 = kvm_s390_get_base_disp_s(vcpu);
+	operand2 = kvm_s390_get_base_disp_s(vcpu, &ar);
 
 	/* must be word boundary */
 	if (operand2 & 3)
@@ -122,7 +125,7 @@ static int handle_store_prefix(struct kvm_vcpu *vcpu)
 	address = kvm_s390_get_prefix(vcpu);
 
 	/* get the value */
-	rc = write_guest(vcpu, operand2, &address, sizeof(address));
+	rc = write_guest(vcpu, operand2, ar, &address, sizeof(address));
 	if (rc)
 		return kvm_s390_inject_prog_cond(vcpu, rc);
 
@@ -136,18 +139,19 @@ static int handle_store_cpu_address(struct kvm_vcpu *vcpu)
 	u16 vcpu_id = vcpu->vcpu_id;
 	u64 ga;
 	int rc;
+	ar_t ar;
 
 	vcpu->stat.instruction_stap++;
 
 	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
 		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
 
-	ga = kvm_s390_get_base_disp_s(vcpu);
+	ga = kvm_s390_get_base_disp_s(vcpu, &ar);
 
 	if (ga & 1)
 		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
 
-	rc = write_guest(vcpu, ga, &vcpu_id, sizeof(vcpu_id));
+	rc = write_guest(vcpu, ga, ar, &vcpu_id, sizeof(vcpu_id));
 	if (rc)
 		return kvm_s390_inject_prog_cond(vcpu, rc);
 
@@ -231,8 +235,9 @@ static int handle_tpi(struct kvm_vcpu *vcpu)
 	u32 tpi_data[3];
 	int rc;
 	u64 addr;
+	ar_t ar;
 
-	addr = kvm_s390_get_base_disp_s(vcpu);
+	addr = kvm_s390_get_base_disp_s(vcpu, &ar);
 	if (addr & 3)
 		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
 
@@ -251,7 +256,7 @@ static int handle_tpi(struct kvm_vcpu *vcpu)
 		 * provided area.
 		 */
 		len = sizeof(tpi_data) - 4;
-		rc = write_guest(vcpu, addr, &tpi_data, len);
+		rc = write_guest(vcpu, addr, ar, &tpi_data, len);
 		if (rc) {
 			rc = kvm_s390_inject_prog_cond(vcpu, rc);
 			goto reinject_interrupt;
@@ -395,15 +400,16 @@ int kvm_s390_handle_lpsw(struct kvm_vcpu *vcpu)
 	psw_compat_t new_psw;
 	u64 addr;
 	int rc;
+	ar_t ar;
 
 	if (gpsw->mask & PSW_MASK_PSTATE)
 		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
 
-	addr = kvm_s390_get_base_disp_s(vcpu);
+	addr = kvm_s390_get_base_disp_s(vcpu, &ar);
 	if (addr & 7)
 		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
 
-	rc = read_guest(vcpu, addr, &new_psw, sizeof(new_psw));
+	rc = read_guest(vcpu, addr, ar, &new_psw, sizeof(new_psw));
 	if (rc)
 		return kvm_s390_inject_prog_cond(vcpu, rc);
 	if (!(new_psw.mask & PSW32_MASK_BASE))
@@ -421,14 +427,15 @@ static int handle_lpswe(struct kvm_vcpu *vcpu)
 	psw_t new_psw;
 	u64 addr;
 	int rc;
+	ar_t ar;
 
 	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
 		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
 
-	addr = kvm_s390_get_base_disp_s(vcpu);
+	addr = kvm_s390_get_base_disp_s(vcpu, &ar);
 	if (addr & 7)
 		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
-	rc = read_guest(vcpu, addr, &new_psw, sizeof(new_psw));
+	rc = read_guest(vcpu, addr, ar, &new_psw, sizeof(new_psw));
 	if (rc)
 		return kvm_s390_inject_prog_cond(vcpu, rc);
 	vcpu->arch.sie_block->gpsw = new_psw;
@@ -442,18 +449,19 @@ static int handle_stidp(struct kvm_vcpu *vcpu)
 	u64 stidp_data = vcpu->arch.stidp_data;
 	u64 operand2;
 	int rc;
+	ar_t ar;
 
 	vcpu->stat.instruction_stidp++;
 
 	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
 		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
 
-	operand2 = kvm_s390_get_base_disp_s(vcpu);
+	operand2 = kvm_s390_get_base_disp_s(vcpu, &ar);
 
 	if (operand2 & 7)
 		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
 
-	rc = write_guest(vcpu, operand2, &stidp_data, sizeof(stidp_data));
+	rc = write_guest(vcpu, operand2, ar, &stidp_data, sizeof(stidp_data));
 	if (rc)
 		return kvm_s390_inject_prog_cond(vcpu, rc);
 
@@ -488,11 +496,12 @@ static void handle_stsi_3_2_2(struct kvm_vcpu *vcpu, struct sysinfo_3_2_2 *mem)
 	ASCEBC(mem->vm[0].cpi, 16);
 }
 
-static void insert_stsi_usr_data(struct kvm_vcpu *vcpu, u64 addr, u8 fc,
-				 u8 sel1, u16 sel2)
+static void insert_stsi_usr_data(struct kvm_vcpu *vcpu, u64 addr, ar_t ar,
+				 u8 fc, u8 sel1, u16 sel2)
 {
 	vcpu->run->exit_reason = KVM_EXIT_S390_STSI;
 	vcpu->run->s390_stsi.addr = addr;
+	vcpu->run->s390_stsi.ar = ar;
 	vcpu->run->s390_stsi.fc = fc;
 	vcpu->run->s390_stsi.sel1 = sel1;
 	vcpu->run->s390_stsi.sel2 = sel2;
@@ -506,6 +515,7 @@ static int handle_stsi(struct kvm_vcpu *vcpu)
 	unsigned long mem = 0;
 	u64 operand2;
 	int rc = 0;
+	ar_t ar;
 
 	vcpu->stat.instruction_stsi++;
 	VCPU_EVENT(vcpu, 4, "stsi: fc: %x sel1: %x sel2: %x", fc, sel1, sel2);
@@ -528,7 +538,7 @@ static int handle_stsi(struct kvm_vcpu *vcpu)
 		return 0;
 	}
 
-	operand2 = kvm_s390_get_base_disp_s(vcpu);
+	operand2 = kvm_s390_get_base_disp_s(vcpu, &ar);
 
 	if (operand2 & 0xfff)
 		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
@@ -552,13 +562,13 @@ static int handle_stsi(struct kvm_vcpu *vcpu)
 		break;
 	}
 
-	rc = write_guest(vcpu, operand2, (void *)mem, PAGE_SIZE);
+	rc = write_guest(vcpu, operand2, ar, (void *)mem, PAGE_SIZE);
 	if (rc) {
 		rc = kvm_s390_inject_prog_cond(vcpu, rc);
 		goto out;
 	}
 	if (vcpu->kvm->arch.user_stsi) {
-		insert_stsi_usr_data(vcpu, operand2, fc, sel1, sel2);
+		insert_stsi_usr_data(vcpu, operand2, ar, fc, sel1, sel2);
 		rc = -EREMOTE;
 	}
 	trace_kvm_s390_handle_stsi(vcpu, fc, sel1, sel2, operand2);
@@ -800,13 +810,14 @@ int kvm_s390_handle_lctl(struct kvm_vcpu *vcpu)
 	int reg, rc, nr_regs;
 	u32 ctl_array[16];
 	u64 ga;
+	ar_t ar;
 
 	vcpu->stat.instruction_lctl++;
 
 	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
 		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
 
-	ga = kvm_s390_get_base_disp_rs(vcpu);
+	ga = kvm_s390_get_base_disp_rs(vcpu, &ar);
 
 	if (ga & 3)
 		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
@@ -815,7 +826,7 @@ int kvm_s390_handle_lctl(struct kvm_vcpu *vcpu)
 	trace_kvm_s390_handle_lctl(vcpu, 0, reg1, reg3, ga);
 
 	nr_regs = ((reg3 - reg1) & 0xf) + 1;
-	rc = read_guest(vcpu, ga, ctl_array, nr_regs * sizeof(u32));
+	rc = read_guest(vcpu, ga, ar, ctl_array, nr_regs * sizeof(u32));
 	if (rc)
 		return kvm_s390_inject_prog_cond(vcpu, rc);
 	reg = reg1;
@@ -838,13 +849,14 @@ int kvm_s390_handle_stctl(struct kvm_vcpu *vcpu)
 	int reg, rc, nr_regs;
 	u32 ctl_array[16];
 	u64 ga;
+	ar_t ar;
 
 	vcpu->stat.instruction_stctl++;
 
 	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
 		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
 
-	ga = kvm_s390_get_base_disp_rs(vcpu);
+	ga = kvm_s390_get_base_disp_rs(vcpu, &ar);
 
 	if (ga & 3)
 		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
@@ -860,7 +872,7 @@ int kvm_s390_handle_stctl(struct kvm_vcpu *vcpu)
 			break;
 		reg = (reg + 1) % 16;
 	} while (1);
-	rc = write_guest(vcpu, ga, ctl_array, nr_regs * sizeof(u32));
+	rc = write_guest(vcpu, ga, ar, ctl_array, nr_regs * sizeof(u32));
 	return rc ? kvm_s390_inject_prog_cond(vcpu, rc) : 0;
 }
 
@@ -871,13 +883,14 @@ static int handle_lctlg(struct kvm_vcpu *vcpu)
 	int reg, rc, nr_regs;
 	u64 ctl_array[16];
 	u64 ga;
+	ar_t ar;
 
 	vcpu->stat.instruction_lctlg++;
 
 	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
 		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
 
-	ga = kvm_s390_get_base_disp_rsy(vcpu);
+	ga = kvm_s390_get_base_disp_rsy(vcpu, &ar);
 
 	if (ga & 7)
 		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
@@ -886,7 +899,7 @@ static int handle_lctlg(struct kvm_vcpu *vcpu)
 	trace_kvm_s390_handle_lctl(vcpu, 1, reg1, reg3, ga);
 
 	nr_regs = ((reg3 - reg1) & 0xf) + 1;
-	rc = read_guest(vcpu, ga, ctl_array, nr_regs * sizeof(u64));
+	rc = read_guest(vcpu, ga, ar, ctl_array, nr_regs * sizeof(u64));
 	if (rc)
 		return kvm_s390_inject_prog_cond(vcpu, rc);
 	reg = reg1;
@@ -908,13 +921,14 @@ static int handle_stctg(struct kvm_vcpu *vcpu)
 	int reg, rc, nr_regs;
 	u64 ctl_array[16];
 	u64 ga;
+	ar_t ar;
 
 	vcpu->stat.instruction_stctg++;
 
 	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
 		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
 
-	ga = kvm_s390_get_base_disp_rsy(vcpu);
+	ga = kvm_s390_get_base_disp_rsy(vcpu, &ar);
 
 	if (ga & 7)
 		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
@@ -930,7 +944,7 @@ static int handle_stctg(struct kvm_vcpu *vcpu)
 			break;
 		reg = (reg + 1) % 16;
 	} while (1);
-	rc = write_guest(vcpu, ga, ctl_array, nr_regs * sizeof(u64));
+	rc = write_guest(vcpu, ga, ar, ctl_array, nr_regs * sizeof(u64));
 	return rc ? kvm_s390_inject_prog_cond(vcpu, rc) : 0;
 }
 
@@ -955,13 +969,14 @@ static int handle_tprot(struct kvm_vcpu *vcpu)
 	unsigned long hva, gpa;
 	int ret = 0, cc = 0;
 	bool writable;
+	ar_t ar;
 
 	vcpu->stat.instruction_tprot++;
 
 	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
 		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
 
-	kvm_s390_get_base_disp_sse(vcpu, &address1, &address2);
+	kvm_s390_get_base_disp_sse(vcpu, &address1, &address2, &ar, NULL);
 
 	/* we only handle the Linux memory detection case:
 	 * access key == 0
@@ -970,11 +985,11 @@ static int handle_tprot(struct kvm_vcpu *vcpu)
 		return -EOPNOTSUPP;
 	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_DAT)
 		ipte_lock(vcpu);
-	ret = guest_translate_address(vcpu, address1, &gpa, 1);
+	ret = guest_translate_address(vcpu, address1, ar, &gpa, 1);
 	if (ret == PGM_PROTECTION) {
 		/* Write protected? Try again with read-only... */
 		cc = 1;
-		ret = guest_translate_address(vcpu, address1, &gpa, 0);
+		ret = guest_translate_address(vcpu, address1, ar, &gpa, 0);
 	}
 	if (ret) {
 		if (ret == PGM_ADDRESSING || ret == PGM_TRANSLATION_SPEC) {
diff --git a/arch/s390/kvm/sigp.c b/arch/s390/kvm/sigp.c
index 755a733..72e58bd 100644
--- a/arch/s390/kvm/sigp.c
+++ b/arch/s390/kvm/sigp.c
@@ -434,7 +434,7 @@ int kvm_s390_handle_sigp(struct kvm_vcpu *vcpu)
 	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
 		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
 
-	order_code = kvm_s390_get_base_disp_rs(vcpu);
+	order_code = kvm_s390_get_base_disp_rs(vcpu, NULL);
 	if (handle_sigp_order_in_user_space(vcpu, order_code))
 		return -EOPNOTSUPP;
 
@@ -476,7 +476,7 @@ int kvm_s390_handle_sigp_pei(struct kvm_vcpu *vcpu)
 	int r3 = vcpu->arch.sie_block->ipa & 0x000f;
 	u16 cpu_addr = vcpu->run->s.regs.gprs[r3];
 	struct kvm_vcpu *dest_vcpu;
-	u8 order_code = kvm_s390_get_base_disp_rs(vcpu);
+	u8 order_code = kvm_s390_get_base_disp_rs(vcpu, NULL);
 
 	trace_kvm_s390_handle_sigp_pei(vcpu, order_code, cpu_addr);
 
-- 
2.3.0

Re: [PATCH/RFC 6/9] KVM: s390: Guest's memory access functions get access registers

[Cornelia Huck]

On Mon, 16 Mar 2015 09:51:42 +0100
Christian Borntraeger <borntraeger@de.ibm.com> wrote:

> From: Alexander Yarygin <yarygin@linux.vnet.ibm.com>
> 
> In access register mode, the write_guest() read_guest() and other
> functions will invoke the access register translation, which
> requires an ar, designated by one of the instruction fields.
> 
> Signed-off-by: Alexander Yarygin <yarygin@linux.vnet.ibm.com>
> Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
> ---
>  arch/s390/kvm/diag.c      |  4 +--
>  arch/s390/kvm/gaccess.c   |  8 ++---
>  arch/s390/kvm/gaccess.h   | 16 +++++-----
>  arch/s390/kvm/intercept.c |  4 +--
>  arch/s390/kvm/kvm-s390.c  | 10 +++---
>  arch/s390/kvm/kvm-s390.h  | 26 +++++++++++++---
>  arch/s390/kvm/priv.c      | 79 ++++++++++++++++++++++++++++-------------------
>  arch/s390/kvm/sigp.c      |  4 +--
>  8 files changed, 93 insertions(+), 58 deletions(-)

Acked-by: Cornelia Huck <cornelia.huck@de.ibm.com>

[PATCH/RFC 7/9] KVM: s390: Optimize paths where get_vcpu_asce() is invoked

[Christian Borntraeger]

From: Alexander Yarygin <yarygin@linux.vnet.ibm.com>

During dynamic address translation the get_vcpu_asce()
function can be invoked several times. It's ok for usual modes, but will
be slow if CPUs are in AR mode. Let's call the get_vcpu_asce() once and
pass the result to the called functions.

Signed-off-by: Alexander Yarygin <yarygin@linux.vnet.ibm.com>
Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Reviewed-by: David Hildenbrand <dahi@linux.vnet.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 arch/s390/kvm/gaccess.c | 25 ++++++++++++-------------
 1 file changed, 12 insertions(+), 13 deletions(-)

diff --git a/arch/s390/kvm/gaccess.c b/arch/s390/kvm/gaccess.c
index 33b5ae1..44d062a 100644
--- a/arch/s390/kvm/gaccess.c
+++ b/arch/s390/kvm/gaccess.c
@@ -330,6 +330,7 @@ static int deref_table(struct kvm *kvm, unsigned long gpa, unsigned long *val)
  * @vcpu: virtual cpu
  * @gva: guest virtual address
  * @gpa: points to where guest physical (absolute) address should be stored
+ * @asce: effective asce
  * @write: indicates if access is a write access
  *
  * Translate a guest virtual address into a guest absolute address by means
@@ -345,7 +346,8 @@ static int deref_table(struct kvm *kvm, unsigned long gpa, unsigned long *val)
  *	      by the architecture
  */
 static unsigned long guest_translate(struct kvm_vcpu *vcpu, unsigned long gva,
-				     unsigned long *gpa, int write)
+				     unsigned long *gpa, const union asce asce,
+				     int write)
 {
 	union vaddress vaddr = {.addr = gva};
 	union raddress raddr = {.addr = gva};
@@ -354,12 +356,10 @@ static unsigned long guest_translate(struct kvm_vcpu *vcpu, unsigned long gva,
 	union ctlreg0 ctlreg0;
 	unsigned long ptr;
 	int edat1, edat2;
-	union asce asce;
 
 	ctlreg0.val = vcpu->arch.sie_block->gcr[0];
 	edat1 = ctlreg0.edat && test_kvm_facility(vcpu->kvm, 8);
 	edat2 = edat1 && test_kvm_facility(vcpu->kvm, 78);
-	asce.val = get_vcpu_asce(vcpu);
 	if (asce.r)
 		goto real_address;
 	ptr = asce.origin * 4096;
@@ -506,15 +506,14 @@ static inline int is_low_address(unsigned long ga)
 	return (ga & ~0x11fful) == 0;
 }
 
-static int low_address_protection_enabled(struct kvm_vcpu *vcpu)
+static int low_address_protection_enabled(struct kvm_vcpu *vcpu,
+					  const union asce asce)
 {
 	union ctlreg0 ctlreg0 = {.val = vcpu->arch.sie_block->gcr[0]};
 	psw_t *psw = &vcpu->arch.sie_block->gpsw;
-	union asce asce;
 
 	if (!ctlreg0.lap)
 		return 0;
-	asce.val = get_vcpu_asce(vcpu);
 	if (psw_bits(*psw).t && asce.p)
 		return 0;
 	return 1;
@@ -536,7 +535,7 @@ enum {
 
 static int guest_page_range(struct kvm_vcpu *vcpu, unsigned long ga,
 			    unsigned long *pages, unsigned long nr_pages,
-			    int write)
+			    const union asce asce, int write)
 {
 	struct kvm_s390_pgm_info *pgm = &vcpu->arch.pgm;
 	psw_t *psw = &vcpu->arch.sie_block->gpsw;
@@ -547,7 +546,7 @@ static int guest_page_range(struct kvm_vcpu *vcpu, unsigned long ga,
 	tec_bits = (struct trans_exc_code_bits *)&pgm->trans_exc_code;
 	tec_bits->fsi = write ? FSI_STORE : FSI_FETCH;
 	tec_bits->as = psw_bits(*psw).as;
-	lap_enabled = low_address_protection_enabled(vcpu);
+	lap_enabled = low_address_protection_enabled(vcpu, asce);
 	while (nr_pages) {
 		ga = kvm_s390_logical_to_effective(vcpu, ga);
 		tec_bits->addr = ga >> PAGE_SHIFT;
@@ -557,7 +556,7 @@ static int guest_page_range(struct kvm_vcpu *vcpu, unsigned long ga,
 		}
 		ga &= PAGE_MASK;
 		if (psw_bits(*psw).t) {
-			rc = guest_translate(vcpu, ga, pages, write);
+			rc = guest_translate(vcpu, ga, pages, asce, write);
 			if (rc < 0)
 				return rc;
 			if (rc == PGM_PROTECTION)
@@ -604,7 +603,7 @@ int access_guest(struct kvm_vcpu *vcpu, unsigned long ga, ar_t ar, void *data,
 	need_ipte_lock = psw_bits(*psw).t && !asce.r;
 	if (need_ipte_lock)
 		ipte_lock(vcpu);
-	rc = guest_page_range(vcpu, ga, pages, nr_pages, write);
+	rc = guest_page_range(vcpu, ga, pages, nr_pages, asce, write);
 	for (idx = 0; idx < nr_pages && !rc; idx++) {
 		gpa = *(pages + idx) + (ga & ~PAGE_MASK);
 		_len = min(PAGE_SIZE - (gpa & ~PAGE_MASK), len);
@@ -671,16 +670,16 @@ int guest_translate_address(struct kvm_vcpu *vcpu, unsigned long gva, ar_t ar,
 	tec->as = psw_bits(*psw).as;
 	tec->fsi = write ? FSI_STORE : FSI_FETCH;
 	tec->addr = gva >> PAGE_SHIFT;
-	if (is_low_address(gva) && low_address_protection_enabled(vcpu)) {
+	asce.val = get_vcpu_asce(vcpu);
+	if (is_low_address(gva) && low_address_protection_enabled(vcpu, asce)) {
 		if (write) {
 			rc = pgm->code = PGM_PROTECTION;
 			return rc;
 		}
 	}
 
-	asce.val = get_vcpu_asce(vcpu);
 	if (psw_bits(*psw).t && !asce.r) {	/* Use DAT? */
-		rc = guest_translate(vcpu, gva, gpa, write);
+		rc = guest_translate(vcpu, gva, gpa, asce, write);
 		if (rc > 0) {
 			if (rc == PGM_PROTECTION)
 				tec->b61 = 1;
-- 
2.3.0

Re: [PATCH/RFC 7/9] KVM: s390: Optimize paths where get_vcpu_asce() is invoked

[Cornelia Huck]

On Mon, 16 Mar 2015 09:51:43 +0100
Christian Borntraeger <borntraeger@de.ibm.com> wrote:

> From: Alexander Yarygin <yarygin@linux.vnet.ibm.com>
> 
> During dynamic address translation the get_vcpu_asce()
> function can be invoked several times. It's ok for usual modes, but will
> be slow if CPUs are in AR mode. Let's call the get_vcpu_asce() once and
> pass the result to the called functions.
> 
> Signed-off-by: Alexander Yarygin <yarygin@linux.vnet.ibm.com>
> Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
> Reviewed-by: David Hildenbrand <dahi@linux.vnet.ibm.com>
> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
> ---
>  arch/s390/kvm/gaccess.c | 25 ++++++++++++-------------
>  1 file changed, 12 insertions(+), 13 deletions(-)

Acked-by: Cornelia Huck <cornelia.huck@de.ibm.com>

[PATCH/RFC 8/9] KVM: s390: Add access register mode

[Christian Borntraeger]

From: Alexander Yarygin <yarygin@linux.vnet.ibm.com>

Access register mode is one of the modes that control dynamic address
translation. In this mode the address space is specified by values of
the access registers. The effective address-space-control element is
obtained from the result of the access register translation. See
the "Access-Register Introduction" section of the chapter 5 "Program
Execution" in "Principles of Operations" for more details.

Signed-off-by: Alexander Yarygin <yarygin@linux.vnet.ibm.com>
Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 arch/s390/kvm/gaccess.c | 234 +++++++++++++++++++++++++++++++++++++++++-------
 arch/s390/kvm/gaccess.h |   3 +-
 2 files changed, 202 insertions(+), 35 deletions(-)

diff --git a/arch/s390/kvm/gaccess.c b/arch/s390/kvm/gaccess.c
index 44d062a..592c8860 100644
--- a/arch/s390/kvm/gaccess.c
+++ b/arch/s390/kvm/gaccess.c
@@ -10,6 +10,7 @@
 #include <asm/pgtable.h>
 #include "kvm-s390.h"
 #include "gaccess.h"
+#include <asm/switch_to.h>
 
 union asce {
 	unsigned long val;
@@ -207,6 +208,54 @@ union raddress {
 	unsigned long pfra : 52; /* Page-Frame Real Address */
 };
 
+union alet {
+	u32 val;
+	struct {
+		u32 reserved : 7;
+		u32 p        : 1;
+		u32 alesn    : 8;
+		u32 alen     : 16;
+	};
+};
+
+union ald {
+	u32 val;
+	struct {
+		u32     : 1;
+		u32 alo : 24;
+		u32 all : 7;
+	};
+};
+
+struct ale {
+	unsigned long i      : 1; /* ALEN-Invalid Bit */
+	unsigned long        : 5;
+	unsigned long fo     : 1; /* Fetch-Only Bit */
+	unsigned long p      : 1; /* Private Bit */
+	unsigned long alesn  : 8; /* Access-List-Entry Sequence Number */
+	unsigned long aleax  : 16; /* Access-List-Entry Authorization Index */
+	unsigned long        : 32;
+	unsigned long        : 1;
+	unsigned long asteo  : 25; /* ASN-Second-Table-Entry Origin */
+	unsigned long        : 6;
+	unsigned long astesn : 32; /* ASTE Sequence Number */
+} __packed;
+
+struct aste {
+	unsigned long i      : 1; /* ASX-Invalid Bit */
+	unsigned long ato    : 29; /* Authority-Table Origin */
+	unsigned long        : 1;
+	unsigned long b      : 1; /* Base-Space Bit */
+	unsigned long ax     : 16; /* Authorization Index */
+	unsigned long atl    : 12; /* Authority-Table Length */
+	unsigned long        : 2;
+	unsigned long ca     : 1; /* Controlled-ASN Bit */
+	unsigned long ra     : 1; /* Reusable-ASN Bit */
+	unsigned long asce   : 64; /* Address-Space-Control Element */
+	unsigned long ald    : 32;
+	unsigned long astesn : 32;
+	/* .. more fields there */
+} __packed;
 
 int ipte_lock_held(struct kvm_vcpu *vcpu)
 {
@@ -307,15 +356,157 @@ void ipte_unlock(struct kvm_vcpu *vcpu)
 		ipte_unlock_simple(vcpu);
 }
 
-static unsigned long get_vcpu_asce(struct kvm_vcpu *vcpu)
+static int ar_translation(struct kvm_vcpu *vcpu, union asce *asce, ar_t ar,
+			  int write)
+{
+	union alet alet;
+	struct ale ale;
+	struct aste aste;
+	unsigned long ald_addr, authority_table_addr;
+	union ald ald;
+	int eax, rc;
+	u8 authority_table;
+
+	if (ar == AR_INVAL || ar >= NUM_ACRS)
+		return -EINVAL;
+
+	save_access_regs(vcpu->run->s.regs.acrs);
+	alet.val = vcpu->run->s.regs.acrs[ar];
+
+	if (ar == 0 || alet.val == 0) {
+		asce->val = vcpu->arch.sie_block->gcr[1];
+		return 0;
+	} else if (alet.val == 1) {
+		asce->val = vcpu->arch.sie_block->gcr[7];
+		return 0;
+	}
+
+	if (alet.reserved)
+		return PGM_ALET_SPECIFICATION;
+
+	if (alet.p)
+		ald_addr = vcpu->arch.sie_block->gcr[5];
+	else
+		ald_addr = vcpu->arch.sie_block->gcr[2];
+	ald_addr &= 0x7fffffc0;
+
+	rc = read_guest_real(vcpu, ald_addr + 16, &ald.val, sizeof(union ald));
+	if (rc)
+		return rc;
+
+	if (alet.alen / 8 > ald.all)
+		return PGM_ALEN_TRANSLATION;
+
+	if (0x7fffffff - ald.alo * 128 < alet.alen * 16)
+		return PGM_ADDRESSING;
+
+	rc = read_guest_real(vcpu, ald.alo * 128 + alet.alen * 16, &ale,
+			     sizeof(struct ale));
+	if (rc)
+		return rc;
+
+	if (ale.i == 1)
+		return PGM_ALEN_TRANSLATION;
+	if (ale.alesn != alet.alesn)
+		return PGM_ALE_SEQUENCE;
+
+	rc = read_guest_real(vcpu, ale.asteo * 64, &aste, sizeof(struct aste));
+	if (rc)
+		return rc;
+
+	if (aste.i)
+		return PGM_ASTE_VALIDITY;
+	if (aste.astesn != ale.astesn)
+		return PGM_ASTE_SEQUENCE;
+
+	if (ale.p == 1) {
+		eax = (vcpu->arch.sie_block->gcr[8] >> 16) & 0xffff;
+		if (ale.aleax != eax) {
+			if (eax / 16 > aste.atl)
+				return PGM_EXTENDED_AUTHORITY;
+
+			authority_table_addr = aste.ato * 4 + eax / 4;
+
+			rc = read_guest_real(vcpu, authority_table_addr,
+					     &authority_table,
+					     sizeof(u8));
+			if (rc)
+				return rc;
+
+			if ((authority_table & (0x40 >> ((eax & 3) * 2))) == 0)
+				return PGM_EXTENDED_AUTHORITY;
+		}
+	}
+
+	if (ale.fo == 1 && write)
+		return PGM_PROTECTION;
+
+	asce->val = aste.asce;
+	return 0;
+}
+
+struct trans_exc_code_bits {
+	unsigned long addr : 52; /* Translation-exception Address */
+	unsigned long fsi  : 2;  /* Access Exception Fetch/Store Indication */
+	unsigned long	   : 6;
+	unsigned long b60  : 1;
+	unsigned long b61  : 1;
+	unsigned long as   : 2;  /* ASCE Identifier */
+};
+
+enum {
+	FSI_UNKNOWN = 0, /* Unknown wether fetch or store */
+	FSI_STORE   = 1, /* Exception was due to store operation */
+	FSI_FETCH   = 2  /* Exception was due to fetch operation */
+};
+
+static int get_vcpu_asce(struct kvm_vcpu *vcpu, union asce *asce,
+			 ar_t ar, int write)
 {
+	int rc;
+	psw_t *psw = &vcpu->arch.sie_block->gpsw;
+	struct kvm_s390_pgm_info *pgm = &vcpu->arch.pgm;
+	struct trans_exc_code_bits *tec_bits;
+
+	memset(pgm, 0, sizeof(*pgm));
+	tec_bits = (struct trans_exc_code_bits *)&pgm->trans_exc_code;
+	tec_bits->fsi = write ? FSI_STORE : FSI_FETCH;
+	tec_bits->as = psw_bits(*psw).as;
+
+	if (!psw_bits(*psw).t) {
+		asce->val = 0;
+		asce->r = 1;
+		return 0;
+	}
+
 	switch (psw_bits(vcpu->arch.sie_block->gpsw).as) {
 	case PSW_AS_PRIMARY:
-		return vcpu->arch.sie_block->gcr[1];
+		asce->val = vcpu->arch.sie_block->gcr[1];
+		return 0;
 	case PSW_AS_SECONDARY:
-		return vcpu->arch.sie_block->gcr[7];
+		asce->val = vcpu->arch.sie_block->gcr[7];
+		return 0;
 	case PSW_AS_HOME:
-		return vcpu->arch.sie_block->gcr[13];
+		asce->val = vcpu->arch.sie_block->gcr[13];
+		return 0;
+	case PSW_AS_ACCREG:
+		rc = ar_translation(vcpu, asce, ar, write);
+		switch (rc) {
+		case PGM_ALEN_TRANSLATION:
+		case PGM_ALE_SEQUENCE:
+		case PGM_ASTE_VALIDITY:
+		case PGM_ASTE_SEQUENCE:
+		case PGM_EXTENDED_AUTHORITY:
+			vcpu->arch.pgm.exc_access_id = ar;
+			break;
+		case PGM_PROTECTION:
+			tec_bits->b60 = 1;
+			tec_bits->b61 = 1;
+			break;
+		}
+		if (rc > 0)
+			pgm->code = rc;
+		return rc;
 	}
 	return 0;
 }
@@ -519,20 +710,6 @@ static int low_address_protection_enabled(struct kvm_vcpu *vcpu,
 	return 1;
 }
 
-struct trans_exc_code_bits {
-	unsigned long addr : 52; /* Translation-exception Address */
-	unsigned long fsi  : 2;  /* Access Exception Fetch/Store Indication */
-	unsigned long	   : 7;
-	unsigned long b61  : 1;
-	unsigned long as   : 2;  /* ASCE Identifier */
-};
-
-enum {
-	FSI_UNKNOWN = 0, /* Unknown wether fetch or store */
-	FSI_STORE   = 1, /* Exception was due to store operation */
-	FSI_FETCH   = 2  /* Exception was due to fetch operation */
-};
-
 static int guest_page_range(struct kvm_vcpu *vcpu, unsigned long ga,
 			    unsigned long *pages, unsigned long nr_pages,
 			    const union asce asce, int write)
@@ -542,10 +719,7 @@ static int guest_page_range(struct kvm_vcpu *vcpu, unsigned long ga,
 	struct trans_exc_code_bits *tec_bits;
 	int lap_enabled, rc;
 
-	memset(pgm, 0, sizeof(*pgm));
 	tec_bits = (struct trans_exc_code_bits *)&pgm->trans_exc_code;
-	tec_bits->fsi = write ? FSI_STORE : FSI_FETCH;
-	tec_bits->as = psw_bits(*psw).as;
 	lap_enabled = low_address_protection_enabled(vcpu, asce);
 	while (nr_pages) {
 		ga = kvm_s390_logical_to_effective(vcpu, ga);
@@ -590,16 +764,15 @@ int access_guest(struct kvm_vcpu *vcpu, unsigned long ga, ar_t ar, void *data,
 
 	if (!len)
 		return 0;
-	/* Access register mode is not supported yet. */
-	if (psw_bits(*psw).t && psw_bits(*psw).as == PSW_AS_ACCREG)
-		return -EOPNOTSUPP;
+	rc = get_vcpu_asce(vcpu, &asce, ar, write);
+	if (rc)
+		return rc;
 	nr_pages = (((ga & ~PAGE_MASK) + len - 1) >> PAGE_SHIFT) + 1;
 	pages = pages_array;
 	if (nr_pages > ARRAY_SIZE(pages_array))
 		pages = vmalloc(nr_pages * sizeof(unsigned long));
 	if (!pages)
 		return -ENOMEM;
-	asce.val = get_vcpu_asce(vcpu);
 	need_ipte_lock = psw_bits(*psw).t && !asce.r;
 	if (need_ipte_lock)
 		ipte_lock(vcpu);
@@ -660,17 +833,12 @@ int guest_translate_address(struct kvm_vcpu *vcpu, unsigned long gva, ar_t ar,
 	union asce asce;
 	int rc;
 
-	/* Access register mode is not supported yet. */
-	if (psw_bits(*psw).t && psw_bits(*psw).as == PSW_AS_ACCREG)
-		return -EOPNOTSUPP;
-
 	gva = kvm_s390_logical_to_effective(vcpu, gva);
-	memset(pgm, 0, sizeof(*pgm));
 	tec = (struct trans_exc_code_bits *)&pgm->trans_exc_code;
-	tec->as = psw_bits(*psw).as;
-	tec->fsi = write ? FSI_STORE : FSI_FETCH;
+	rc = get_vcpu_asce(vcpu, &asce, ar, write);
 	tec->addr = gva >> PAGE_SHIFT;
-	asce.val = get_vcpu_asce(vcpu);
+	if (rc)
+		return rc;
 	if (is_low_address(gva) && low_address_protection_enabled(vcpu, asce)) {
 		if (write) {
 			rc = pgm->code = PGM_PROTECTION;
diff --git a/arch/s390/kvm/gaccess.h b/arch/s390/kvm/gaccess.h
index 68bf4f7..ef03726 100644
--- a/arch/s390/kvm/gaccess.h
+++ b/arch/s390/kvm/gaccess.h
@@ -179,8 +179,7 @@ int access_guest_real(struct kvm_vcpu *vcpu, unsigned long gra,
  * If DAT is off data will be copied to guest real or absolute memory.
  * If DAT is on data will be copied to the address space as specified by
  * the address space bits of the PSW:
- * Primary, secondory or home space (access register mode is currently not
- * implemented).
+ * Primary, secondary, home space or access register mode.
  * The addressing mode of the PSW is also inspected, so that address wrap
  * around is taken into account for 24-, 31- and 64-bit addressing mode,
  * if the to be copied data crosses page boundaries in guest address space.
-- 
2.3.0

[PATCH/RFC 9/9] KVM: s390: Create ioctl for Getting/Setting guest storage keys

[Christian Borntraeger]

From: "Jason J. Herne" <jjherne@linux.vnet.ibm.com>

Provide the KVM_S390_GET_SKEYS and KVM_S390_SET_SKEYS ioctl which can be used
to get/set guest storage keys. This functionality is needed for live migration
of s390 guests that use storage keys.

Signed-off-by: Jason J. Herne <jjherne@linux.vnet.ibm.com>
Reviewed-by: David Hildenbrand <dahi@linux.vnet.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 Documentation/virtual/kvm/api.txt |  58 ++++++++++++++++++
 arch/s390/kvm/kvm-s390.c          | 122 ++++++++++++++++++++++++++++++++++++++
 include/uapi/linux/kvm.h          |  14 +++++
 3 files changed, 194 insertions(+)

diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 74d18b6..589b9ac 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2762,6 +2762,64 @@ register number to be used.
 The "reserved" field is meant for future extensions. It is not used by
 KVM with the currently defined set of flags.
 
+4.90 KVM_S390_GET_SKEYS
+
+Capability: KVM_CAP_S390_SKEYS
+Architectures: s390
+Type: vm ioctl
+Parameters: struct kvm_s390_skeys
+Returns: 0 on success, KVM_S390_GET_KEYS_NONE if guest is not using storage
+         keys, negative value on error
+
+This ioctl is used to get guest storage key values on the s390
+architecture. The ioctl takes parameters via the kvm_s390_skeys struct.
+
+struct kvm_s390_skeys {
+	__u64 start_gfn;
+	__u64 count;
+	__u64 skeydata_addr;
+	__u32 flags;
+	__u32 reserved[9];
+};
+
+The start_gfn field is the number of the first guest frame whose storage keys
+you want to get.
+
+The count field is the number of consecutive frames (starting from start_gfn)
+whose storage keys to get. The count field must be at least 1 and the maximum
+allowed value is defined as KVM_S390_SKEYS_ALLOC_MAX. Values outside this range
+will cause the ioctl to return -EINVAL.
+
+The skeydata_addr field is the address to a buffer large enough to hold count
+bytes. This buffer will be filled with storage key data by the ioctl.
+
+4.91 KVM_S390_SET_SKEYS
+
+Capability: KVM_CAP_S390_SKEYS
+Architectures: s390
+Type: vm ioctl
+Parameters: struct kvm_s390_skeys
+Returns: 0 on success, negative value on error
+
+This ioctl is used to set guest storage key values on the s390
+architecture. The ioctl takes parameters via the kvm_s390_skeys struct.
+See section on KVM_S390_GET_SKEYS for struct definition.
+
+The start_gfn field is the number of the first guest frame whose storage keys
+you want to set.
+
+The count field is the number of consecutive frames (starting from start_gfn)
+whose storage keys to get. The count field must be at least 1 and the maximum
+allowed value is defined as KVM_S390_SKEYS_ALLOC_MAX. Values outside this range
+will cause the ioctl to return -EINVAL.
+
+The skeydata_addr field is the address to a buffer containing count bytes of
+storage keys. Each byte in the buffer will be set as the storage key for a
+single frame starting at start_gfn for count frames.
+
+Note: If any architecturally invalid key value is found in the given data then
+the ioctl will return -EINVAL.
+
 5. The kvm_run structure
 ------------------------
 
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index fdfa106..16d33f0 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -729,6 +729,108 @@ static int kvm_s390_vm_has_attr(struct kvm *kvm, struct kvm_device_attr *attr)
 	return ret;
 }
 
+static long kvm_s390_get_skeys(struct kvm *kvm, struct kvm_s390_skeys *args)
+{
+	uint8_t *keys;
+	uint64_t hva;
+	unsigned long curkey;
+	int i, r = 0;
+
+	if (args->flags != 0)
+		return -EINVAL;
+
+	/* Is this guest using storage keys? */
+	if (!mm_use_skey(current->mm))
+		return KVM_S390_GET_SKEYS_NONE;
+
+	/* Enforce sane limit on memory allocation */
+	if (args->count < 1 || args->count > KVM_S390_SKEYS_MAX)
+		return -EINVAL;
+
+	keys = kmalloc_array(args->count, sizeof(uint8_t),
+			     GFP_KERNEL | __GFP_NOWARN);
+	if (!keys)
+		keys = vmalloc(sizeof(uint8_t) * args->count);
+	if (!keys)
+		return -ENOMEM;
+
+	for (i = 0; i < args->count; i++) {
+		hva = gfn_to_hva(kvm, args->start_gfn + i);
+		if (kvm_is_error_hva(hva)) {
+			r = -EFAULT;
+			goto out;
+		}
+
+		curkey = get_guest_storage_key(current->mm, hva);
+		if (IS_ERR_VALUE(curkey)) {
+			r = curkey;
+			goto out;
+		}
+		keys[i] = curkey;
+	}
+
+	r = copy_to_user((uint8_t __user *)args->skeydata_addr, keys,
+			 sizeof(uint8_t) * args->count);
+	if (r)
+		r = -EFAULT;
+out:
+	kvfree(keys);
+	return r;
+}
+
+static long kvm_s390_set_skeys(struct kvm *kvm, struct kvm_s390_skeys *args)
+{
+	uint8_t *keys;
+	uint64_t hva;
+	int i, r = 0;
+
+	if (args->flags != 0)
+		return -EINVAL;
+
+	/* Enforce sane limit on memory allocation */
+	if (args->count < 1 || args->count > KVM_S390_SKEYS_MAX)
+		return -EINVAL;
+
+	keys = kmalloc_array(args->count, sizeof(uint8_t),
+			     GFP_KERNEL | __GFP_NOWARN);
+	if (!keys)
+		keys = vmalloc(sizeof(uint8_t) * args->count);
+	if (!keys)
+		return -ENOMEM;
+
+	r = copy_from_user(keys, (uint8_t __user *)args->skeydata_addr,
+			   sizeof(uint8_t) * args->count);
+	if (r) {
+		r = -EFAULT;
+		goto out;
+	}
+
+	/* Enable storage key handling for the guest */
+	s390_enable_skey();
+
+	for (i = 0; i < args->count; i++) {
+		hva = gfn_to_hva(kvm, args->start_gfn + i);
+		if (kvm_is_error_hva(hva)) {
+			r = -EFAULT;
+			goto out;
+		}
+
+		/* Lowest order bit is reserved */
+		if (keys[i] & 0x01) {
+			r = -EINVAL;
+			goto out;
+		}
+
+		r = set_guest_storage_key(current->mm, hva,
+					  (unsigned long)keys[i], 0);
+		if (r)
+			goto out;
+	}
+out:
+	kvfree(keys);
+	return r;
+}
+
 long kvm_arch_vm_ioctl(struct file *filp,
 		       unsigned int ioctl, unsigned long arg)
 {
@@ -788,6 +890,26 @@ long kvm_arch_vm_ioctl(struct file *filp,
 		r = kvm_s390_vm_has_attr(kvm, &attr);
 		break;
 	}
+	case KVM_S390_GET_SKEYS: {
+		struct kvm_s390_skeys args;
+
+		r = -EFAULT;
+		if (copy_from_user(&args, argp,
+				   sizeof(struct kvm_s390_skeys)))
+			break;
+		r = kvm_s390_get_skeys(kvm, &args);
+		break;
+	}
+	case KVM_S390_SET_SKEYS: {
+		struct kvm_s390_skeys args;
+
+		r = -EFAULT;
+		if (copy_from_user(&args, argp,
+				   sizeof(struct kvm_s390_skeys)))
+			break;
+		r = kvm_s390_set_skeys(kvm, &args);
+		break;
+	}
 	default:
 		r = -ENOTTY;
 	}
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 279ffd2..55dfd4c 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -147,6 +147,16 @@ struct kvm_pit_config {
 
 #define KVM_PIT_SPEAKER_DUMMY     1
 
+struct kvm_s390_skeys {
+	__u64 start_gfn;
+	__u64 count;
+	__u64 skeydata_addr;
+	__u32 flags;
+	__u32 reserved[9];
+};
+#define KVM_S390_GET_SKEYS_NONE   1
+#define KVM_S390_SKEYS_MAX        1048576
+
 #define KVM_EXIT_UNKNOWN          0
 #define KVM_EXIT_EXCEPTION        1
 #define KVM_EXIT_IO               2
@@ -791,6 +801,7 @@ struct kvm_ppc_smmu_info {
 #define KVM_CAP_S390_VECTOR_REGISTERS 107
 #define KVM_CAP_S390_MEM_OP 108
 #define KVM_CAP_S390_USER_STSI 109
+#define KVM_CAP_S390_SKEYS 110
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
@@ -1168,6 +1179,9 @@ struct kvm_s390_ucas_mapping {
 #define KVM_GET_REG_LIST	  _IOWR(KVMIO, 0xb0, struct kvm_reg_list)
 /* Available with KVM_CAP_S390_MEM_OP */
 #define KVM_S390_MEM_OP		  _IOW(KVMIO,  0xb1, struct kvm_s390_mem_op)
+/* Available with KVM_CAP_S390_SKEYS */
+#define KVM_S390_GET_SKEYS      _IOW(KVMIO, 0xb2, struct kvm_s390_skeys)
+#define KVM_S390_SET_SKEYS      _IOW(KVMIO, 0xb3, struct kvm_s390_skeys)
 
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3		(1 << 1)
-- 
2.3.0

Re: [PATCH/RFC 9/9] KVM: s390: Create ioctl for Getting/Setting guest storage keys

[Cornelia Huck]

On Mon, 16 Mar 2015 09:51:45 +0100
Christian Borntraeger <borntraeger@de.ibm.com> wrote:

> From: "Jason J. Herne" <jjherne@linux.vnet.ibm.com>
> 
> Provide the KVM_S390_GET_SKEYS and KVM_S390_SET_SKEYS ioctl which can be used
> to get/set guest storage keys. This functionality is needed for live migration
> of s390 guests that use storage keys.
> 
> Signed-off-by: Jason J. Herne <jjherne@linux.vnet.ibm.com>
> Reviewed-by: David Hildenbrand <dahi@linux.vnet.ibm.com>
> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
> ---
>  Documentation/virtual/kvm/api.txt |  58 ++++++++++++++++++
>  arch/s390/kvm/kvm-s390.c          | 122 ++++++++++++++++++++++++++++++++++++++
>  include/uapi/linux/kvm.h          |  14 +++++
>  3 files changed, 194 insertions(+)
> 

> @@ -791,6 +801,7 @@ struct kvm_ppc_smmu_info {
>  #define KVM_CAP_S390_VECTOR_REGISTERS 107
>  #define KVM_CAP_S390_MEM_OP 108
>  #define KVM_CAP_S390_USER_STSI 109
> +#define KVM_CAP_S390_SKEYS 110

Hm... did you forget to actually advertise this cap?

> 
>  #ifdef KVM_CAP_IRQ_ROUTING
> 

Re: [PATCH/RFC 4/9] KVM: s390: Add MEMOP ioctls for reading/writing guest memory

[Thomas Huth]

> Date: Wed, 18 Mar 2015 21:23:48 -0300
> From: Marcelo Tosatti <mtosatti@redhat.com>
>
> On Mon, Mar 16, 2015 at 09:51:40AM +0100, Christian Borntraeger wrote:
> > From: Thomas Huth <thuth@linux.vnet.ibm.com>
> >
> > On s390, we've got to make sure to hold the IPTE lock while accessing
> > logical memory. So let's add an ioctl for reading and writing logical
> > memory to provide this feature for userspace, too.
...
>
> What i was wondering is why you can't translate the address
> in the kernel and let userspace perform the actual read/write?

The idea here is to protect the read/write access with the ipte-lock, too.
That way, the whole address translation _and_ the read/write access are
protected together against invalidate-page-table operations from other
CPUs,
so the whole memory access looks atomic for other VCPUs. And since we do
not
want to expose the ipte lock directly to user space, both has to be done
in the kernel.

We already had a long internal discussion about this in our team, and
indeed, if the ipte-lock would be the only "problem" that we face on s390,
we might also come up with a solution where the memory read/write access
is done in userspace instead. However, for full architecture compliance,
we later have got to support the so-called "storage keys" during memory
accesses, too, and this can hardly be done accurately and safely from
userspace. So I'm afraid, it's somewhat ugly that we've got to provide an
ioctl here to read/write the guest memory, but it's the only feasible
solution that I could think of.

 Thomas

Re: [PATCH/RFC 4/9] KVM: s390: Add MEMOP ioctls for reading/writing guest memory

[Marcelo Tosatti]

On Thu, Mar 19, 2015 at 02:15:18PM +0100, Thomas Huth wrote:
> > Date: Wed, 18 Mar 2015 21:23:48 -0300
> > From: Marcelo Tosatti <mtosatti@redhat.com>
> >
> > On Mon, Mar 16, 2015 at 09:51:40AM +0100, Christian Borntraeger wrote:
> > > From: Thomas Huth <thuth@linux.vnet.ibm.com>
> > >
> > > On s390, we've got to make sure to hold the IPTE lock while accessing
> > > logical memory. So let's add an ioctl for reading and writing logical
> > > memory to provide this feature for userspace, too.
> ...
> >
> > What i was wondering is why you can't translate the address
> > in the kernel and let userspace perform the actual read/write?
> 
> The idea here is to protect the read/write access with the ipte-lock, too.
> That way, the whole address translation _and_ the read/write access are
> protected together against invalidate-page-table operations from other
> CPUs,
> so the whole memory access looks atomic for other VCPUs. And since we do
> not
> want to expose the ipte lock directly to user space, both has to be done
> in the kernel.
> We already had a long internal discussion about this in our team, and
> indeed, if the ipte-lock would be the only "problem" that we face on s390,
> we might also come up with a solution where the memory read/write access
> is done in userspace instead. However, for full architecture compliance,
> we later have got to support the so-called "storage keys" during memory
> accesses, too, and this can hardly be done accurately and safely from
> userspace. So I'm afraid, it's somewhat ugly that we've got to provide an
> ioctl here to read/write the guest memory, but it's the only feasible
> solution that I could think of.

I see, thanks.