[kvm-unit-tests PATCH 0/3] s390x: Add PV SIE intercepts and ipl tests

[kvm-unit-tests PATCH 0/3] s390x: Add PV SIE intercepts and ipl tests

[Janosch Frank]

Extend the coverage of the UVC interface.
The patches might be a bit dusty, they've been on a branch for a while.

Janosch Frank (3):
  lib: s390x: Introduce UV validity function
  s390x: pv: Test sie entry intercepts and validities
  s390x: pv: Add IPL reset tests

 lib/s390x/uv.h                                |   7 +
 s390x/Makefile                                |   7 +
 s390x/pv-icptcode.c                           | 366 ++++++++++++++++++
 s390x/pv-ipl.c                                | 237 ++++++++++++
 s390x/snippets/asm/snippet-loop.S             |  12 +
 s390x/snippets/asm/snippet-pv-diag-308.S      |  67 ++++
 s390x/snippets/asm/snippet-pv-icpt-112.S      |  77 ++++
 s390x/snippets/asm/snippet-pv-icpt-loop.S     |  15 +
 .../snippets/asm/snippet-pv-icpt-vir-timing.S |  22 ++
 9 files changed, 810 insertions(+)
 create mode 100644 s390x/pv-icptcode.c
 create mode 100644 s390x/pv-ipl.c
 create mode 100644 s390x/snippets/asm/snippet-loop.S
 create mode 100644 s390x/snippets/asm/snippet-pv-diag-308.S
 create mode 100644 s390x/snippets/asm/snippet-pv-icpt-112.S
 create mode 100644 s390x/snippets/asm/snippet-pv-icpt-loop.S
 create mode 100644 s390x/snippets/asm/snippet-pv-icpt-vir-timing.S

-- 
2.34.1

[kvm-unit-tests PATCH 1/3] lib: s390x: Introduce UV validity function

[Janosch Frank]

PV related validities are in the 0x20** range but the last byte might
be implementation specific, so everytime we check for a UV validity we
need to mask the last byte.

Let's add a function that checks for a UV validity and returns a
boolean.

Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
---
 lib/s390x/uv.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/lib/s390x/uv.h b/lib/s390x/uv.h
index 5fe29bda..78b979b7 100644
--- a/lib/s390x/uv.h
+++ b/lib/s390x/uv.h
@@ -35,4 +35,11 @@ static inline void uv_setup_asces(void)
 	lctlg(13, asce);
 }
 
+static inline bool uv_validity_check(struct vm *vm)
+{
+	uint16_t vir = sie_get_validity(vm);
+
+	return vm->sblk->icptcode == ICPT_VALIDITY && (vir & 0xff00) == 0x2000;
+}
+
 #endif /* UV_H */
-- 
2.34.1

[kvm-unit-tests PATCH 2/3] s390x: pv: Test sie entry intercepts and validities

[Janosch Frank]

The lowcore is an important part of any s390 cpu so we need to make
sure it's always available when we virtualize one. For non-PV guests
that would mean ensuring that the lowcore page is read and writable by
the guest.

For PV guests we additionally need to make sure that the page is owned
by the guest as it is only allowed to access them if that's the
case. The code 112 SIE intercept tells us if the lowcore pages aren't
secure anymore.

Let's check if that intercept is reported by SIE if we export the
lowcore pages. Additionally check if that's also the case if the guest
shares the lowcore which will make it readable to the host but
ownership of the page should not change.

Also we check for validities in these conditions:
     * Manipulated cpu timer
     * Double SIE for same vcpu
     * Re-use of VCPU handle from another secure configuration
     * ASCE re-use

Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
---
 s390x/Makefile                                |   5 +
 s390x/pv-icptcode.c                           | 366 ++++++++++++++++++
 s390x/snippets/asm/snippet-loop.S             |  12 +
 s390x/snippets/asm/snippet-pv-icpt-112.S      |  77 ++++
 s390x/snippets/asm/snippet-pv-icpt-loop.S     |  15 +
 .../snippets/asm/snippet-pv-icpt-vir-timing.S |  22 ++
 6 files changed, 497 insertions(+)
 create mode 100644 s390x/pv-icptcode.c
 create mode 100644 s390x/snippets/asm/snippet-loop.S
 create mode 100644 s390x/snippets/asm/snippet-pv-icpt-112.S
 create mode 100644 s390x/snippets/asm/snippet-pv-icpt-loop.S
 create mode 100644 s390x/snippets/asm/snippet-pv-icpt-vir-timing.S

diff --git a/s390x/Makefile b/s390x/Makefile
index 97a61611..858f5af4 100644
--- a/s390x/Makefile
+++ b/s390x/Makefile
@@ -41,6 +41,7 @@ tests += $(TEST_DIR)/migration-sck.elf
 tests += $(TEST_DIR)/exittime.elf
 
 pv-tests += $(TEST_DIR)/pv-diags.elf
+pv-tests += $(TEST_DIR)/pv-icptcode.elf
 
 ifneq ($(HOST_KEY_DOCUMENT),)
 ifneq ($(GEN_SE_HEADER),)
@@ -119,6 +120,10 @@ $(TEST_DIR)/spec_ex-sie.elf: snippets = $(SNIPPET_DIR)/c/spec_ex.gbin
 $(TEST_DIR)/pv-diags.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-diag-yield.gbin
 $(TEST_DIR)/pv-diags.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-diag-288.gbin
 $(TEST_DIR)/pv-diags.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-diag-500.gbin
+$(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-112.gbin
+$(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-loop.gbin
+$(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-loop.gbin
+$(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-vir-timing.gbin
 
 ifneq ($(GEN_SE_HEADER),)
 snippets += $(pv-snippets)
diff --git a/s390x/pv-icptcode.c b/s390x/pv-icptcode.c
new file mode 100644
index 00000000..1a2a4123
--- /dev/null
+++ b/s390x/pv-icptcode.c
@@ -0,0 +1,366 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * PV virtualization interception tests for intercepts that are not
+ * caused by an instruction.
+ *
+ * Copyright (c) 2023 IBM Corp
+ *
+ * Authors:
+ *  Janosch Frank <frankja@linux.ibm.com>
+ */
+#include <libcflat.h>
+#include <sie.h>
+#include <smp.h>
+#include <sclp.h>
+#include <snippet.h>
+#include <asm/facility.h>
+#include <asm/barrier.h>
+#include <asm/sigp.h>
+#include <asm/uv.h>
+
+static struct vm vm, vm2;
+
+static void test_validity_timing(void)
+{
+	extern const char SNIPPET_NAME_START(asm, snippet_pv_icpt_vir_timing)[];
+	extern const char SNIPPET_NAME_END(asm, snippet_pv_icpt_vir_timing)[];
+	extern const char SNIPPET_HDR_START(asm, snippet_pv_icpt_vir_timing)[];
+	extern const char SNIPPET_HDR_END(asm, snippet_pv_icpt_vir_timing)[];
+	int size_hdr = SNIPPET_HDR_LEN(asm, snippet_pv_icpt_vir_timing);
+	int size_gbin = SNIPPET_LEN(asm, snippet_pv_icpt_vir_timing);
+
+	report_prefix_push("manipulated cpu time");
+	snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_icpt_vir_timing),
+			SNIPPET_HDR_START(asm, snippet_pv_icpt_vir_timing),
+			size_gbin, size_hdr, SNIPPET_UNPACK_OFF);
+
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
+	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x44,
+	       "stp done");
+	vm.sblk->cputm -= 0x280de80000 / 2;
+
+	sie_expect_validity(&vm);
+	sie(&vm);
+	report(uv_validity_check(&vm), "validity");
+	uv_destroy_guest(&vm);
+	report_prefix_pop();
+}
+
+static void run_loop(void)
+{
+	sie(&vm);
+	sigp_retry(stap(), SIGP_STOP, 0, NULL);
+}
+
+static void test_validity_already_running(void)
+{
+	extern const char SNIPPET_NAME_START(asm, snippet_loop)[];
+	extern const char SNIPPET_NAME_END(asm, snippet_loop)[];
+	extern const char SNIPPET_HDR_START(asm, snippet_loop)[];
+	extern const char SNIPPET_HDR_END(asm, snippet_loop)[];
+	int size_hdr = SNIPPET_HDR_LEN(asm, snippet_loop);
+	int size_gbin = SNIPPET_LEN(asm, snippet_loop);
+	struct psw psw = {
+		.mask = PSW_MASK_64,
+		.addr = (uint64_t)run_loop,
+	};
+
+	report_prefix_push("already running");
+	if (smp_query_num_cpus() < 3) {
+		report_skip("need at least 3 cpus for this test");
+		goto out;
+	}
+
+	snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_loop),
+			SNIPPET_HDR_START(asm, snippet_loop),
+			size_gbin, size_hdr, SNIPPET_UNPACK_OFF);
+
+	sie_expect_validity(&vm);
+	smp_cpu_setup(1, psw);
+	smp_cpu_setup(2, psw);
+	while (vm.sblk->icptcode != ICPT_VALIDITY) { mb(); }
+	/* Yes I know this is not reliable as one cpu might overwrite it */
+	report(uv_validity_check(&vm), "validity");
+	smp_cpu_stop(1);
+	smp_cpu_stop(2);
+	uv_destroy_guest(&vm);
+
+out:
+	report_prefix_pop();
+}
+
+/* Tests if a vcpu handle from another configuration results in a validity intercept. */
+static void test_validity_handle_not_in_config(void)
+{
+	extern const char SNIPPET_NAME_START(asm, snippet_pv_icpt_loop)[];
+	extern const char SNIPPET_NAME_END(asm, snippet_pv_icpt_loop)[];
+	extern const char SNIPPET_HDR_START(asm, snippet_pv_icpt_loop)[];
+	extern const char SNIPPET_HDR_END(asm, snippet_pv_icpt_loop)[];
+	int size_hdr = SNIPPET_HDR_LEN(asm, snippet_pv_icpt_loop);
+	int size_gbin = SNIPPET_LEN(asm, snippet_pv_icpt_loop);
+
+	report_prefix_push("handle not in config");
+	/* Setup our primary vm */
+	snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_icpt_loop),
+			SNIPPET_HDR_START(asm, snippet_pv_icpt_loop),
+			size_gbin, size_hdr, SNIPPET_UNPACK_OFF);
+
+	/* Setup secondary vm */
+	snippet_setup_guest(&vm2, true);
+	snippet_pv_init(&vm2, SNIPPET_NAME_START(asm, snippet_pv_icpt_loop),
+			SNIPPET_HDR_START(asm, snippet_pv_icpt_loop),
+			size_gbin, size_hdr, SNIPPET_UNPACK_OFF);
+
+	vm.sblk->pv_handle_cpu = vm2.sblk->pv_handle_cpu;
+	sie_expect_validity(&vm);
+	sie(&vm);
+	report(uv_validity_check(&vm), "validity");
+
+	/* Restore cpu handle and destroy the second vm */
+	vm.sblk->pv_handle_cpu = vm.uv.vcpu_handle;
+	uv_destroy_guest(&vm2);
+	sie_guest_destroy(&vm2);
+
+	uv_destroy_guest(&vm);
+	report_prefix_pop();
+}
+
+/* Tests if a wrong vm or vcpu handle results in a validity intercept. */
+static void test_validity_seid(void)
+{
+	extern const char SNIPPET_NAME_START(asm, snippet_pv_icpt_loop)[];
+	extern const char SNIPPET_NAME_END(asm, snippet_pv_icpt_loop)[];
+	extern const char SNIPPET_HDR_START(asm, snippet_pv_icpt_loop)[];
+	extern const char SNIPPET_HDR_END(asm, snippet_pv_icpt_loop)[];
+	int size_hdr = SNIPPET_HDR_LEN(asm, snippet_pv_icpt_loop);
+	int size_gbin = SNIPPET_LEN(asm, snippet_pv_icpt_loop);
+	int fails = 0;
+	int i;
+
+	report_prefix_push("handles");
+	snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_icpt_loop),
+			SNIPPET_HDR_START(asm, snippet_pv_icpt_loop),
+			size_gbin, size_hdr, SNIPPET_UNPACK_OFF);
+
+	for (i = 0; i < 64; i++) {
+		vm.sblk->pv_handle_config ^= 1UL << i;
+		sie_expect_validity(&vm);
+		sie(&vm);
+		if (!uv_validity_check(&vm)) {
+			report_fail("SIE accepted wrong VM SEID, changed bit %d",
+				    63 - i);
+			fails++;
+		}
+		vm.sblk->pv_handle_config ^= 1UL << i;
+	}
+	report(!fails, "No wrong vm handle accepted");
+
+	fails = 0;
+	for (i = 0; i < 64; i++) {
+		vm.sblk->pv_handle_cpu ^= 1UL << i;
+		sie_expect_validity(&vm);
+		sie(&vm);
+		if (!uv_validity_check(&vm)) {
+			report_fail("SIE accepted wrong CPU SEID, changed bit %d",
+				    63 - i);
+			fails++;
+		}
+		vm.sblk->pv_handle_cpu ^= 1UL << i;
+	}
+	report(!fails, "No wrong cpu handle accepted");
+
+	uv_destroy_guest(&vm);
+	report_prefix_pop();
+}
+
+/*
+ * Tests if we get a validity intercept if the CR1 asce at SIE entry
+ * is not the same as the one given at the UV creation of the VM.
+ */
+static void test_validity_asce(void)
+{
+	extern const char SNIPPET_NAME_START(asm, snippet_pv_icpt_112)[];
+	extern const char SNIPPET_NAME_END(asm, snippet_pv_icpt_112)[];
+	extern const char SNIPPET_HDR_START(asm, snippet_pv_icpt_112)[];
+	extern const char SNIPPET_HDR_END(asm, snippet_pv_icpt_112)[];
+	int size_hdr = SNIPPET_HDR_LEN(asm, snippet_pv_icpt_112);
+	int size_gbin = SNIPPET_LEN(asm, snippet_pv_icpt_112);
+	uint64_t asce_old, asce_new;
+	void *pgd_new, *pgd_old;
+
+	report_prefix_push("asce");
+	snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_icpt_112),
+			SNIPPET_HDR_START(asm, snippet_pv_icpt_112),
+			size_gbin, size_hdr, SNIPPET_UNPACK_OFF);
+
+	asce_old = vm.save_area.guest.asce;
+	pgd_new = memalign_pages_flags(PAGE_SIZE, PAGE_SIZE * 4, 0);
+	pgd_old = (void *)(asce_old & PAGE_MASK);
+
+	/* Copy the contents of the top most table */
+	memcpy(pgd_new, pgd_old, PAGE_SIZE * 4);
+
+	/* Create the replacement ASCE */
+	asce_new = __pa(pgd_new) | ASCE_DT_REGION1 | REGION_TABLE_LENGTH | ASCE_P;
+	vm.save_area.guest.asce = asce_new;
+
+	sie_expect_validity(&vm);
+	sie(&vm);
+	report(uv_validity_check(&vm), "wrong CR1 validity");
+
+	/* Restore the old ASCE */
+	vm.save_area.guest.asce = asce_old;
+
+	/* Try if we can still do an entry with the correct asce */
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
+	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x44,
+	       "re-entry with valid CR1");
+	uv_destroy_guest(&vm);
+	free_pages(pgd_new);
+	report_prefix_pop();
+}
+
+static void test_icpt_112(void)
+{
+	extern const char SNIPPET_NAME_START(asm, snippet_pv_icpt_112)[];
+	extern const char SNIPPET_NAME_END(asm, snippet_pv_icpt_112)[];
+	extern const char SNIPPET_HDR_START(asm, snippet_pv_icpt_112)[];
+	extern const char SNIPPET_HDR_END(asm, snippet_pv_icpt_112)[];
+	int size_hdr = SNIPPET_HDR_LEN(asm, snippet_pv_icpt_112);
+	int size_gbin = SNIPPET_LEN(asm, snippet_pv_icpt_112);
+
+	u64 lc_off = 0;
+
+	report_prefix_push("prefix");
+
+	snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_icpt_112),
+			SNIPPET_HDR_START(asm, snippet_pv_icpt_112),
+			size_gbin, size_hdr, SNIPPET_UNPACK_OFF);
+
+	report_prefix_push("0x0");
+	sie(&vm);
+
+	/* Guest indicates that it has been setup via the diag 0x44 */
+	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
+	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x44,
+	       "guest set up");
+
+	/*
+	 * Let's export the standard prefix 0x0 and check for the 112
+	 * intercept.
+	 */
+	uv_export(vm.sblk->mso + lc_off);
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PV_PREF, "Intercept 112 for page 0");
+	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off);
+
+	uv_export(vm.sblk->mso + lc_off + PAGE_SIZE);
+	report(vm.sblk->icptcode == ICPT_PV_PREF, "Intercept 112 for page 1");
+
+	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off + PAGE_SIZE);
+
+	/*
+	 * Guest will share the lowcore and we need to check if that
+	 * makes a difference (which it should not).
+	 */
+	report_prefix_push("shared");
+	sie(&vm);
+	/* Guest indicates that it has been setup via the diag 0x44 */
+	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
+	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x44,
+	       "guest shared the first lowcore page");
+
+	/*
+	 * Let's export the standard prefix 0x0 and check for the 112
+	 * intercept.
+	 */
+	uv_export(vm.sblk->mso + lc_off);
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PV_PREF, "Intercept 112");
+	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off);
+	report_prefix_pop();
+
+	report_prefix_pop();
+
+
+	report_prefix_push("0x8000");
+	/*
+	 * Import the new prefix pages so we don't get a PGM 0x3E on
+	 * the guest's spx instruction.
+	 */
+	lc_off = 0x8000;
+	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off);
+	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off + PAGE_SIZE);
+	sie(&vm);
+
+	/* SPX generates a PV instruction notification */
+	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0xb210,
+	       "Received a PV instruction notification intercept for spx");
+	report(*(u32 *)vm.sblk->sidad == 0x8000, "New prefix is 0x8000");
+
+	/* Let's export the prefix at 0x8000 and check for the 112 intercept */
+	uv_export(vm.sblk->mso + lc_off);
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PV_PREF, "Intercept 112 for page 0");
+	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off);
+
+	uv_export(vm.sblk->mso + lc_off + PAGE_SIZE);
+	report(vm.sblk->icptcode == ICPT_PV_PREF, "Intercept 112 for page 1");
+
+	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off + PAGE_SIZE);
+
+	report_prefix_push("shared");
+	sie(&vm);
+	/* Guest indicates that it has shared the new lowcore */
+	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
+	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x44,
+	       "intercept values");
+
+	uv_export(vm.sblk->mso + lc_off);
+	uv_export(vm.sblk->mso + lc_off + PAGE_SIZE);
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PV_PREF, "Intercept 112");
+	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off);
+	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off + PAGE_SIZE);
+
+	/* Try a re-entry */
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
+	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x9c &&
+	       vm.save_area.guest.grs[0] == 42,
+	       "re-entry successful");
+	report_prefix_pop();
+
+	report_prefix_pop();
+
+	report_prefix_pop();
+	uv_destroy_guest(&vm);
+}
+
+int main(void)
+{
+	report_prefix_push("pv-icpts");
+	if (!test_facility(158)) {
+		report_skip("UV Call facility unavailable");
+		goto done;
+	}
+	if (!sclp_facilities.has_sief2) {
+		report_skip("SIEF2 facility unavailable");
+		goto done;
+	}
+
+	snippet_setup_guest(&vm, true);
+	test_icpt_112();
+	test_validity_asce();
+	test_validity_seid();
+	test_validity_handle_not_in_config();
+	test_validity_already_running();
+	test_validity_timing();
+	sie_guest_destroy(&vm);
+
+done:
+	report_prefix_pop();
+	return report_summary();
+}
diff --git a/s390x/snippets/asm/snippet-loop.S b/s390x/snippets/asm/snippet-loop.S
new file mode 100644
index 00000000..ee7cd863
--- /dev/null
+++ b/s390x/snippets/asm/snippet-loop.S
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Infinite loop snippet with no exit
+ *
+ * Copyright (c) 2023 IBM Corp
+ *
+ * Authors:
+ *  Janosch Frank <frankja@linux.ibm.com>
+ */
+.section .text
+retry:
+j 	retry
diff --git a/s390x/snippets/asm/snippet-pv-icpt-112.S b/s390x/snippets/asm/snippet-pv-icpt-112.S
new file mode 100644
index 00000000..aef82dbb
--- /dev/null
+++ b/s390x/snippets/asm/snippet-pv-icpt-112.S
@@ -0,0 +1,77 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Intercept 112 PV snippet
+ *
+ * We setup and share a prefix at 0x0 and 0x8000 which the hypervisor
+ * test will try to export and then execute a SIE entry which
+ * should result in a 112 SIE intercept.
+ *
+ * Copyright (c) 2023 IBM Corp
+ *
+ * Authors:
+ *  Janosch Frank <frankja@linux.ibm.com>
+ */
+#include <asm/asm-offsets.h>
+
+.section .text
+xgr	%r0, %r0
+xgr	%r1, %r1
+
+# Let's tell the hypervisor we're ready to start
+diag	0,0,0x44
+
+/*
+ * Hypervisor will export the lowcore and try a SIE entry which should
+ * result in a 112. It will then import the lowcore again and we
+ * should continue with the code below.
+ */
+
+# Share the lowcore
+larl	%r1, share
+.insn rrf,0xB9A40000,0,1,0,0
+xgr	%r1, %r1
+
+# Let's tell the hypervisor we're ready to start shared testing
+diag	0,0,0x44
+/* Host: icpt:  PV instruction diag 0x44 */
+/* Host: icpt:  112 */
+
+# Copy the invalid PGM new PSW to the new lowcore
+larl	%r1, prfx
+l	%r2, 0(%r1)
+mvc     GEN_LC_PGM_NEW_PSW(16, %r2), GEN_LC_PGM_NEW_PSW(%r0)
+
+# Change the prefix to 0x8000 and re-try
+xgr	%r1, %r1
+xgr	%r2, %r2
+larl	%r2, prfx
+spx	0(%r2)
+/* Host: icpt:  PV instruction notification SPX*/
+/* Host: icpt:  112 */
+
+# Share the new lowcore
+larl	%r3, share_addr
+stg	%r2, 0(%r3)
+larl	%r2, share
+.insn rrf,0xB9A40000,0,2,0,0
+
+# Let's tell the hypervisor we're ready to start shared testing
+diag	0,0,0x44
+/* Host: icpt:  PV instruction diag 0x44 */
+/* Host: icpt:  112 */
+
+# Test re-entry
+lghi	%r1, 42
+diag	1,0,0x9c
+/* Host: icpt:  PV instruction diag 0x9c */
+
+.align 8
+share:
+	.quad 0x0030100000000000
+	.quad 0x0, 0x0, 0x0
+share_addr:
+	.quad 0x0
+	.quad 0x0
+.align 4
+prfx:
+	.long 0x00008000
diff --git a/s390x/snippets/asm/snippet-pv-icpt-loop.S b/s390x/snippets/asm/snippet-pv-icpt-loop.S
new file mode 100644
index 00000000..2aa59c01
--- /dev/null
+++ b/s390x/snippets/asm/snippet-pv-icpt-loop.S
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Infinite loop snippet which can be used to test manipulated SIE
+ * control block intercepts. E.g. when manipulating the PV handles.
+ *
+ * Copyright (c) 2023 IBM Corp
+ *
+ * Authors:
+ *  Janosch Frank <frankja@linux.ibm.com>
+ */
+.section .text
+xgr	%r0, %r0
+retry:
+diag	0,0,0x44
+j 	retry
diff --git a/s390x/snippets/asm/snippet-pv-icpt-vir-timing.S b/s390x/snippets/asm/snippet-pv-icpt-vir-timing.S
new file mode 100644
index 00000000..a0f9fe21
--- /dev/null
+++ b/s390x/snippets/asm/snippet-pv-icpt-vir-timing.S
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Sets a cpu timer which the host can manipulate to check if it will
+ *receive a validity
+ *
+ * Copyright (c) 2023 IBM Corp
+ *
+ * Authors:
+ *  Janosch Frank <frankja@linux.ibm.com>
+ */
+.section .text
+larl	%r1, time_val
+spt	0(%r1)
+diag    0,0,0x44
+xgr	%r1, %r1
+lghi	%r1, 42
+diag	1,0,0x9c
+
+
+.align 8
+time_val:
+	.quad 0x280de80000
-- 
2.34.1

Re: [kvm-unit-tests PATCH 2/3] s390x: pv: Test sie entry intercepts and validities

[Claudio Imbrenda]

On Wed,  1 Feb 2023 08:48:32 +0000
Janosch Frank <frankja@linux.ibm.com> wrote:

> The lowcore is an important part of any s390 cpu so we need to make
> sure it's always available when we virtualize one. For non-PV guests
> that would mean ensuring that the lowcore page is read and writable by
> the guest.
> 
> For PV guests we additionally need to make sure that the page is owned
> by the guest as it is only allowed to access them if that's the
> case. The code 112 SIE intercept tells us if the lowcore pages aren't
> secure anymore.
> 
> Let's check if that intercept is reported by SIE if we export the
> lowcore pages. Additionally check if that's also the case if the guest
> shares the lowcore which will make it readable to the host but
> ownership of the page should not change.
> 
> Also we check for validities in these conditions:
>      * Manipulated cpu timer
>      * Double SIE for same vcpu
>      * Re-use of VCPU handle from another secure configuration
>      * ASCE re-use
> 
> Signed-off-by: Janosch Frank <frankja@linux.ibm.com>

looks good, see some questions below

> ---
>  s390x/Makefile                                |   5 +
>  s390x/pv-icptcode.c                           | 366 ++++++++++++++++++
>  s390x/snippets/asm/snippet-loop.S             |  12 +
>  s390x/snippets/asm/snippet-pv-icpt-112.S      |  77 ++++
>  s390x/snippets/asm/snippet-pv-icpt-loop.S     |  15 +
>  .../snippets/asm/snippet-pv-icpt-vir-timing.S |  22 ++
>  6 files changed, 497 insertions(+)
>  create mode 100644 s390x/pv-icptcode.c
>  create mode 100644 s390x/snippets/asm/snippet-loop.S
>  create mode 100644 s390x/snippets/asm/snippet-pv-icpt-112.S
>  create mode 100644 s390x/snippets/asm/snippet-pv-icpt-loop.S
>  create mode 100644 s390x/snippets/asm/snippet-pv-icpt-vir-timing.S
> 
> diff --git a/s390x/Makefile b/s390x/Makefile
> index 97a61611..858f5af4 100644
> --- a/s390x/Makefile
> +++ b/s390x/Makefile
> @@ -41,6 +41,7 @@ tests += $(TEST_DIR)/migration-sck.elf
>  tests += $(TEST_DIR)/exittime.elf
>  
>  pv-tests += $(TEST_DIR)/pv-diags.elf
> +pv-tests += $(TEST_DIR)/pv-icptcode.elf
>  
>  ifneq ($(HOST_KEY_DOCUMENT),)
>  ifneq ($(GEN_SE_HEADER),)
> @@ -119,6 +120,10 @@ $(TEST_DIR)/spec_ex-sie.elf: snippets = $(SNIPPET_DIR)/c/spec_ex.gbin
>  $(TEST_DIR)/pv-diags.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-diag-yield.gbin
>  $(TEST_DIR)/pv-diags.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-diag-288.gbin
>  $(TEST_DIR)/pv-diags.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-diag-500.gbin
> +$(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-112.gbin
> +$(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-loop.gbin
> +$(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-loop.gbin
> +$(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-vir-timing.gbin
>  
>  ifneq ($(GEN_SE_HEADER),)
>  snippets += $(pv-snippets)
> diff --git a/s390x/pv-icptcode.c b/s390x/pv-icptcode.c
> new file mode 100644
> index 00000000..1a2a4123
> --- /dev/null
> +++ b/s390x/pv-icptcode.c
> @@ -0,0 +1,366 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * PV virtualization interception tests for intercepts that are not
> + * caused by an instruction.
> + *
> + * Copyright (c) 2023 IBM Corp
> + *
> + * Authors:
> + *  Janosch Frank <frankja@linux.ibm.com>
> + */
> +#include <libcflat.h>
> +#include <sie.h>
> +#include <smp.h>
> +#include <sclp.h>
> +#include <snippet.h>
> +#include <asm/facility.h>
> +#include <asm/barrier.h>
> +#include <asm/sigp.h>
> +#include <asm/uv.h>
> +
> +static struct vm vm, vm2;
> +
> +static void test_validity_timing(void)
> +{
> +	extern const char SNIPPET_NAME_START(asm, snippet_pv_icpt_vir_timing)[];
> +	extern const char SNIPPET_NAME_END(asm, snippet_pv_icpt_vir_timing)[];
> +	extern const char SNIPPET_HDR_START(asm, snippet_pv_icpt_vir_timing)[];
> +	extern const char SNIPPET_HDR_END(asm, snippet_pv_icpt_vir_timing)[];
> +	int size_hdr = SNIPPET_HDR_LEN(asm, snippet_pv_icpt_vir_timing);
> +	int size_gbin = SNIPPET_LEN(asm, snippet_pv_icpt_vir_timing);
> +
> +	report_prefix_push("manipulated cpu time");
> +	snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_icpt_vir_timing),
> +			SNIPPET_HDR_START(asm, snippet_pv_icpt_vir_timing),
> +			size_gbin, size_hdr, SNIPPET_UNPACK_OFF);
> +
> +	sie(&vm);
> +	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
> +	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x44,
> +	       "stp done");
> +	vm.sblk->cputm -= 0x280de80000 / 2;

so you are subtracting half of the value?

why not vm.sblk->cputm /= 2?
or just set a fixed (very low) magic value?

what should happen if the cpu timer is higher instead of lower?

> +
> +	sie_expect_validity(&vm);
> +	sie(&vm);
> +	report(uv_validity_check(&vm), "validity");
> +	uv_destroy_guest(&vm);
> +	report_prefix_pop();
> +}
> +
> +static void run_loop(void)
> +{
> +	sie(&vm);
> +	sigp_retry(stap(), SIGP_STOP, 0, NULL);
> +}
> +
> +static void test_validity_already_running(void)
> +{
> +	extern const char SNIPPET_NAME_START(asm, snippet_loop)[];
> +	extern const char SNIPPET_NAME_END(asm, snippet_loop)[];
> +	extern const char SNIPPET_HDR_START(asm, snippet_loop)[];
> +	extern const char SNIPPET_HDR_END(asm, snippet_loop)[];
> +	int size_hdr = SNIPPET_HDR_LEN(asm, snippet_loop);
> +	int size_gbin = SNIPPET_LEN(asm, snippet_loop);
> +	struct psw psw = {
> +		.mask = PSW_MASK_64,
> +		.addr = (uint64_t)run_loop,
> +	};
> +
> +	report_prefix_push("already running");
> +	if (smp_query_num_cpus() < 3) {
> +		report_skip("need at least 3 cpus for this test");
> +		goto out;
> +	}
> +
> +	snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_loop),
> +			SNIPPET_HDR_START(asm, snippet_loop),
> +			size_gbin, size_hdr, SNIPPET_UNPACK_OFF);
> +
> +	sie_expect_validity(&vm);
> +	smp_cpu_setup(1, psw);
> +	smp_cpu_setup(2, psw);
> +	while (vm.sblk->icptcode != ICPT_VALIDITY) { mb(); }

maybe put the mb(); in a separate line

> +	/* Yes I know this is not reliable as one cpu might overwrite it */

the wording in this comment could be improved

> +	report(uv_validity_check(&vm), "validity");
> +	smp_cpu_stop(1);
> +	smp_cpu_stop(2);
> +	uv_destroy_guest(&vm);
> +
> +out:
> +	report_prefix_pop();
> +}
> +
> +/* Tests if a vcpu handle from another configuration results in a validity intercept. */
> +static void test_validity_handle_not_in_config(void)
> +{
> +	extern const char SNIPPET_NAME_START(asm, snippet_pv_icpt_loop)[];
> +	extern const char SNIPPET_NAME_END(asm, snippet_pv_icpt_loop)[];
> +	extern const char SNIPPET_HDR_START(asm, snippet_pv_icpt_loop)[];
> +	extern const char SNIPPET_HDR_END(asm, snippet_pv_icpt_loop)[];
> +	int size_hdr = SNIPPET_HDR_LEN(asm, snippet_pv_icpt_loop);
> +	int size_gbin = SNIPPET_LEN(asm, snippet_pv_icpt_loop);
> +
> +	report_prefix_push("handle not in config");
> +	/* Setup our primary vm */
> +	snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_icpt_loop),
> +			SNIPPET_HDR_START(asm, snippet_pv_icpt_loop),
> +			size_gbin, size_hdr, SNIPPET_UNPACK_OFF);
> +
> +	/* Setup secondary vm */
> +	snippet_setup_guest(&vm2, true);
> +	snippet_pv_init(&vm2, SNIPPET_NAME_START(asm, snippet_pv_icpt_loop),
> +			SNIPPET_HDR_START(asm, snippet_pv_icpt_loop),
> +			size_gbin, size_hdr, SNIPPET_UNPACK_OFF);
> +
> +	vm.sblk->pv_handle_cpu = vm2.sblk->pv_handle_cpu;
> +	sie_expect_validity(&vm);
> +	sie(&vm);
> +	report(uv_validity_check(&vm), "validity");
> +
> +	/* Restore cpu handle and destroy the second vm */
> +	vm.sblk->pv_handle_cpu = vm.uv.vcpu_handle;
> +	uv_destroy_guest(&vm2);
> +	sie_guest_destroy(&vm2);
> +
> +	uv_destroy_guest(&vm);
> +	report_prefix_pop();
> +}
> +
> +/* Tests if a wrong vm or vcpu handle results in a validity intercept. */
> +static void test_validity_seid(void)
> +{
> +	extern const char SNIPPET_NAME_START(asm, snippet_pv_icpt_loop)[];
> +	extern const char SNIPPET_NAME_END(asm, snippet_pv_icpt_loop)[];
> +	extern const char SNIPPET_HDR_START(asm, snippet_pv_icpt_loop)[];
> +	extern const char SNIPPET_HDR_END(asm, snippet_pv_icpt_loop)[];
> +	int size_hdr = SNIPPET_HDR_LEN(asm, snippet_pv_icpt_loop);
> +	int size_gbin = SNIPPET_LEN(asm, snippet_pv_icpt_loop);
> +	int fails = 0;
> +	int i;
> +
> +	report_prefix_push("handles");
> +	snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_icpt_loop),
> +			SNIPPET_HDR_START(asm, snippet_pv_icpt_loop),
> +			size_gbin, size_hdr, SNIPPET_UNPACK_OFF);
> +
> +	for (i = 0; i < 64; i++) {
> +		vm.sblk->pv_handle_config ^= 1UL << i;
> +		sie_expect_validity(&vm);
> +		sie(&vm);
> +		if (!uv_validity_check(&vm)) {
> +			report_fail("SIE accepted wrong VM SEID, changed bit %d",
> +				    63 - i);
> +			fails++;
> +		}
> +		vm.sblk->pv_handle_config ^= 1UL << i;
> +	}
> +	report(!fails, "No wrong vm handle accepted");
> +
> +	fails = 0;
> +	for (i = 0; i < 64; i++) {
> +		vm.sblk->pv_handle_cpu ^= 1UL << i;
> +		sie_expect_validity(&vm);
> +		sie(&vm);
> +		if (!uv_validity_check(&vm)) {
> +			report_fail("SIE accepted wrong CPU SEID, changed bit %d",
> +				    63 - i);
> +			fails++;
> +		}
> +		vm.sblk->pv_handle_cpu ^= 1UL << i;
> +	}
> +	report(!fails, "No wrong cpu handle accepted");
> +
> +	uv_destroy_guest(&vm);
> +	report_prefix_pop();
> +}
> +
> +/*
> + * Tests if we get a validity intercept if the CR1 asce at SIE entry
> + * is not the same as the one given at the UV creation of the VM.
> + */
> +static void test_validity_asce(void)
> +{
> +	extern const char SNIPPET_NAME_START(asm, snippet_pv_icpt_112)[];
> +	extern const char SNIPPET_NAME_END(asm, snippet_pv_icpt_112)[];
> +	extern const char SNIPPET_HDR_START(asm, snippet_pv_icpt_112)[];
> +	extern const char SNIPPET_HDR_END(asm, snippet_pv_icpt_112)[];
> +	int size_hdr = SNIPPET_HDR_LEN(asm, snippet_pv_icpt_112);
> +	int size_gbin = SNIPPET_LEN(asm, snippet_pv_icpt_112);
> +	uint64_t asce_old, asce_new;
> +	void *pgd_new, *pgd_old;
> +
> +	report_prefix_push("asce");
> +	snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_icpt_112),
> +			SNIPPET_HDR_START(asm, snippet_pv_icpt_112),
> +			size_gbin, size_hdr, SNIPPET_UNPACK_OFF);
> +
> +	asce_old = vm.save_area.guest.asce;
> +	pgd_new = memalign_pages_flags(PAGE_SIZE, PAGE_SIZE * 4, 0);
> +	pgd_old = (void *)(asce_old & PAGE_MASK);
> +
> +	/* Copy the contents of the top most table */
> +	memcpy(pgd_new, pgd_old, PAGE_SIZE * 4);
> +
> +	/* Create the replacement ASCE */
> +	asce_new = __pa(pgd_new) | ASCE_DT_REGION1 | REGION_TABLE_LENGTH | ASCE_P;
> +	vm.save_area.guest.asce = asce_new;
> +
> +	sie_expect_validity(&vm);
> +	sie(&vm);
> +	report(uv_validity_check(&vm), "wrong CR1 validity");
> +
> +	/* Restore the old ASCE */
> +	vm.save_area.guest.asce = asce_old;
> +
> +	/* Try if we can still do an entry with the correct asce */
> +	sie(&vm);
> +	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
> +	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x44,
> +	       "re-entry with valid CR1");
> +	uv_destroy_guest(&vm);
> +	free_pages(pgd_new);
> +	report_prefix_pop();
> +}
> +
> +static void test_icpt_112(void)
> +{
> +	extern const char SNIPPET_NAME_START(asm, snippet_pv_icpt_112)[];
> +	extern const char SNIPPET_NAME_END(asm, snippet_pv_icpt_112)[];
> +	extern const char SNIPPET_HDR_START(asm, snippet_pv_icpt_112)[];
> +	extern const char SNIPPET_HDR_END(asm, snippet_pv_icpt_112)[];
> +	int size_hdr = SNIPPET_HDR_LEN(asm, snippet_pv_icpt_112);
> +	int size_gbin = SNIPPET_LEN(asm, snippet_pv_icpt_112);
> +
> +	u64 lc_off = 0;
> +
> +	report_prefix_push("prefix");
> +
> +	snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_icpt_112),
> +			SNIPPET_HDR_START(asm, snippet_pv_icpt_112),
> +			size_gbin, size_hdr, SNIPPET_UNPACK_OFF);
> +
> +	report_prefix_push("0x0");
> +	sie(&vm);
> +
> +	/* Guest indicates that it has been setup via the diag 0x44 */
> +	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
> +	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x44,
> +	       "guest set up");
> +
> +	/*
> +	 * Let's export the standard prefix 0x0 and check for the 112
> +	 * intercept.
> +	 */
> +	uv_export(vm.sblk->mso + lc_off);
> +	sie(&vm);
> +	report(vm.sblk->icptcode == ICPT_PV_PREF, "Intercept 112 for page 0");
> +	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off);
> +
> +	uv_export(vm.sblk->mso + lc_off + PAGE_SIZE);
> +	report(vm.sblk->icptcode == ICPT_PV_PREF, "Intercept 112 for page 1");
> +
> +	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off + PAGE_SIZE);
> +
> +	/*
> +	 * Guest will share the lowcore and we need to check if that
> +	 * makes a difference (which it should not).
> +	 */
> +	report_prefix_push("shared");
> +	sie(&vm);
> +	/* Guest indicates that it has been setup via the diag 0x44 */
> +	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
> +	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x44,
> +	       "guest shared the first lowcore page");
> +
> +	/*
> +	 * Let's export the standard prefix 0x0 and check for the 112
> +	 * intercept.
> +	 */
> +	uv_export(vm.sblk->mso + lc_off);
> +	sie(&vm);
> +	report(vm.sblk->icptcode == ICPT_PV_PREF, "Intercept 112");
> +	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off);
> +	report_prefix_pop();
> +
> +	report_prefix_pop();
> +
> +
> +	report_prefix_push("0x8000");
> +	/*
> +	 * Import the new prefix pages so we don't get a PGM 0x3E on
> +	 * the guest's spx instruction.
> +	 */
> +	lc_off = 0x8000;
> +	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off);
> +	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off + PAGE_SIZE);
> +	sie(&vm);
> +
> +	/* SPX generates a PV instruction notification */
> +	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0xb210,
> +	       "Received a PV instruction notification intercept for spx");
> +	report(*(u32 *)vm.sblk->sidad == 0x8000, "New prefix is 0x8000");
> +
> +	/* Let's export the prefix at 0x8000 and check for the 112 intercept */
> +	uv_export(vm.sblk->mso + lc_off);
> +	sie(&vm);
> +	report(vm.sblk->icptcode == ICPT_PV_PREF, "Intercept 112 for page 0");
> +	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off);
> +
> +	uv_export(vm.sblk->mso + lc_off + PAGE_SIZE);
> +	report(vm.sblk->icptcode == ICPT_PV_PREF, "Intercept 112 for page 1");
> +
> +	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off + PAGE_SIZE);
> +
> +	report_prefix_push("shared");
> +	sie(&vm);
> +	/* Guest indicates that it has shared the new lowcore */
> +	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
> +	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x44,
> +	       "intercept values");
> +
> +	uv_export(vm.sblk->mso + lc_off);
> +	uv_export(vm.sblk->mso + lc_off + PAGE_SIZE);

why are you not testing both pages individually here, like you did
above?

> +	sie(&vm);
> +	report(vm.sblk->icptcode == ICPT_PV_PREF, "Intercept 112");
> +	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off);
> +	uv_import(vm.uv.vm_handle, vm.sblk->mso + lc_off + PAGE_SIZE);
> +
> +	/* Try a re-entry */
> +	sie(&vm);
> +	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
> +	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x9c &&
> +	       vm.save_area.guest.grs[0] == 42,
> +	       "re-entry successful");
> +	report_prefix_pop();
> +
> +	report_prefix_pop();
> +
> +	report_prefix_pop();
> +	uv_destroy_guest(&vm);
> +}
> +
> +int main(void)
> +{
> +	report_prefix_push("pv-icpts");
> +	if (!test_facility(158)) {
> +		report_skip("UV Call facility unavailable");
> +		goto done;
> +	}
> +	if (!sclp_facilities.has_sief2) {
> +		report_skip("SIEF2 facility unavailable");
> +		goto done;
> +	}
> +
> +	snippet_setup_guest(&vm, true);
> +	test_icpt_112();
> +	test_validity_asce();
> +	test_validity_seid();
> +	test_validity_handle_not_in_config();
> +	test_validity_already_running();
> +	test_validity_timing();
> +	sie_guest_destroy(&vm);
> +
> +done:
> +	report_prefix_pop();
> +	return report_summary();
> +}
> diff --git a/s390x/snippets/asm/snippet-loop.S b/s390x/snippets/asm/snippet-loop.S
> new file mode 100644
> index 00000000..ee7cd863
> --- /dev/null
> +++ b/s390x/snippets/asm/snippet-loop.S
> @@ -0,0 +1,12 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Infinite loop snippet with no exit
> + *
> + * Copyright (c) 2023 IBM Corp
> + *
> + * Authors:
> + *  Janosch Frank <frankja@linux.ibm.com>
> + */
> +.section .text
> +retry:
> +j 	retry
> diff --git a/s390x/snippets/asm/snippet-pv-icpt-112.S b/s390x/snippets/asm/snippet-pv-icpt-112.S
> new file mode 100644
> index 00000000..aef82dbb
> --- /dev/null
> +++ b/s390x/snippets/asm/snippet-pv-icpt-112.S
> @@ -0,0 +1,77 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Intercept 112 PV snippet
> + *
> + * We setup and share a prefix at 0x0 and 0x8000 which the hypervisor
> + * test will try to export and then execute a SIE entry which
> + * should result in a 112 SIE intercept.
> + *
> + * Copyright (c) 2023 IBM Corp
> + *
> + * Authors:
> + *  Janosch Frank <frankja@linux.ibm.com>
> + */
> +#include <asm/asm-offsets.h>
> +
> +.section .text
> +xgr	%r0, %r0
> +xgr	%r1, %r1
> +
> +# Let's tell the hypervisor we're ready to start
> +diag	0,0,0x44
> +
> +/*
> + * Hypervisor will export the lowcore and try a SIE entry which should
> + * result in a 112. It will then import the lowcore again and we
> + * should continue with the code below.
> + */
> +
> +# Share the lowcore
> +larl	%r1, share
> +.insn rrf,0xB9A40000,0,1,0,0
> +xgr	%r1, %r1
> +
> +# Let's tell the hypervisor we're ready to start shared testing
> +diag	0,0,0x44
> +/* Host: icpt:  PV instruction diag 0x44 */
> +/* Host: icpt:  112 */
> +
> +# Copy the invalid PGM new PSW to the new lowcore
> +larl	%r1, prfx
> +l	%r2, 0(%r1)
> +mvc     GEN_LC_PGM_NEW_PSW(16, %r2), GEN_LC_PGM_NEW_PSW(%r0)
> +
> +# Change the prefix to 0x8000 and re-try
> +xgr	%r1, %r1
> +xgr	%r2, %r2
> +larl	%r2, prfx
> +spx	0(%r2)
> +/* Host: icpt:  PV instruction notification SPX*/
> +/* Host: icpt:  112 */
> +
> +# Share the new lowcore
> +larl	%r3, share_addr
> +stg	%r2, 0(%r3)
> +larl	%r2, share
> +.insn rrf,0xB9A40000,0,2,0,0
> +
> +# Let's tell the hypervisor we're ready to start shared testing
> +diag	0,0,0x44
> +/* Host: icpt:  PV instruction diag 0x44 */
> +/* Host: icpt:  112 */
> +
> +# Test re-entry
> +lghi	%r1, 42
> +diag	1,0,0x9c
> +/* Host: icpt:  PV instruction diag 0x9c */
> +
> +.align 8
> +share:
> +	.quad 0x0030100000000000
> +	.quad 0x0, 0x0, 0x0
> +share_addr:
> +	.quad 0x0
> +	.quad 0x0
> +.align 4
> +prfx:
> +	.long 0x00008000
> diff --git a/s390x/snippets/asm/snippet-pv-icpt-loop.S b/s390x/snippets/asm/snippet-pv-icpt-loop.S
> new file mode 100644
> index 00000000..2aa59c01
> --- /dev/null
> +++ b/s390x/snippets/asm/snippet-pv-icpt-loop.S
> @@ -0,0 +1,15 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Infinite loop snippet which can be used to test manipulated SIE
> + * control block intercepts. E.g. when manipulating the PV handles.
> + *
> + * Copyright (c) 2023 IBM Corp
> + *
> + * Authors:
> + *  Janosch Frank <frankja@linux.ibm.com>
> + */
> +.section .text
> +xgr	%r0, %r0
> +retry:
> +diag	0,0,0x44
> +j 	retry
> diff --git a/s390x/snippets/asm/snippet-pv-icpt-vir-timing.S b/s390x/snippets/asm/snippet-pv-icpt-vir-timing.S
> new file mode 100644
> index 00000000..a0f9fe21
> --- /dev/null
> +++ b/s390x/snippets/asm/snippet-pv-icpt-vir-timing.S
> @@ -0,0 +1,22 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Sets a cpu timer which the host can manipulate to check if it will
> + *receive a validity
> + *
> + * Copyright (c) 2023 IBM Corp
> + *
> + * Authors:
> + *  Janosch Frank <frankja@linux.ibm.com>
> + */
> +.section .text
> +larl	%r1, time_val
> +spt	0(%r1)
> +diag    0,0,0x44
> +xgr	%r1, %r1
> +lghi	%r1, 42
> +diag	1,0,0x9c
> +
> +
> +.align 8
> +time_val:
> +	.quad 0x280de80000

Re: [kvm-unit-tests PATCH 2/3] s390x: pv: Test sie entry intercepts and validities

[Janosch Frank]

On 2/15/23 18:06, Claudio Imbrenda wrote:
> On Wed,  1 Feb 2023 08:48:32 +0000
> Janosch Frank <frankja@linux.ibm.com> wrote:
> 
>> The lowcore is an important part of any s390 cpu so we need to make
>> sure it's always available when we virtualize one. For non-PV guests
>> that would mean ensuring that the lowcore page is read and writable by
>> the guest.
>>
>> For PV guests we additionally need to make sure that the page is owned
>> by the guest as it is only allowed to access them if that's the
>> case. The code 112 SIE intercept tells us if the lowcore pages aren't
>> secure anymore.
>>
>> Let's check if that intercept is reported by SIE if we export the
>> lowcore pages. Additionally check if that's also the case if the guest
>> shares the lowcore which will make it readable to the host but
>> ownership of the page should not change.
>>
>> Also we check for validities in these conditions:
>>       * Manipulated cpu timer
>>       * Double SIE for same vcpu
>>       * Re-use of VCPU handle from another secure configuration
>>       * ASCE re-use
>>
>> Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
> 
> looks good, see some questions below
> 
>> ---
[...]
>> +	extern const char SNIPPET_NAME_START(asm, snippet_pv_icpt_vir_timing)[];
>> +	extern const char SNIPPET_NAME_END(asm, snippet_pv_icpt_vir_timing)[];
>> +	extern const char SNIPPET_HDR_START(asm, snippet_pv_icpt_vir_timing)[];
>> +	extern const char SNIPPET_HDR_END(asm, snippet_pv_icpt_vir_timing)[];
>> +	int size_hdr = SNIPPET_HDR_LEN(asm, snippet_pv_icpt_vir_timing);
>> +	int size_gbin = SNIPPET_LEN(asm, snippet_pv_icpt_vir_timing);
>> +
>> +	report_prefix_push("manipulated cpu time");
>> +	snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_icpt_vir_timing),
>> +			SNIPPET_HDR_START(asm, snippet_pv_icpt_vir_timing),
>> +			size_gbin, size_hdr, SNIPPET_UNPACK_OFF);
>> +
>> +	sie(&vm);
>> +	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
>> +	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x44,
>> +	       "stp done");
>> +	vm.sblk->cputm -= 0x280de80000 / 2;
> 
> so you are subtracting half of the value?
> 
> why not vm.sblk->cputm /= 2?
> or just set a fixed (very low) magic value?
> 
> what should happen if the cpu timer is higher instead of lower?

I'll need to do some digging to find out why I used this specific 
procedure. It's been a very long time since I wrote those tests.

[...]

>> +	snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_loop),
>> +			SNIPPET_HDR_START(asm, snippet_loop),
>> +			size_gbin, size_hdr, SNIPPET_UNPACK_OFF);
>> +
>> +	sie_expect_validity(&vm);
>> +	smp_cpu_setup(1, psw);
>> +	smp_cpu_setup(2, psw);
>> +	while (vm.sblk->icptcode != ICPT_VALIDITY) { mb(); }
> 
> maybe put the mb(); in a separate line

Can do

> 
>> +	/* Yes I know this is not reliable as one cpu might overwrite it */
> 
> the wording in this comment could be improved

How about:
This might not be fully reliable but it should be sufficient for our 
current goals.

[...]

>> +	report_prefix_push("shared");
>> +	sie(&vm);
>> +	/* Guest indicates that it has shared the new lowcore */
>> +	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
>> +	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x44,
>> +	       "intercept values");
>> +
>> +	uv_export(vm.sblk->mso + lc_off);
>> +	uv_export(vm.sblk->mso + lc_off + PAGE_SIZE);
> 
> why are you not testing both pages individually here, like you did
> above?

Hmm, I don't think there was a reason behind this. I'll add it.

[...]

Re: [kvm-unit-tests PATCH 2/3] s390x: pv: Test sie entry intercepts and validities

[Claudio Imbrenda]

On Tue, 21 Feb 2023 10:22:13 +0100
Janosch Frank <frankja@linux.ibm.com> wrote:

[...]

> >> +	/* Yes I know this is not reliable as one cpu might overwrite it */  
> > 
> > the wording in this comment could be improved  
> 
> How about:
> This might not be fully reliable but it should be sufficient for our 
> current goals.

looks good, thanks

> 
> [...]
> 
> >> +	report_prefix_push("shared");
> >> +	sie(&vm);
> >> +	/* Guest indicates that it has shared the new lowcore */
> >> +	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
> >> +	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x44,
> >> +	       "intercept values");
> >> +
> >> +	uv_export(vm.sblk->mso + lc_off);
> >> +	uv_export(vm.sblk->mso + lc_off + PAGE_SIZE);  
> > 
> > why are you not testing both pages individually here, like you did
> > above?  
> 
> Hmm, I don't think there was a reason behind this. I'll add it.
> 
> [...]

[kvm-unit-tests PATCH 3/3] s390x: pv: Add IPL reset tests

[Janosch Frank]

The diag308 requires extensive cooperation between the hypervisor and
the Ultravisor so the Ultravisor can make sure all necessary reset
steps have been done.

Let's check if we get the correct validity errors.

Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
---
 s390x/Makefile                           |   2 +
 s390x/pv-ipl.c                           | 237 +++++++++++++++++++++++
 s390x/snippets/asm/snippet-pv-diag-308.S |  67 +++++++
 3 files changed, 306 insertions(+)
 create mode 100644 s390x/pv-ipl.c
 create mode 100644 s390x/snippets/asm/snippet-pv-diag-308.S

diff --git a/s390x/Makefile b/s390x/Makefile
index 858f5af4..e8559a4e 100644
--- a/s390x/Makefile
+++ b/s390x/Makefile
@@ -42,6 +42,7 @@ tests += $(TEST_DIR)/exittime.elf
 
 pv-tests += $(TEST_DIR)/pv-diags.elf
 pv-tests += $(TEST_DIR)/pv-icptcode.elf
+pv-tests += $(TEST_DIR)/pv-ipl.elf
 
 ifneq ($(HOST_KEY_DOCUMENT),)
 ifneq ($(GEN_SE_HEADER),)
@@ -124,6 +125,7 @@ $(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-1
 $(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-loop.gbin
 $(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-loop.gbin
 $(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-vir-timing.gbin
+$(TEST_DIR)/pv-ipl.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-diag-308.gbin
 
 ifneq ($(GEN_SE_HEADER),)
 snippets += $(pv-snippets)
diff --git a/s390x/pv-ipl.c b/s390x/pv-ipl.c
new file mode 100644
index 00000000..61afde5c
--- /dev/null
+++ b/s390x/pv-ipl.c
@@ -0,0 +1,237 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * PV diagnose 308 (IPL) tests
+ *
+ * Copyright (c) 2023 IBM Corp
+ *
+ * Authors:
+ *  Janosch Frank <frankja@linux.ibm.com>
+ */
+#include <libcflat.h>
+#include <sie.h>
+#include <sclp.h>
+#include <snippet.h>
+#include <asm/facility.h>
+#include <asm/uv.h>
+
+static struct vm vm;
+
+static void setup_gbin(void)
+{
+	extern const char SNIPPET_NAME_START(asm, snippet_pv_diag_308)[];
+	extern const char SNIPPET_NAME_END(asm, snippet_pv_diag_308)[];
+	extern const char SNIPPET_HDR_START(asm, snippet_pv_diag_308)[];
+	extern const char SNIPPET_HDR_END(asm, snippet_pv_diag_308)[];
+	int size_hdr = SNIPPET_HDR_LEN(asm, snippet_pv_diag_308);
+	int size_gbin = SNIPPET_LEN(asm, snippet_pv_diag_308);
+
+	snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_diag_308),
+			SNIPPET_HDR_START(asm, snippet_pv_diag_308),
+			size_gbin, size_hdr, SNIPPET_UNPACK_OFF);
+}
+
+static void test_diag_308_1(void)
+{
+	uint16_t rc, rrc;
+	int cc;
+
+	report_prefix_push("subcode 1");
+	setup_gbin();
+
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PV_INSTR && vm.sblk->ipa == 0x8302 &&
+	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x500,
+	       "intercept values diag 500");
+	/* The snippet asked us for the subcode and we answer with 1 in gr2 */
+	vm.save_area.guest.grs[2] = 1;
+
+	/* Continue after diag 0x500, next icpt should be the 0x308 */
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
+	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x308,
+	       "intercept values diag 0x308");
+	report(vm.save_area.guest.grs[2] == 1,
+	       "subcode 1");
+
+	/*
+	 * We need to perform several UV calls to emulate the subcode
+	 * 1. Failing to do that should result in a validity
+	 *
+	 * - Mark all cpus as stopped
+	 * - Reset the cpus, calling one gets an initial reset
+	 * - Load the reset PSW
+	 * - Unshare all
+	 * - Load the reset PSW
+	 */
+	sie_expect_validity(&vm);
+	sie(&vm);
+	report((sie_get_validity(&vm) & 0xff00) == 0x2000, "validity no UVCs");
+
+	/* Mark the CPU as stopped so we can unshare and reset */
+	cc = uv_set_cpu_state(vm.sblk->pv_handle_cpu, PV_CPU_STATE_STP);
+	report(!cc, "Set cpu stopped");
+
+	sie_expect_validity(&vm);
+	sie(&vm);
+	report((sie_get_validity(&vm) & 0xff00) == 0x2000, "validity stopped");
+
+	/* Unshare all memory */
+	cc = uv_cmd_nodata(vm.sblk->pv_handle_config,
+			   UVC_CMD_SET_UNSHARED_ALL, &rc, &rrc);
+	report(cc == 0 && rc == 1, "Unshare all");
+
+	sie_expect_validity(&vm);
+	sie(&vm);
+	report((sie_get_validity(&vm) & 0xff00) == 0x2000,
+	       "validity stopped, unshared");
+
+	/* Prepare the CPU reset */
+	cc = uv_cmd_nodata(vm.sblk->pv_handle_config,
+			   UVC_CMD_PREPARE_RESET, &rc, &rrc);
+	report(cc == 0 && rc == 1, "Prepare reset call");
+
+	sie_expect_validity(&vm);
+	sie(&vm);
+	report((sie_get_validity(&vm) & 0xff00) == 0x2000,
+	       "validity stopped, unshared, prepare");
+
+	/* Do the reset */
+	cc = uv_cmd_nodata(vm.sblk->pv_handle_cpu,
+			   UVC_CMD_CPU_RESET_INITIAL, &rc, &rrc);
+	report(cc == 0 && rc == 1, "Clear reset cpu");
+
+	sie_expect_validity(&vm);
+	sie(&vm);
+	report((sie_get_validity(&vm) & 0xff00) == 0x2000,
+	       "validity stopped, unshared, prepare, reset");
+
+	/* Load the PSW from 0x0 */
+	cc = uv_set_cpu_state(vm.sblk->pv_handle_cpu, PV_CPU_STATE_OPR_LOAD);
+	report(!cc, "Set cpu load");
+
+	/*
+	 * Check if we executed the iaddr of the reset PSW, we should
+	 * see a diagnose 0x9c PV instruction notification.
+	 */
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
+	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x9c &&
+	       vm.save_area.guest.grs[0] == 42,
+	       "intercept values after diag 0x308");
+
+
+	uv_destroy_guest(&vm);
+	report_prefix_pop();
+}
+
+static void test_diag_308_0(void)
+{
+	uint16_t rc, rrc;
+	int cc;
+
+	report_prefix_push("subcode 0");
+	setup_gbin();
+
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PV_INSTR && vm.sblk->ipa == 0x8302 &&
+	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x500,
+	       "intercept values diag 500");
+	/* The snippet asked us for the subcode and we answer with 0 in gr2 */
+	vm.save_area.guest.grs[2] = 0;
+
+	/* Continue after diag 0x500, next icpt should be the 0x308 */
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
+	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x308,
+	       "intercept values");
+	report(vm.save_area.guest.grs[2] == 0,
+	       "subcode 0");
+
+	/*
+	 * We need to perform several UV calls to emulate the subcode
+	 * 0. Failing to do that should result in a validity
+	 *
+	 * - Mark all cpus as stopped
+	 * - Unshare all memory
+	 * - Prepare the reset
+	 * - Reset the cpus
+	 * - Load the reset PSW
+	 */
+	sie_expect_validity(&vm);
+	sie(&vm);
+	report((sie_get_validity(&vm) & 0xff00) == 0x2000, "validity, no action");
+
+	/* Mark the CPU as stopped so we can unshare and reset */
+	cc = uv_set_cpu_state(vm.sblk->pv_handle_cpu, PV_CPU_STATE_STP);
+	report(!cc, "Set cpu stopped");
+
+	sie_expect_validity(&vm);
+	sie(&vm);
+	report((sie_get_validity(&vm) & 0xff00) == 0x2000, "validity, stopped");
+
+	/* Unshare all memory */
+	cc = uv_cmd_nodata(vm.sblk->pv_handle_config,
+			   UVC_CMD_SET_UNSHARED_ALL, &rc, &rrc);
+	report(cc == 0 && rc == 1, "Unshare all");
+
+	sie_expect_validity(&vm);
+	sie(&vm);
+	report((sie_get_validity(&vm) & 0xff00) == 0x2000, "validity stopped, unshared");
+
+	/* Prepare the CPU reset */
+	cc = uv_cmd_nodata(vm.sblk->pv_handle_config,
+			   UVC_CMD_PREPARE_RESET, &rc, &rrc);
+	report(cc == 0 && rc == 1, "Prepare reset call");
+
+	sie_expect_validity(&vm);
+	sie(&vm);
+	report((sie_get_validity(&vm) & 0xff00) == 0x2000, "validity stopped, unshared, prep reset");
+
+	/* Do the reset */
+	cc = uv_cmd_nodata(vm.sblk->pv_handle_cpu,
+			   UVC_CMD_CPU_RESET_CLEAR, &rc, &rrc);
+	report(cc == 0 && rc == 1, "Clear reset cpu");
+
+	sie_expect_validity(&vm);
+	sie(&vm);
+	report((sie_get_validity(&vm) & 0xff00) == 0x2000, "validity stopped, unshared, prep reset, cpu reset");
+
+	/* Load the PSW from 0x0 */
+	cc = uv_set_cpu_state(vm.sblk->pv_handle_cpu, PV_CPU_STATE_OPR_LOAD);
+	report(!cc, "Set cpu load");
+
+	/*
+	 * Check if we executed the iaddr of the reset PSW, we should
+	 * see a diagnose 0x9c PV instruction notification.
+	 */
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
+	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x9c &&
+	       vm.save_area.guest.grs[0] == 42,
+	       "intercept values");
+
+	uv_destroy_guest(&vm);
+	report_prefix_pop();
+}
+
+int main(void)
+{
+	report_prefix_push("uv-sie");
+	if (!test_facility(158)) {
+		report_skip("UV Call facility unavailable");
+		goto done;
+	}
+	if (!sclp_facilities.has_sief2) {
+		report_skip("SIEF2 facility unavailable");
+		goto done;
+	}
+
+	snippet_setup_guest(&vm, true);
+	test_diag_308_0();
+	test_diag_308_1();
+	sie_guest_destroy(&vm);
+
+done:
+	report_prefix_pop();
+	return report_summary();
+}
diff --git a/s390x/snippets/asm/snippet-pv-diag-308.S b/s390x/snippets/asm/snippet-pv-diag-308.S
new file mode 100644
index 00000000..58c96173
--- /dev/null
+++ b/s390x/snippets/asm/snippet-pv-diag-308.S
@@ -0,0 +1,67 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Diagnose 0x308 snippet used for PV IPL and reset testing
+ *
+ * Copyright (c) 2023 IBM Corp
+ *
+ * Authors:
+ *  Janosch Frank <frankja@linux.ibm.com>
+ */
+#include <asm/asm-offsets.h>
+.section .text
+
+/* Sets a reset PSW with the given PSW address */
+.macro SET_RESET_PSW_ADDR label
+lgrl	%r5, reset_psw
+larl	%r6, \label
+ogr	%r5, %r6
+stg	%r5, 0
+.endm
+
+/* Does a diagnose 308 with the given subcode */
+.macro DIAG308 subcode
+xgr	%r3, %r3
+lghi	%r3, \subcode
+diag	1, 3, 0x308
+.endm
+
+sam64
+
+/* Execute the diag500 which will set the subcode we execute in gr2 */
+diag	0, 0, 0x500
+
+/*
+ * A valid PGM new PSW can be a real problem since we never fall out
+ * of SIE and therefore effectively loop forever. 0 is a valid PSW
+ * therefore we re-use the reset_psw as this has the short PSW
+ * bit set which is invalid for a long PSW like the exception new
+ * PSWs.
+ *
+ * For subcode 0/1 there are no PGMs to consider.
+ */
+lgrl   %r5, reset_psw
+stg    %r5, GEN_LC_PGM_NEW_PSW
+
+/* Clean registers that are used */
+xgr	%r0, %r0
+xgr	%r1, %r1
+xgr	%r3, %r3
+xgr	%r4, %r4
+xgr	%r5, %r5
+xgr	%r6, %r6
+
+/* Subcode 0 - Modified Clear */
+SET_RESET_PSW_ADDR done
+diag	%r0, %r2, 0x308
+
+/* Should never be executed because of the reset PSW */
+diag	0, 0, 0x44
+
+done:
+lghi	%r1, 42
+diag	%r1, 0, 0x9c
+
+
+	.align	8
+reset_psw:
+	.quad	0x0008000180000000
-- 
2.34.1

Re: [kvm-unit-tests PATCH 3/3] s390x: pv: Add IPL reset tests

[Claudio Imbrenda]

On Wed,  1 Feb 2023 08:48:33 +0000
Janosch Frank <frankja@linux.ibm.com> wrote:

> The diag308 requires extensive cooperation between the hypervisor and
> the Ultravisor so the Ultravisor can make sure all necessary reset
> steps have been done.
> 
> Let's check if we get the correct validity errors.
> 
> Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
> ---

[...]

> +
> +	/*
> +	 * We need to perform several UV calls to emulate the subcode
> +	 * 1. Failing to do that should result in a validity
> +	 *
> +	 * - Mark all cpus as stopped
> +	 * - Reset the cpus, calling one gets an initial reset
> +	 * - Load the reset PSW
> +	 * - Unshare all
> +	 * - Load the reset PSW

you forgot to mention prepare the reset, and the list does not reflect
the order things are done in the code

> +	 */
> +	sie_expect_validity(&vm);
> +	sie(&vm);
> +	report((sie_get_validity(&vm) & 0xff00) == 0x2000, "validity no UVCs");
> +
> +	/* Mark the CPU as stopped so we can unshare and reset */
> +	cc = uv_set_cpu_state(vm.sblk->pv_handle_cpu, PV_CPU_STATE_STP);
> +	report(!cc, "Set cpu stopped");
> +
> +	sie_expect_validity(&vm);
> +	sie(&vm);
> +	report((sie_get_validity(&vm) & 0xff00) == 0x2000, "validity stopped");
> +
> +	/* Unshare all memory */
> +	cc = uv_cmd_nodata(vm.sblk->pv_handle_config,
> +			   UVC_CMD_SET_UNSHARED_ALL, &rc, &rrc);
> +	report(cc == 0 && rc == 1, "Unshare all");
> +
> +	sie_expect_validity(&vm);
> +	sie(&vm);
> +	report((sie_get_validity(&vm) & 0xff00) == 0x2000,
> +	       "validity stopped, unshared");
> +
> +	/* Prepare the CPU reset */
> +	cc = uv_cmd_nodata(vm.sblk->pv_handle_config,
> +			   UVC_CMD_PREPARE_RESET, &rc, &rrc);
> +	report(cc == 0 && rc == 1, "Prepare reset call");
> +
> +	sie_expect_validity(&vm);
> +	sie(&vm);
> +	report((sie_get_validity(&vm) & 0xff00) == 0x2000,
> +	       "validity stopped, unshared, prepare");
> +
> +	/* Do the reset */
> +	cc = uv_cmd_nodata(vm.sblk->pv_handle_cpu,
> +			   UVC_CMD_CPU_RESET_INITIAL, &rc, &rrc);
> +	report(cc == 0 && rc == 1, "Clear reset cpu");
> +
> +	sie_expect_validity(&vm);
> +	sie(&vm);
> +	report((sie_get_validity(&vm) & 0xff00) == 0x2000,
> +	       "validity stopped, unshared, prepare, reset");
> +
> +	/* Load the PSW from 0x0 */
> +	cc = uv_set_cpu_state(vm.sblk->pv_handle_cpu, PV_CPU_STATE_OPR_LOAD);
> +	report(!cc, "Set cpu load");
> +
> +	/*
> +	 * Check if we executed the iaddr of the reset PSW, we should
> +	 * see a diagnose 0x9c PV instruction notification.
> +	 */
> +	sie(&vm);
> +	report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 &&
> +	       vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x9c &&
> +	       vm.save_area.guest.grs[0] == 42,
> +	       "intercept values after diag 0x308");
> +
> +
> +	uv_destroy_guest(&vm);
> +	report_prefix_pop();
> +}
> +

[...]

> diff --git a/s390x/snippets/asm/snippet-pv-diag-308.S b/s390x/snippets/asm/snippet-pv-diag-308.S
> new file mode 100644
> index 00000000..58c96173
> --- /dev/null
> +++ b/s390x/snippets/asm/snippet-pv-diag-308.S
> @@ -0,0 +1,67 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Diagnose 0x308 snippet used for PV IPL and reset testing
> + *
> + * Copyright (c) 2023 IBM Corp
> + *
> + * Authors:
> + *  Janosch Frank <frankja@linux.ibm.com>
> + */
> +#include <asm/asm-offsets.h>
> +.section .text
> +
> +/* Sets a reset PSW with the given PSW address */
> +.macro SET_RESET_PSW_ADDR label
> +lgrl	%r5, reset_psw
> +larl	%r6, \label
> +ogr	%r5, %r6
> +stg	%r5, 0
> +.endm
> +
> +/* Does a diagnose 308 with the given subcode */
> +.macro DIAG308 subcode
> +xgr	%r3, %r3
> +lghi	%r3, \subcode
> +diag	1, 3, 0x308
> +.endm
> +
> +sam64
> +
> +/* Execute the diag500 which will set the subcode we execute in gr2 */
> +diag	0, 0, 0x500
> +
> +/*
> + * A valid PGM new PSW can be a real problem since we never fall out
> + * of SIE and therefore effectively loop forever. 0 is a valid PSW
> + * therefore we re-use the reset_psw as this has the short PSW
> + * bit set which is invalid for a long PSW like the exception new
> + * PSWs.
> + *
> + * For subcode 0/1 there are no PGMs to consider.
> + */
> +lgrl   %r5, reset_psw
> +stg    %r5, GEN_LC_PGM_NEW_PSW
> +
> +/* Clean registers that are used */
> +xgr	%r0, %r0
> +xgr	%r1, %r1
> +xgr	%r3, %r3
> +xgr	%r4, %r4
> +xgr	%r5, %r5
> +xgr	%r6, %r6
> +
> +/* Subcode 0 - Modified Clear */

what about subcode 1?

> +SET_RESET_PSW_ADDR done
> +diag	%r0, %r2, 0x308
> +
> +/* Should never be executed because of the reset PSW */
> +diag	0, 0, 0x44
> +
> +done:
> +lghi	%r1, 42
> +diag	%r1, 0, 0x9c
> +
> +
> +	.align	8
> +reset_psw:
> +	.quad	0x0008000180000000

Re: [kvm-unit-tests PATCH 3/3] s390x: pv: Add IPL reset tests

[Janosch Frank]

On 2/17/23 17:42, Claudio Imbrenda wrote:
> On Wed,  1 Feb 2023 08:48:33 +0000
> Janosch Frank <frankja@linux.ibm.com> wrote:
> 
>> The diag308 requires extensive cooperation between the hypervisor and
>> the Ultravisor so the Ultravisor can make sure all necessary reset
>> steps have been done.
>>
>> Let's check if we get the correct validity errors.
>>
>> Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
>> ---
> 
> [...]
> 
>> +
>> +	/*
>> +	 * We need to perform several UV calls to emulate the subcode
>> +	 * 1. Failing to do that should result in a validity
>> +	 *
>> +	 * - Mark all cpus as stopped
>> +	 * - Reset the cpus, calling one gets an initial reset
>> +	 * - Load the reset PSW
>> +	 * - Unshare all
>> +	 * - Load the reset PSW
> 
> you forgot to mention prepare the reset, and the list does not reflect
> the order things are done in the code

Ok


> [...]

>> +/* Execute the diag500 which will set the subcode we execute in gr2 */
>> +diag	0, 0, 0x500
>> +
>> +/*
>> + * A valid PGM new PSW can be a real problem since we never fall out
>> + * of SIE and therefore effectively loop forever. 0 is a valid PSW
>> + * therefore we re-use the reset_psw as this has the short PSW
>> + * bit set which is invalid for a long PSW like the exception new
>> + * PSWs.
>> + *
>> + * For subcode 0/1 there are no PGMs to consider.
>> + */
>> +lgrl   %r5, reset_psw
>> +stg    %r5, GEN_LC_PGM_NEW_PSW
>> +
>> +/* Clean registers that are used */
>> +xgr	%r0, %r0
>> +xgr	%r1, %r1
>> +xgr	%r3, %r3
>> +xgr	%r4, %r4
>> +xgr	%r5, %r5
>> +xgr	%r6, %r6
>> +
>> +/* Subcode 0 - Modified Clear */
> 
> what about subcode 1?

My guess is that this hasn't been removed after a re-work of the code.
I suggest to remove the comment.

> 
>> +SET_RESET_PSW_ADDR done
>> +diag	%r0, %r2, 0x308
>> +
>> +/* Should never be executed because of the reset PSW */
>> +diag	0, 0, 0x44
>> +
>> +done:
>> +lghi	%r1, 42
>> +diag	%r1, 0, 0x9c
>> +
>> +
>> +	.align	8
>> +reset_psw:
>> +	.quad	0x0008000180000000
> 

Re: [kvm-unit-tests PATCH 3/3] s390x: pv: Add IPL reset tests

[Claudio Imbrenda]

On Tue, 21 Feb 2023 10:26:11 +0100
Janosch Frank <frankja@linux.ibm.com> wrote:

[...]

> 
> >> +/* Execute the diag500 which will set the subcode we execute in gr2 */
> >> +diag	0, 0, 0x500
> >> +
> >> +/*
> >> + * A valid PGM new PSW can be a real problem since we never fall out
> >> + * of SIE and therefore effectively loop forever. 0 is a valid PSW
> >> + * therefore we re-use the reset_psw as this has the short PSW
> >> + * bit set which is invalid for a long PSW like the exception new
> >> + * PSWs.
> >> + *
> >> + * For subcode 0/1 there are no PGMs to consider.
> >> + */
> >> +lgrl   %r5, reset_psw
> >> +stg    %r5, GEN_LC_PGM_NEW_PSW
> >> +
> >> +/* Clean registers that are used */
> >> +xgr	%r0, %r0
> >> +xgr	%r1, %r1
> >> +xgr	%r3, %r3
> >> +xgr	%r4, %r4
> >> +xgr	%r5, %r5
> >> +xgr	%r6, %r6
> >> +
> >> +/* Subcode 0 - Modified Clear */  
> > 
> > what about subcode 1?  
> 
> My guess is that this hasn't been removed after a re-work of the code.
> I suggest to remove the comment.

sounds good

> 
> >   
> >> +SET_RESET_PSW_ADDR done
> >> +diag	%r0, %r2, 0x308
> >> +
> >> +/* Should never be executed because of the reset PSW */
> >> +diag	0, 0, 0x44
> >> +
> >> +done:
> >> +lghi	%r1, 42
> >> +diag	%r1, 0, 0x9c
> >> +
> >> +
> >> +	.align	8
> >> +reset_psw:
> >> +	.quad	0x0008000180000000  
> >   
>