* [kvm-unit-tests PATCH v2 0/7] s390x: PV fixups
@ 2022-10-20 9:00 Janosch Frank
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 1/7] s390x: snippets: asm: Add a macro to write an exception PSW Janosch Frank
` (6 more replies)
0 siblings, 7 replies; 17+ messages in thread
From: Janosch Frank @ 2022-10-20 9:00 UTC (permalink / raw)
To: kvm; +Cc: imbrenda, seiden, nrb, scgl, thuth
A small set of patches that clean up the PV snippet handling.
v2:
* Macro uses 64bit PSW mask
* SBLK reset on PV destroy and uv_init() early return have been split off
Janosch Frank (7):
s390x: snippets: asm: Add a macro to write an exception PSW
s390x: MAKEFILE: Use $< instead of pathsubst
s390x: Add a linker script to assembly snippets
lib: s390x: sie: Improve validity handling and make it vm specific
lib: s390x: Use a new asce for each PV guest
lib: s390x: Enable reusability of VMs that were in PV mode
lib: s390x: sie: Properly populate SCA
lib/s390x/asm-offsets.c | 2 ++
lib/s390x/sie.c | 37 +++++++++++++-------
lib/s390x/sie.h | 43 ++++++++++++++++++++++--
lib/s390x/snippet.h | 14 +++-----
lib/s390x/uv.c | 35 +++++++++++++++++--
lib/s390x/uv.h | 5 ++-
s390x/Makefile | 18 +++++++---
s390x/cpu.S | 6 ++++
s390x/mvpg-sie.c | 2 +-
s390x/pv-diags.c | 6 ++--
s390x/snippets/asm/flat.lds | 35 +++++++++++++++++++
s390x/snippets/asm/macros.S | 28 +++++++++++++++
s390x/snippets/asm/snippet-pv-diag-288.S | 4 +--
s390x/snippets/asm/snippet-pv-diag-500.S | 6 ++--
14 files changed, 197 insertions(+), 44 deletions(-)
create mode 100644 s390x/snippets/asm/flat.lds
create mode 100644 s390x/snippets/asm/macros.S
--
2.34.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* [kvm-unit-tests PATCH v2 1/7] s390x: snippets: asm: Add a macro to write an exception PSW
2022-10-20 9:00 [kvm-unit-tests PATCH v2 0/7] s390x: PV fixups Janosch Frank
@ 2022-10-20 9:00 ` Janosch Frank
2022-10-20 9:14 ` Claudio Imbrenda
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 2/7] s390x: MAKEFILE: Use $< instead of pathsubst Janosch Frank
` (5 subsequent siblings)
6 siblings, 1 reply; 17+ messages in thread
From: Janosch Frank @ 2022-10-20 9:00 UTC (permalink / raw)
To: kvm; +Cc: imbrenda, seiden, nrb, scgl, thuth
Setting exception new PSWs is commonly needed so let's add a macro for
that.
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
Reviewed-by: Nico Boehr <nrb@linux.ibm.com>
---
s390x/snippets/asm/macros.S | 28 ++++++++++++++++++++++++
s390x/snippets/asm/snippet-pv-diag-288.S | 4 ++--
s390x/snippets/asm/snippet-pv-diag-500.S | 6 ++---
3 files changed, 32 insertions(+), 6 deletions(-)
create mode 100644 s390x/snippets/asm/macros.S
diff --git a/s390x/snippets/asm/macros.S b/s390x/snippets/asm/macros.S
new file mode 100644
index 00000000..667fb6dc
--- /dev/null
+++ b/s390x/snippets/asm/macros.S
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Commonly used assembly macros
+ *
+ * Copyright (c) 2022 IBM Corp
+ *
+ * Authors:
+ * Janosch Frank <frankja@linux.ibm.com>
+ */
+#include <asm/asm-offsets.h>
+
+/*
+ * Writes a PSW to addr_psw, useful for exception PSWs in lowcore
+ *
+ * reg is the scratch register used for temporary storage, it's NOT restored
+ * The psw address part is defined via psw_new_addr
+ * The psw mask part is always 64 bit
+ */
+.macro SET_PSW_NEW_ADDR reg, psw_new_addr, addr_psw
+larl \reg, psw_mask_64
+stg \reg, \addr_psw
+larl \reg, \psw_new_addr
+stg \reg, \addr_psw + 8
+.endm
+
+.section .rodata
+psw_mask_64:
+ .quad 0x0000000180000000
diff --git a/s390x/snippets/asm/snippet-pv-diag-288.S b/s390x/snippets/asm/snippet-pv-diag-288.S
index aaee3cd1..63f2113b 100644
--- a/s390x/snippets/asm/snippet-pv-diag-288.S
+++ b/s390x/snippets/asm/snippet-pv-diag-288.S
@@ -8,6 +8,7 @@
* Janosch Frank <frankja@linux.ibm.com>
*/
#include <asm/asm-offsets.h>
+#include "macros.S"
.section .text
/* Clean and pre-load registers that are used for diag 288 */
@@ -19,8 +20,7 @@ lghi %r1, 2
lghi %r2, 3
/* Let's jump to the pgm exit label on a PGM */
-larl %r4, exit_pgm
-stg %r4, GEN_LC_PGM_NEW_PSW + 8
+SET_PSW_NEW_ADDR 4, exit_pgm, GEN_LC_PGM_NEW_PSW
/* Execute the diag288 */
diag %r0, %r2, 0x288
diff --git a/s390x/snippets/asm/snippet-pv-diag-500.S b/s390x/snippets/asm/snippet-pv-diag-500.S
index 8dd66bd9..f4d75388 100644
--- a/s390x/snippets/asm/snippet-pv-diag-500.S
+++ b/s390x/snippets/asm/snippet-pv-diag-500.S
@@ -8,6 +8,7 @@
* Janosch Frank <frankja@linux.ibm.com>
*/
#include <asm/asm-offsets.h>
+#include "macros.S"
.section .text
/* Clean and pre-load registers that are used for diag 500 */
@@ -21,10 +22,7 @@ lghi %r3, 3
lghi %r4, 4
/* Let's jump to the next label on a PGM */
-xgr %r5, %r5
-stg %r5, GEN_LC_PGM_NEW_PSW
-larl %r5, next
-stg %r5, GEN_LC_PGM_NEW_PSW + 8
+SET_PSW_NEW_ADDR 5, next, GEN_LC_PGM_NEW_PSW
/* Execute the diag500 */
diag 0, 0, 0x500
--
2.34.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [kvm-unit-tests PATCH v2 2/7] s390x: MAKEFILE: Use $< instead of pathsubst
2022-10-20 9:00 [kvm-unit-tests PATCH v2 0/7] s390x: PV fixups Janosch Frank
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 1/7] s390x: snippets: asm: Add a macro to write an exception PSW Janosch Frank
@ 2022-10-20 9:00 ` Janosch Frank
2022-10-20 9:14 ` Claudio Imbrenda
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 3/7] s390x: Add a linker script to assembly snippets Janosch Frank
` (4 subsequent siblings)
6 siblings, 1 reply; 17+ messages in thread
From: Janosch Frank @ 2022-10-20 9:00 UTC (permalink / raw)
To: kvm; +Cc: imbrenda, seiden, nrb, scgl, thuth
No need to mangle strings if we already have the value at hand.
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
Reviewed-by: Steffen Eiden <seiden@linux.ibm.com>
Reviewed-by: Nico Boehr <nrb@linux.ibm.com>
---
s390x/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/s390x/Makefile b/s390x/Makefile
index 649486f2..3b175015 100644
--- a/s390x/Makefile
+++ b/s390x/Makefile
@@ -136,7 +136,7 @@ $(SNIPPET_DIR)/asm/%.gbin: $(SNIPPET_DIR)/asm/%.o
truncate -s '%4096' $@
$(SNIPPET_DIR)/c/%.gbin: $(SNIPPET_DIR)/c/%.o $(snippet_lib) $(FLATLIBS)
- $(CC) $(LDFLAGS) -o $@ -T $(SRCDIR)/s390x/snippets/c/flat.lds $(patsubst %.gbin,%.o,$@) $(snippet_lib) $(FLATLIBS)
+ $(CC) $(LDFLAGS) -o $@ -T $(SRCDIR)/s390x/snippets/c/flat.lds $< $(snippet_lib) $(FLATLIBS)
$(OBJCOPY) -O binary -j ".rodata" -j ".lowcore" -j ".text" -j ".data" -j ".bss" --set-section-flags .bss=alloc,load,contents $@ $@
truncate -s '%4096' $@
--
2.34.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [kvm-unit-tests PATCH v2 3/7] s390x: Add a linker script to assembly snippets
2022-10-20 9:00 [kvm-unit-tests PATCH v2 0/7] s390x: PV fixups Janosch Frank
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 1/7] s390x: snippets: asm: Add a macro to write an exception PSW Janosch Frank
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 2/7] s390x: MAKEFILE: Use $< instead of pathsubst Janosch Frank
@ 2022-10-20 9:00 ` Janosch Frank
2022-10-20 10:02 ` Claudio Imbrenda
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 4/7] lib: s390x: sie: Improve validity handling and make it vm specific Janosch Frank
` (3 subsequent siblings)
6 siblings, 1 reply; 17+ messages in thread
From: Janosch Frank @ 2022-10-20 9:00 UTC (permalink / raw)
To: kvm; +Cc: imbrenda, seiden, nrb, scgl, thuth
A linker script has a few benefits:
- We can easily define a lowcore and load the snippet from 0x0 instead
of 0x4000 which makes asm snippets behave like c snippets
- We can easily define an invalid PGM new PSW to ensure an exit on a
guest PGM
- We can remove a lot of the offset variables from lib/s390x/snippet.h
As we gain another step and file extension by linking, a comment
explains which file extensions are generated and why.
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
---
lib/s390x/snippet.h | 14 +++++---------
s390x/Makefile | 18 ++++++++++++++----
s390x/mvpg-sie.c | 2 +-
s390x/pv-diags.c | 6 +++---
s390x/snippets/asm/flat.lds | 35 +++++++++++++++++++++++++++++++++++
5 files changed, 58 insertions(+), 17 deletions(-)
create mode 100644 s390x/snippets/asm/flat.lds
diff --git a/lib/s390x/snippet.h b/lib/s390x/snippet.h
index b17b2a4c..fcd04081 100644
--- a/lib/s390x/snippet.h
+++ b/lib/s390x/snippet.h
@@ -32,8 +32,6 @@
#define SNIPPET_PV_TWEAK0 0x42UL
#define SNIPPET_PV_TWEAK1 0UL
-#define SNIPPET_OFF_C 0
-#define SNIPPET_OFF_ASM 0x4000
/*
@@ -57,15 +55,14 @@ static const struct psw snippet_psw = {
* @vm: VM that this function will populated, has to be initialized already
* @gbin: Snippet gbin data pointer
* @gbin_len: Length of the gbin data
- * @off: Offset from guest absolute 0x0 where snippet is copied to
*/
static inline void snippet_init(struct vm *vm, const char *gbin,
- uint64_t gbin_len, uint64_t off)
+ uint64_t gbin_len)
{
uint64_t mso = vm->sblk->mso;
/* Copy test image to guest memory */
- memcpy((void *)mso + off, gbin, gbin_len);
+ memcpy((void *)mso, gbin, gbin_len);
/* Setup guest PSW */
vm->sblk->gpsw = snippet_psw;
@@ -87,23 +84,22 @@ static inline void snippet_init(struct vm *vm, const char *gbin,
* @hdr: Snippet SE header data pointer
* @gbin_len: Length of the gbin data
* @hdr_len: Length of the hdr data
- * @off: Offset from guest absolute 0x0 where snippet is copied to
*/
static inline void snippet_pv_init(struct vm *vm, const char *gbin,
const char *hdr, uint64_t gbin_len,
- uint64_t hdr_len, uint64_t off)
+ uint64_t hdr_len)
{
uint64_t tweak[2] = {SNIPPET_PV_TWEAK0, SNIPPET_PV_TWEAK1};
uint64_t mso = vm->sblk->mso;
int i;
- snippet_init(vm, gbin, gbin_len, off);
+ snippet_init(vm, gbin, gbin_len);
uv_create_guest(vm);
uv_set_se_hdr(vm->uv.vm_handle, (void *)hdr, hdr_len);
/* Unpack works on guest addresses so we only need off */
- uv_unpack(vm, off, gbin_len, tweak[0]);
+ uv_unpack(vm, 0, gbin_len, tweak[0]);
uv_verify_load(vm);
/*
diff --git a/s390x/Makefile b/s390x/Makefile
index 3b175015..0eaa72f4 100644
--- a/s390x/Makefile
+++ b/s390x/Makefile
@@ -124,6 +124,13 @@ else
snippet-hdr-obj =
endif
+# Each snippet will generate the following files (in order): \
+ *.o is a snippet that has been compiled \
+ *.ol is a snippet that has been linked \
+ *.gbin is a snippet that has been converted to binary \
+ *.gobj is the final format after converting the binary into a elf file again, \
+ it will be linked into the tests
+
# the asm/c snippets %.o have additional generated files as dependencies
$(SNIPPET_DIR)/asm/%.o: $(SNIPPET_DIR)/asm/%.S $(asm-offsets)
$(CC) $(CFLAGS) -c -nostdlib -o $@ $<
@@ -131,17 +138,20 @@ $(SNIPPET_DIR)/asm/%.o: $(SNIPPET_DIR)/asm/%.S $(asm-offsets)
$(SNIPPET_DIR)/c/%.o: $(SNIPPET_DIR)/c/%.c $(asm-offsets)
$(CC) $(CFLAGS) -c -nostdlib -o $@ $<
-$(SNIPPET_DIR)/asm/%.gbin: $(SNIPPET_DIR)/asm/%.o
- $(OBJCOPY) -O binary -j ".rodata" -j ".text" -j ".data" -j ".bss" --set-section-flags .bss=alloc,load,contents $(patsubst %.gbin,%.o,$@) $@
+$(SNIPPET_DIR)/asm/%.ol: $(SNIPPET_DIR)/asm/%.o
+ $(CC) $(LDFLAGS) -o $@ -T $(SNIPPET_DIR)/asm/flat.lds $<
+
+$(SNIPPET_DIR)/asm/%.gbin: $(SNIPPET_DIR)/asm/%.ol
+ $(OBJCOPY) -O binary -j ".rodata" -j ".lowcore" -j ".text" -j ".data" -j ".bss" --set-section-flags .bss=alloc,load,contents $< $@
truncate -s '%4096' $@
$(SNIPPET_DIR)/c/%.gbin: $(SNIPPET_DIR)/c/%.o $(snippet_lib) $(FLATLIBS)
- $(CC) $(LDFLAGS) -o $@ -T $(SRCDIR)/s390x/snippets/c/flat.lds $< $(snippet_lib) $(FLATLIBS)
+ $(CC) $(LDFLAGS) -o $@ -T $(SNIPPET_DIR)/c/flat.lds $< $(snippet_lib) $(FLATLIBS)
$(OBJCOPY) -O binary -j ".rodata" -j ".lowcore" -j ".text" -j ".data" -j ".bss" --set-section-flags .bss=alloc,load,contents $@ $@
truncate -s '%4096' $@
$(SNIPPET_DIR)/asm/%.hdr: $(SNIPPET_DIR)/asm/%.gbin $(HOST_KEY_DOCUMENT)
- $(GEN_SE_HEADER) -k $(HOST_KEY_DOCUMENT) -c $<,0x4000,0x00000000000000420000000000000000 --psw-addr 0x4000 -o $@
+ $(GEN_SE_HEADER) -k $(HOST_KEY_DOCUMENT) -c $<,0x0,0x00000000000000420000000000000000 --psw-addr 0x4000 -o $@
$(SNIPPET_DIR)/c/%.hdr: $(SNIPPET_DIR)/c/%.gbin $(HOST_KEY_DOCUMENT)
$(GEN_SE_HEADER) -k $(HOST_KEY_DOCUMENT) -c $<,0x0,0x00000000000000420000000000000000 --psw-addr 0x4000 -o $@
diff --git a/s390x/mvpg-sie.c b/s390x/mvpg-sie.c
index 46a2edb6..17e209ad 100644
--- a/s390x/mvpg-sie.c
+++ b/s390x/mvpg-sie.c
@@ -87,7 +87,7 @@ static void setup_guest(void)
snippet_setup_guest(&vm, false);
snippet_init(&vm, SNIPPET_NAME_START(c, mvpg_snippet),
- SNIPPET_LEN(c, mvpg_snippet), SNIPPET_OFF_C);
+ SNIPPET_LEN(c, mvpg_snippet));
/* Enable MVPG interpretation as we want to test KVM and not ourselves */
vm.sblk->eca = ECA_MVPGI;
diff --git a/s390x/pv-diags.c b/s390x/pv-diags.c
index 9ced68c7..d472c994 100644
--- a/s390x/pv-diags.c
+++ b/s390x/pv-diags.c
@@ -28,7 +28,7 @@ static void test_diag_500(void)
snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_diag_500),
SNIPPET_HDR_START(asm, snippet_pv_diag_500),
- size_gbin, size_hdr, SNIPPET_OFF_ASM);
+ size_gbin, size_hdr);
sie(&vm);
report(vm.sblk->icptcode == ICPT_PV_INSTR && vm.sblk->ipa == 0x8302 &&
@@ -83,7 +83,7 @@ static void test_diag_288(void)
snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_diag_288),
SNIPPET_HDR_START(asm, snippet_pv_diag_288),
- size_gbin, size_hdr, SNIPPET_OFF_ASM);
+ size_gbin, size_hdr);
sie(&vm);
report(vm.sblk->icptcode == ICPT_PV_INSTR && vm.sblk->ipa == 0x8302 &&
@@ -124,7 +124,7 @@ static void test_diag_yield(void)
snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_diag_yield),
SNIPPET_HDR_START(asm, snippet_pv_diag_yield),
- size_gbin, size_hdr, SNIPPET_OFF_ASM);
+ size_gbin, size_hdr);
/* 0x44 */
report_prefix_push("0x44");
diff --git a/s390x/snippets/asm/flat.lds b/s390x/snippets/asm/flat.lds
new file mode 100644
index 00000000..388d7d5d
--- /dev/null
+++ b/s390x/snippets/asm/flat.lds
@@ -0,0 +1,35 @@
+SECTIONS
+{
+ .lowcore : {
+ /* Restart new PSW for booting via PSW restart. */
+ . = 0x1a0;
+ QUAD(0x0000000180000000)
+ QUAD(0x0000000000004000)
+ /*
+ * Invalid PGM new PSW so we hopefully get a code 8
+ * intercept on a PGM for PV snippets.
+ */
+ . = 0x1d0;
+ QUAD(0x0008000000000000)
+ QUAD(0x0000000000000001)
+ }
+ . = 0x4000;
+ .text : {
+ *(.text)
+ *(.text.*)
+ }
+ . = ALIGN(64K);
+ etext = .;
+ . = ALIGN(16);
+ .data : {
+ *(.data)
+ *(.data.rel*)
+ }
+ . = ALIGN(16);
+ .rodata : { *(.rodata) *(.rodata.*) }
+ . = ALIGN(16);
+ __bss_start = .;
+ .bss : { *(.bss) }
+ __bss_end = .;
+ . = ALIGN(64K);
+}
--
2.34.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [kvm-unit-tests PATCH v2 4/7] lib: s390x: sie: Improve validity handling and make it vm specific
2022-10-20 9:00 [kvm-unit-tests PATCH v2 0/7] s390x: PV fixups Janosch Frank
` (2 preceding siblings ...)
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 3/7] s390x: Add a linker script to assembly snippets Janosch Frank
@ 2022-10-20 9:00 ` Janosch Frank
2022-10-20 9:18 ` Claudio Imbrenda
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 5/7] lib: s390x: Use a new asce for each PV guest Janosch Frank
` (2 subsequent siblings)
6 siblings, 1 reply; 17+ messages in thread
From: Janosch Frank @ 2022-10-20 9:00 UTC (permalink / raw)
To: kvm; +Cc: imbrenda, seiden, nrb, scgl, thuth
The current library doesn't support running multiple vms at once as it
stores the validity once and not per vm. Let's move the validity
handling into the vm and introduce a new function to retrieve the vir.
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
Reviewed-by: Steffen Eiden <seiden@linux.ibm.com>
Reviewed-by: Nico Boehr <nrb@linux.ibm.com>
---
lib/s390x/sie.c | 27 ++++++++++++++-------------
lib/s390x/sie.h | 6 ++++--
2 files changed, 18 insertions(+), 15 deletions(-)
diff --git a/lib/s390x/sie.c b/lib/s390x/sie.c
index 00aff713..3fee3def 100644
--- a/lib/s390x/sie.c
+++ b/lib/s390x/sie.c
@@ -15,19 +15,22 @@
#include <libcflat.h>
#include <alloc_page.h>
-static bool validity_expected;
-static uint16_t vir; /* Validity interception reason */
-
-void sie_expect_validity(void)
+void sie_expect_validity(struct vm *vm)
{
- validity_expected = true;
- vir = 0;
+ vm->validity_expected = true;
}
-void sie_check_validity(uint16_t vir_exp)
+uint16_t sie_get_validity(struct vm *vm)
{
+ assert(vm->sblk->icptcode == ICPT_VALIDITY);
+ return vm->sblk->ipb >> 16;
+}
+
+void sie_check_validity(struct vm *vm, uint16_t vir_exp)
+{
+ uint16_t vir = sie_get_validity(vm);
+
report(vir_exp == vir, "VALIDITY: %x", vir);
- vir = 0;
}
void sie_handle_validity(struct vm *vm)
@@ -35,11 +38,9 @@ void sie_handle_validity(struct vm *vm)
if (vm->sblk->icptcode != ICPT_VALIDITY)
return;
- vir = vm->sblk->ipb >> 16;
-
- if (!validity_expected)
- report_abort("VALIDITY: %x", vir);
- validity_expected = false;
+ if (!vm->validity_expected)
+ report_abort("VALIDITY: %x", sie_get_validity(vm));
+ vm->validity_expected = false;
}
void sie(struct vm *vm)
diff --git a/lib/s390x/sie.h b/lib/s390x/sie.h
index de91ea5a..320c4218 100644
--- a/lib/s390x/sie.h
+++ b/lib/s390x/sie.h
@@ -233,14 +233,16 @@ struct vm {
struct vm_uv uv; /* PV UV information */
/* Ptr to first guest page */
uint8_t *guest_mem;
+ bool validity_expected;
};
extern void sie_entry(void);
extern void sie_exit(void);
extern void sie64a(struct kvm_s390_sie_block *sblk, struct vm_save_area *save_area);
void sie(struct vm *vm);
-void sie_expect_validity(void);
-void sie_check_validity(uint16_t vir_exp);
+void sie_expect_validity(struct vm *vm);
+uint16_t sie_get_validity(struct vm *vm);
+void sie_check_validity(struct vm *vm, uint16_t vir_exp);
void sie_handle_validity(struct vm *vm);
void sie_guest_sca_create(struct vm *vm);
void sie_guest_create(struct vm *vm, uint64_t guest_mem, uint64_t guest_mem_len);
--
2.34.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [kvm-unit-tests PATCH v2 5/7] lib: s390x: Use a new asce for each PV guest
2022-10-20 9:00 [kvm-unit-tests PATCH v2 0/7] s390x: PV fixups Janosch Frank
` (3 preceding siblings ...)
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 4/7] lib: s390x: sie: Improve validity handling and make it vm specific Janosch Frank
@ 2022-10-20 9:00 ` Janosch Frank
2022-10-20 9:25 ` Claudio Imbrenda
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 6/7] lib: s390x: Enable reusability of VMs that were in PV mode Janosch Frank
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 7/7] lib: s390x: sie: Properly populate SCA Janosch Frank
6 siblings, 1 reply; 17+ messages in thread
From: Janosch Frank @ 2022-10-20 9:00 UTC (permalink / raw)
To: kvm; +Cc: imbrenda, seiden, nrb, scgl, thuth
Every PV guest needs its own ASCE so let's copy the topmost table
designated by CR1 to create a new ASCE for the PV guest. Before and
after SIE we now need to switch ASCEs to and from the PV guest / test
ASCE. The SIE assembly function does that automatically.
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
---
lib/s390x/asm-offsets.c | 2 ++
lib/s390x/sie.c | 2 ++
lib/s390x/sie.h | 2 ++
lib/s390x/uv.c | 24 +++++++++++++++++++++++-
lib/s390x/uv.h | 5 ++---
s390x/cpu.S | 6 ++++++
6 files changed, 37 insertions(+), 4 deletions(-)
diff --git a/lib/s390x/asm-offsets.c b/lib/s390x/asm-offsets.c
index fbea3278..f612f327 100644
--- a/lib/s390x/asm-offsets.c
+++ b/lib/s390x/asm-offsets.c
@@ -75,9 +75,11 @@ int main(void)
OFFSET(SIE_SAVEAREA_HOST_GRS, vm_save_area, host.grs[0]);
OFFSET(SIE_SAVEAREA_HOST_FPRS, vm_save_area, host.fprs[0]);
OFFSET(SIE_SAVEAREA_HOST_FPC, vm_save_area, host.fpc);
+ OFFSET(SIE_SAVEAREA_HOST_ASCE, vm_save_area, host.asce);
OFFSET(SIE_SAVEAREA_GUEST_GRS, vm_save_area, guest.grs[0]);
OFFSET(SIE_SAVEAREA_GUEST_FPRS, vm_save_area, guest.fprs[0]);
OFFSET(SIE_SAVEAREA_GUEST_FPC, vm_save_area, guest.fpc);
+ OFFSET(SIE_SAVEAREA_GUEST_ASCE, vm_save_area, guest.asce);
OFFSET(STACK_FRAME_INT_BACKCHAIN, stack_frame_int, back_chain);
OFFSET(STACK_FRAME_INT_FPC, stack_frame_int, fpc);
OFFSET(STACK_FRAME_INT_FPRS, stack_frame_int, fprs);
diff --git a/lib/s390x/sie.c b/lib/s390x/sie.c
index 3fee3def..6efad965 100644
--- a/lib/s390x/sie.c
+++ b/lib/s390x/sie.c
@@ -85,6 +85,8 @@ void sie_guest_create(struct vm *vm, uint64_t guest_mem, uint64_t guest_mem_len)
/* Guest memory chunks are always 1MB */
assert(!(guest_mem_len & ~HPAGE_MASK));
+ /* For non-PV guests we re-use the host's ASCE for ease of use */
+ vm->save_area.guest.asce = stctg(1);
/* Currently MSO/MSL is the easiest option */
vm->sblk->mso = (uint64_t)guest_mem;
vm->sblk->msl = (uint64_t)guest_mem + ((guest_mem_len - 1) & HPAGE_MASK);
diff --git a/lib/s390x/sie.h b/lib/s390x/sie.h
index 320c4218..3e3605c9 100644
--- a/lib/s390x/sie.h
+++ b/lib/s390x/sie.h
@@ -205,12 +205,14 @@ union {
struct vm_uv {
uint64_t vm_handle;
uint64_t vcpu_handle;
+ uint64_t asce;
void *conf_base_stor;
void *conf_var_stor;
void *cpu_stor;
};
struct vm_save_regs {
+ uint64_t asce;
uint64_t grs[16];
uint64_t fprs[16];
uint32_t fpc;
diff --git a/lib/s390x/uv.c b/lib/s390x/uv.c
index 3b4cafa9..0b6eb843 100644
--- a/lib/s390x/uv.c
+++ b/lib/s390x/uv.c
@@ -90,6 +90,25 @@ void uv_init(void)
initialized = true;
}
+/*
+ * Create a new ASCE for the UV config because they can't be shared
+ * for security reasons. We just simply copy the top most table into a
+ * fresh set of allocated pages and use those pages as the asce.
+ */
+static uint64_t create_asce(void)
+{
+ void *pgd_new, *pgd_old;
+ uint64_t asce = stctg(1);
+
+ pgd_new = memalign_pages_flags(PAGE_SIZE, PAGE_SIZE * 4, 0);
+ pgd_old = (void *)(asce & PAGE_MASK);
+
+ memcpy(pgd_new, pgd_old, PAGE_SIZE * 4);
+
+ asce = __pa(pgd_new) | ASCE_DT_REGION1 | REGION_TABLE_LENGTH | ASCE_P;
+ return asce;
+}
+
void uv_create_guest(struct vm *vm)
{
struct uv_cb_cgc uvcb_cgc = {
@@ -125,7 +144,8 @@ void uv_create_guest(struct vm *vm)
vm->uv.cpu_stor = memalign_pages_flags(PAGE_SIZE, uvcb_qui.cpu_stor_len, 0);
uvcb_csc.stor_origin = (uint64_t)vm->uv.cpu_stor;
- uvcb_cgc.guest_asce = (uint64_t)stctg(1);
+ uvcb_cgc.guest_asce = create_asce();
+ vm->save_area.guest.asce = uvcb_cgc.guest_asce;
uvcb_cgc.guest_sca = (uint64_t)vm->sca;
cc = uv_call(0, (uint64_t)&uvcb_cgc);
@@ -166,6 +186,8 @@ void uv_destroy_guest(struct vm *vm)
assert(cc == 0);
free_pages(vm->uv.conf_base_stor);
free_pages(vm->uv.conf_var_stor);
+
+ free_pages((void *)(vm->uv.asce & PAGE_MASK));
}
int uv_unpack(struct vm *vm, uint64_t addr, uint64_t len, uint64_t tweak)
diff --git a/lib/s390x/uv.h b/lib/s390x/uv.h
index 44264861..5fe29bda 100644
--- a/lib/s390x/uv.h
+++ b/lib/s390x/uv.h
@@ -28,9 +28,8 @@ static inline void uv_setup_asces(void)
/* We need to have a valid primary ASCE to run guests. */
setup_vm();
- /* Set P bit in ASCE as it is required for PV guests */
- asce = stctg(1) | ASCE_P;
- lctlg(1, asce);
+ /* Grab the ASCE which setup_vm() just set up */
+ asce = stctg(1);
/* Copy ASCE into home space CR */
lctlg(13, asce);
diff --git a/s390x/cpu.S b/s390x/cpu.S
index 82b5e25d..45bd551a 100644
--- a/s390x/cpu.S
+++ b/s390x/cpu.S
@@ -76,6 +76,9 @@ sie64a:
.endr
stfpc SIE_SAVEAREA_HOST_FPC(%r3)
+ stctg %c1, %c1, SIE_SAVEAREA_HOST_ASCE(%r3)
+ lctlg %c1, %c1, SIE_SAVEAREA_GUEST_ASCE(%r3)
+
# Store scb and save_area pointer into stack frame
stg %r2,__SF_SIE_CONTROL(%r15) # save control block pointer
stg %r3,__SF_SIE_SAVEAREA(%r15) # save guest register save area
@@ -102,6 +105,9 @@ sie_exit:
# Load guest register save area
lg %r14,__SF_SIE_SAVEAREA(%r15)
+ # Restore the host asce
+ lctlg %c1, %c1, SIE_SAVEAREA_HOST_ASCE(%r14)
+
# Store guest's gprs, fprs and fpc
stmg %r0,%r13,SIE_SAVEAREA_GUEST_GRS(%r14) # save guest gprs 0-13
.irp i, 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15
--
2.34.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [kvm-unit-tests PATCH v2 6/7] lib: s390x: Enable reusability of VMs that were in PV mode
2022-10-20 9:00 [kvm-unit-tests PATCH v2 0/7] s390x: PV fixups Janosch Frank
` (4 preceding siblings ...)
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 5/7] lib: s390x: Use a new asce for each PV guest Janosch Frank
@ 2022-10-20 9:00 ` Janosch Frank
2022-10-20 9:26 ` Claudio Imbrenda
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 7/7] lib: s390x: sie: Properly populate SCA Janosch Frank
6 siblings, 1 reply; 17+ messages in thread
From: Janosch Frank @ 2022-10-20 9:00 UTC (permalink / raw)
To: kvm; +Cc: imbrenda, seiden, nrb, scgl, thuth
Convert the sblk to non-PV when the PV guest is destroyed.
Early return in uv_init() instead of running into the assert. This is
necessary since snippet_pv_init() will always call uv_init().
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
---
lib/s390x/uv.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/lib/s390x/uv.c b/lib/s390x/uv.c
index 0b6eb843..99775929 100644
--- a/lib/s390x/uv.c
+++ b/lib/s390x/uv.c
@@ -76,7 +76,8 @@ void uv_init(void)
int cc;
/* Let's not do this twice */
- assert(!initialized);
+ if (initialized)
+ return;
/* Query is done on initialization but let's check anyway */
assert(uvcb_qui.header.rc == 1 || uvcb_qui.header.rc == 0x100);
@@ -188,6 +189,14 @@ void uv_destroy_guest(struct vm *vm)
free_pages(vm->uv.conf_var_stor);
free_pages((void *)(vm->uv.asce & PAGE_MASK));
+ memset(&vm->uv, 0, sizeof(vm->uv));
+
+ /* Convert the sblk back to non-PV */
+ vm->save_area.guest.asce = stctg(1);
+ vm->sblk->sdf = 0;
+ vm->sblk->sidad = 0;
+ vm->sblk->pv_handle_cpu = 0;
+ vm->sblk->pv_handle_config = 0;
}
int uv_unpack(struct vm *vm, uint64_t addr, uint64_t len, uint64_t tweak)
--
2.34.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [kvm-unit-tests PATCH v2 7/7] lib: s390x: sie: Properly populate SCA
2022-10-20 9:00 [kvm-unit-tests PATCH v2 0/7] s390x: PV fixups Janosch Frank
` (5 preceding siblings ...)
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 6/7] lib: s390x: Enable reusability of VMs that were in PV mode Janosch Frank
@ 2022-10-20 9:00 ` Janosch Frank
2022-10-20 9:50 ` Claudio Imbrenda
6 siblings, 1 reply; 17+ messages in thread
From: Janosch Frank @ 2022-10-20 9:00 UTC (permalink / raw)
To: kvm; +Cc: imbrenda, seiden, nrb, scgl, thuth
CPU0 is the only cpu that's being used but we should still mark it as
online and set the SDA in the SCA.
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
Reviewed-by: Nico Boehr <nrb@linux.ibm.com>
---
lib/s390x/sie.c | 8 ++++++++
lib/s390x/sie.h | 35 ++++++++++++++++++++++++++++++++++-
2 files changed, 42 insertions(+), 1 deletion(-)
diff --git a/lib/s390x/sie.c b/lib/s390x/sie.c
index 6efad965..a71985b6 100644
--- a/lib/s390x/sie.c
+++ b/lib/s390x/sie.c
@@ -9,6 +9,7 @@
*/
#include <asm/barrier.h>
+#include <bitops.h>
#include <libcflat.h>
#include <sie.h>
#include <asm/page.h>
@@ -72,6 +73,13 @@ void sie_guest_sca_create(struct vm *vm)
vm->sblk->scaoh = ((uint64_t)vm->sca >> 32);
vm->sblk->scaol = (uint64_t)vm->sca & ~0x3fU;
vm->sblk->ecb2 |= ECB2_ESCA;
+
+ /* Enable SIGP sense running interpretation */
+ vm->sblk->ecb |= ECB_SRSI;
+
+ /* We assume that cpu 0 is always part of the vm */
+ vm->sca->mcn[0] = BIT(63);
+ vm->sca->cpu[0].sda = (uint64_t)vm->sblk;
}
/* Initializes the struct vm members like the SIE control block. */
diff --git a/lib/s390x/sie.h b/lib/s390x/sie.h
index 3e3605c9..a27a8401 100644
--- a/lib/s390x/sie.h
+++ b/lib/s390x/sie.h
@@ -202,6 +202,39 @@ union {
uint64_t pv_grregs[16]; /* 0x0380 */
} __attribute__((packed));
+union esca_sigp_ctrl {
+ uint16_t value;
+ struct {
+ uint8_t c : 1;
+ uint8_t reserved: 7;
+ uint8_t scn;
+ };
+};
+
+struct esca_entry {
+ union esca_sigp_ctrl sigp_ctrl;
+ uint16_t reserved1[3];
+ uint64_t sda;
+ uint64_t reserved2[6];
+};
+
+union ipte_control {
+ unsigned long val;
+ struct {
+ unsigned long k : 1;
+ unsigned long kh : 31;
+ unsigned long kg : 32;
+ };
+};
+
+struct esca_block {
+ union ipte_control ipte_control;
+ uint64_t reserved1[7];
+ uint64_t mcn[4];
+ uint64_t reserved2[20];
+ struct esca_entry cpu[256];
+};
+
struct vm_uv {
uint64_t vm_handle;
uint64_t vcpu_handle;
@@ -230,7 +263,7 @@ struct vm_save_area {
struct vm {
struct kvm_s390_sie_block *sblk;
struct vm_save_area save_area;
- void *sca; /* System Control Area */
+ struct esca_block *sca; /* System Control Area */
uint8_t *crycb; /* Crypto Control Block */
struct vm_uv uv; /* PV UV information */
/* Ptr to first guest page */
--
2.34.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [kvm-unit-tests PATCH v2 1/7] s390x: snippets: asm: Add a macro to write an exception PSW
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 1/7] s390x: snippets: asm: Add a macro to write an exception PSW Janosch Frank
@ 2022-10-20 9:14 ` Claudio Imbrenda
0 siblings, 0 replies; 17+ messages in thread
From: Claudio Imbrenda @ 2022-10-20 9:14 UTC (permalink / raw)
To: Janosch Frank; +Cc: kvm, seiden, nrb, scgl, thuth
On Thu, 20 Oct 2022 09:00:03 +0000
Janosch Frank <frankja@linux.ibm.com> wrote:
> Setting exception new PSWs is commonly needed so let's add a macro for
> that.
>
> Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
> Reviewed-by: Nico Boehr <nrb@linux.ibm.com>
Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
> ---
> s390x/snippets/asm/macros.S | 28 ++++++++++++++++++++++++
> s390x/snippets/asm/snippet-pv-diag-288.S | 4 ++--
> s390x/snippets/asm/snippet-pv-diag-500.S | 6 ++---
> 3 files changed, 32 insertions(+), 6 deletions(-)
> create mode 100644 s390x/snippets/asm/macros.S
>
> diff --git a/s390x/snippets/asm/macros.S b/s390x/snippets/asm/macros.S
> new file mode 100644
> index 00000000..667fb6dc
> --- /dev/null
> +++ b/s390x/snippets/asm/macros.S
> @@ -0,0 +1,28 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Commonly used assembly macros
> + *
> + * Copyright (c) 2022 IBM Corp
> + *
> + * Authors:
> + * Janosch Frank <frankja@linux.ibm.com>
> + */
> +#include <asm/asm-offsets.h>
> +
> +/*
> + * Writes a PSW to addr_psw, useful for exception PSWs in lowcore
> + *
> + * reg is the scratch register used for temporary storage, it's NOT restored
> + * The psw address part is defined via psw_new_addr
> + * The psw mask part is always 64 bit
> + */
> +.macro SET_PSW_NEW_ADDR reg, psw_new_addr, addr_psw
> +larl \reg, psw_mask_64
> +stg \reg, \addr_psw
> +larl \reg, \psw_new_addr
> +stg \reg, \addr_psw + 8
> +.endm
> +
> +.section .rodata
> +psw_mask_64:
> + .quad 0x0000000180000000
> diff --git a/s390x/snippets/asm/snippet-pv-diag-288.S b/s390x/snippets/asm/snippet-pv-diag-288.S
> index aaee3cd1..63f2113b 100644
> --- a/s390x/snippets/asm/snippet-pv-diag-288.S
> +++ b/s390x/snippets/asm/snippet-pv-diag-288.S
> @@ -8,6 +8,7 @@
> * Janosch Frank <frankja@linux.ibm.com>
> */
> #include <asm/asm-offsets.h>
> +#include "macros.S"
> .section .text
>
> /* Clean and pre-load registers that are used for diag 288 */
> @@ -19,8 +20,7 @@ lghi %r1, 2
> lghi %r2, 3
>
> /* Let's jump to the pgm exit label on a PGM */
> -larl %r4, exit_pgm
> -stg %r4, GEN_LC_PGM_NEW_PSW + 8
> +SET_PSW_NEW_ADDR 4, exit_pgm, GEN_LC_PGM_NEW_PSW
>
> /* Execute the diag288 */
> diag %r0, %r2, 0x288
> diff --git a/s390x/snippets/asm/snippet-pv-diag-500.S b/s390x/snippets/asm/snippet-pv-diag-500.S
> index 8dd66bd9..f4d75388 100644
> --- a/s390x/snippets/asm/snippet-pv-diag-500.S
> +++ b/s390x/snippets/asm/snippet-pv-diag-500.S
> @@ -8,6 +8,7 @@
> * Janosch Frank <frankja@linux.ibm.com>
> */
> #include <asm/asm-offsets.h>
> +#include "macros.S"
> .section .text
>
> /* Clean and pre-load registers that are used for diag 500 */
> @@ -21,10 +22,7 @@ lghi %r3, 3
> lghi %r4, 4
>
> /* Let's jump to the next label on a PGM */
> -xgr %r5, %r5
> -stg %r5, GEN_LC_PGM_NEW_PSW
> -larl %r5, next
> -stg %r5, GEN_LC_PGM_NEW_PSW + 8
> +SET_PSW_NEW_ADDR 5, next, GEN_LC_PGM_NEW_PSW
>
> /* Execute the diag500 */
> diag 0, 0, 0x500
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [kvm-unit-tests PATCH v2 2/7] s390x: MAKEFILE: Use $< instead of pathsubst
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 2/7] s390x: MAKEFILE: Use $< instead of pathsubst Janosch Frank
@ 2022-10-20 9:14 ` Claudio Imbrenda
0 siblings, 0 replies; 17+ messages in thread
From: Claudio Imbrenda @ 2022-10-20 9:14 UTC (permalink / raw)
To: Janosch Frank; +Cc: kvm, seiden, nrb, scgl, thuth
On Thu, 20 Oct 2022 09:00:04 +0000
Janosch Frank <frankja@linux.ibm.com> wrote:
> No need to mangle strings if we already have the value at hand.
>
> Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
> Reviewed-by: Steffen Eiden <seiden@linux.ibm.com>
> Reviewed-by: Nico Boehr <nrb@linux.ibm.com>
Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
> ---
> s390x/Makefile | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/s390x/Makefile b/s390x/Makefile
> index 649486f2..3b175015 100644
> --- a/s390x/Makefile
> +++ b/s390x/Makefile
> @@ -136,7 +136,7 @@ $(SNIPPET_DIR)/asm/%.gbin: $(SNIPPET_DIR)/asm/%.o
> truncate -s '%4096' $@
>
> $(SNIPPET_DIR)/c/%.gbin: $(SNIPPET_DIR)/c/%.o $(snippet_lib) $(FLATLIBS)
> - $(CC) $(LDFLAGS) -o $@ -T $(SRCDIR)/s390x/snippets/c/flat.lds $(patsubst %.gbin,%.o,$@) $(snippet_lib) $(FLATLIBS)
> + $(CC) $(LDFLAGS) -o $@ -T $(SRCDIR)/s390x/snippets/c/flat.lds $< $(snippet_lib) $(FLATLIBS)
> $(OBJCOPY) -O binary -j ".rodata" -j ".lowcore" -j ".text" -j ".data" -j ".bss" --set-section-flags .bss=alloc,load,contents $@ $@
> truncate -s '%4096' $@
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [kvm-unit-tests PATCH v2 4/7] lib: s390x: sie: Improve validity handling and make it vm specific
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 4/7] lib: s390x: sie: Improve validity handling and make it vm specific Janosch Frank
@ 2022-10-20 9:18 ` Claudio Imbrenda
0 siblings, 0 replies; 17+ messages in thread
From: Claudio Imbrenda @ 2022-10-20 9:18 UTC (permalink / raw)
To: Janosch Frank; +Cc: kvm, seiden, nrb, scgl, thuth
On Thu, 20 Oct 2022 09:00:06 +0000
Janosch Frank <frankja@linux.ibm.com> wrote:
> The current library doesn't support running multiple vms at once as it
> stores the validity once and not per vm. Let's move the validity
> handling into the vm and introduce a new function to retrieve the vir.
>
> Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
> Reviewed-by: Steffen Eiden <seiden@linux.ibm.com>
> Reviewed-by: Nico Boehr <nrb@linux.ibm.com>
Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
> ---
> lib/s390x/sie.c | 27 ++++++++++++++-------------
> lib/s390x/sie.h | 6 ++++--
> 2 files changed, 18 insertions(+), 15 deletions(-)
>
> diff --git a/lib/s390x/sie.c b/lib/s390x/sie.c
> index 00aff713..3fee3def 100644
> --- a/lib/s390x/sie.c
> +++ b/lib/s390x/sie.c
> @@ -15,19 +15,22 @@
> #include <libcflat.h>
> #include <alloc_page.h>
>
> -static bool validity_expected;
> -static uint16_t vir; /* Validity interception reason */
> -
> -void sie_expect_validity(void)
> +void sie_expect_validity(struct vm *vm)
> {
> - validity_expected = true;
> - vir = 0;
> + vm->validity_expected = true;
> }
>
> -void sie_check_validity(uint16_t vir_exp)
> +uint16_t sie_get_validity(struct vm *vm)
> {
> + assert(vm->sblk->icptcode == ICPT_VALIDITY);
> + return vm->sblk->ipb >> 16;
> +}
> +
> +void sie_check_validity(struct vm *vm, uint16_t vir_exp)
> +{
> + uint16_t vir = sie_get_validity(vm);
> +
> report(vir_exp == vir, "VALIDITY: %x", vir);
> - vir = 0;
> }
>
> void sie_handle_validity(struct vm *vm)
> @@ -35,11 +38,9 @@ void sie_handle_validity(struct vm *vm)
> if (vm->sblk->icptcode != ICPT_VALIDITY)
> return;
>
> - vir = vm->sblk->ipb >> 16;
> -
> - if (!validity_expected)
> - report_abort("VALIDITY: %x", vir);
> - validity_expected = false;
> + if (!vm->validity_expected)
> + report_abort("VALIDITY: %x", sie_get_validity(vm));
> + vm->validity_expected = false;
> }
>
> void sie(struct vm *vm)
> diff --git a/lib/s390x/sie.h b/lib/s390x/sie.h
> index de91ea5a..320c4218 100644
> --- a/lib/s390x/sie.h
> +++ b/lib/s390x/sie.h
> @@ -233,14 +233,16 @@ struct vm {
> struct vm_uv uv; /* PV UV information */
> /* Ptr to first guest page */
> uint8_t *guest_mem;
> + bool validity_expected;
> };
>
> extern void sie_entry(void);
> extern void sie_exit(void);
> extern void sie64a(struct kvm_s390_sie_block *sblk, struct vm_save_area *save_area);
> void sie(struct vm *vm);
> -void sie_expect_validity(void);
> -void sie_check_validity(uint16_t vir_exp);
> +void sie_expect_validity(struct vm *vm);
> +uint16_t sie_get_validity(struct vm *vm);
> +void sie_check_validity(struct vm *vm, uint16_t vir_exp);
> void sie_handle_validity(struct vm *vm);
> void sie_guest_sca_create(struct vm *vm);
> void sie_guest_create(struct vm *vm, uint64_t guest_mem, uint64_t guest_mem_len);
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [kvm-unit-tests PATCH v2 5/7] lib: s390x: Use a new asce for each PV guest
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 5/7] lib: s390x: Use a new asce for each PV guest Janosch Frank
@ 2022-10-20 9:25 ` Claudio Imbrenda
2022-10-20 11:27 ` Janosch Frank
0 siblings, 1 reply; 17+ messages in thread
From: Claudio Imbrenda @ 2022-10-20 9:25 UTC (permalink / raw)
To: Janosch Frank; +Cc: kvm, seiden, nrb, scgl, thuth
On Thu, 20 Oct 2022 09:00:07 +0000
Janosch Frank <frankja@linux.ibm.com> wrote:
> Every PV guest needs its own ASCE so let's copy the topmost table
> designated by CR1 to create a new ASCE for the PV guest. Before and
> after SIE we now need to switch ASCEs to and from the PV guest / test
> ASCE. The SIE assembly function does that automatically.
>
> Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
> ---
> lib/s390x/asm-offsets.c | 2 ++
> lib/s390x/sie.c | 2 ++
> lib/s390x/sie.h | 2 ++
> lib/s390x/uv.c | 24 +++++++++++++++++++++++-
> lib/s390x/uv.h | 5 ++---
> s390x/cpu.S | 6 ++++++
> 6 files changed, 37 insertions(+), 4 deletions(-)
>
> diff --git a/lib/s390x/asm-offsets.c b/lib/s390x/asm-offsets.c
> index fbea3278..f612f327 100644
> --- a/lib/s390x/asm-offsets.c
> +++ b/lib/s390x/asm-offsets.c
> @@ -75,9 +75,11 @@ int main(void)
> OFFSET(SIE_SAVEAREA_HOST_GRS, vm_save_area, host.grs[0]);
> OFFSET(SIE_SAVEAREA_HOST_FPRS, vm_save_area, host.fprs[0]);
> OFFSET(SIE_SAVEAREA_HOST_FPC, vm_save_area, host.fpc);
> + OFFSET(SIE_SAVEAREA_HOST_ASCE, vm_save_area, host.asce);
> OFFSET(SIE_SAVEAREA_GUEST_GRS, vm_save_area, guest.grs[0]);
> OFFSET(SIE_SAVEAREA_GUEST_FPRS, vm_save_area, guest.fprs[0]);
> OFFSET(SIE_SAVEAREA_GUEST_FPC, vm_save_area, guest.fpc);
> + OFFSET(SIE_SAVEAREA_GUEST_ASCE, vm_save_area, guest.asce);
> OFFSET(STACK_FRAME_INT_BACKCHAIN, stack_frame_int, back_chain);
> OFFSET(STACK_FRAME_INT_FPC, stack_frame_int, fpc);
> OFFSET(STACK_FRAME_INT_FPRS, stack_frame_int, fprs);
> diff --git a/lib/s390x/sie.c b/lib/s390x/sie.c
> index 3fee3def..6efad965 100644
> --- a/lib/s390x/sie.c
> +++ b/lib/s390x/sie.c
> @@ -85,6 +85,8 @@ void sie_guest_create(struct vm *vm, uint64_t guest_mem, uint64_t guest_mem_len)
>
> /* Guest memory chunks are always 1MB */
> assert(!(guest_mem_len & ~HPAGE_MASK));
> + /* For non-PV guests we re-use the host's ASCE for ease of use */
> + vm->save_area.guest.asce = stctg(1);
> /* Currently MSO/MSL is the easiest option */
> vm->sblk->mso = (uint64_t)guest_mem;
> vm->sblk->msl = (uint64_t)guest_mem + ((guest_mem_len - 1) & HPAGE_MASK);
> diff --git a/lib/s390x/sie.h b/lib/s390x/sie.h
> index 320c4218..3e3605c9 100644
> --- a/lib/s390x/sie.h
> +++ b/lib/s390x/sie.h
> @@ -205,12 +205,14 @@ union {
> struct vm_uv {
> uint64_t vm_handle;
> uint64_t vcpu_handle;
> + uint64_t asce;
> void *conf_base_stor;
> void *conf_var_stor;
> void *cpu_stor;
> };
>
> struct vm_save_regs {
> + uint64_t asce;
> uint64_t grs[16];
> uint64_t fprs[16];
> uint32_t fpc;
> diff --git a/lib/s390x/uv.c b/lib/s390x/uv.c
> index 3b4cafa9..0b6eb843 100644
> --- a/lib/s390x/uv.c
> +++ b/lib/s390x/uv.c
> @@ -90,6 +90,25 @@ void uv_init(void)
> initialized = true;
> }
>
> +/*
> + * Create a new ASCE for the UV config because they can't be shared
> + * for security reasons. We just simply copy the top most table into a
> + * fresh set of allocated pages and use those pages as the asce.
> + */
> +static uint64_t create_asce(void)
> +{
> + void *pgd_new, *pgd_old;
> + uint64_t asce = stctg(1);
> +
> + pgd_new = memalign_pages_flags(PAGE_SIZE, PAGE_SIZE * 4, 0);
here you can use memalign_pages, since you are not using the flags
> + pgd_old = (void *)(asce & PAGE_MASK);
> +
> + memcpy(pgd_new, pgd_old, PAGE_SIZE * 4);
> +
> + asce = __pa(pgd_new) | ASCE_DT_REGION1 | REGION_TABLE_LENGTH | ASCE_P;
why not taking the flags from the old ASCE? what if we choose to use a
different type of table?
something like this:
asce = _pa(pgd_new) | ASCE_P | (asce & ~PAGE_MASK);
> + return asce;
> +}
> +
> void uv_create_guest(struct vm *vm)
> {
> struct uv_cb_cgc uvcb_cgc = {
> @@ -125,7 +144,8 @@ void uv_create_guest(struct vm *vm)
> vm->uv.cpu_stor = memalign_pages_flags(PAGE_SIZE, uvcb_qui.cpu_stor_len, 0);
> uvcb_csc.stor_origin = (uint64_t)vm->uv.cpu_stor;
>
> - uvcb_cgc.guest_asce = (uint64_t)stctg(1);
> + uvcb_cgc.guest_asce = create_asce();
> + vm->save_area.guest.asce = uvcb_cgc.guest_asce;
> uvcb_cgc.guest_sca = (uint64_t)vm->sca;
>
> cc = uv_call(0, (uint64_t)&uvcb_cgc);
> @@ -166,6 +186,8 @@ void uv_destroy_guest(struct vm *vm)
> assert(cc == 0);
> free_pages(vm->uv.conf_base_stor);
> free_pages(vm->uv.conf_var_stor);
> +
> + free_pages((void *)(vm->uv.asce & PAGE_MASK));
> }
>
> int uv_unpack(struct vm *vm, uint64_t addr, uint64_t len, uint64_t tweak)
> diff --git a/lib/s390x/uv.h b/lib/s390x/uv.h
> index 44264861..5fe29bda 100644
> --- a/lib/s390x/uv.h
> +++ b/lib/s390x/uv.h
> @@ -28,9 +28,8 @@ static inline void uv_setup_asces(void)
> /* We need to have a valid primary ASCE to run guests. */
> setup_vm();
>
> - /* Set P bit in ASCE as it is required for PV guests */
> - asce = stctg(1) | ASCE_P;
> - lctlg(1, asce);
> + /* Grab the ASCE which setup_vm() just set up */
> + asce = stctg(1);
>
> /* Copy ASCE into home space CR */
> lctlg(13, asce);
> diff --git a/s390x/cpu.S b/s390x/cpu.S
> index 82b5e25d..45bd551a 100644
> --- a/s390x/cpu.S
> +++ b/s390x/cpu.S
> @@ -76,6 +76,9 @@ sie64a:
> .endr
> stfpc SIE_SAVEAREA_HOST_FPC(%r3)
>
> + stctg %c1, %c1, SIE_SAVEAREA_HOST_ASCE(%r3)
> + lctlg %c1, %c1, SIE_SAVEAREA_GUEST_ASCE(%r3)
> +
> # Store scb and save_area pointer into stack frame
> stg %r2,__SF_SIE_CONTROL(%r15) # save control block pointer
> stg %r3,__SF_SIE_SAVEAREA(%r15) # save guest register save area
> @@ -102,6 +105,9 @@ sie_exit:
> # Load guest register save area
> lg %r14,__SF_SIE_SAVEAREA(%r15)
>
> + # Restore the host asce
> + lctlg %c1, %c1, SIE_SAVEAREA_HOST_ASCE(%r14)
> +
> # Store guest's gprs, fprs and fpc
> stmg %r0,%r13,SIE_SAVEAREA_GUEST_GRS(%r14) # save guest gprs 0-13
> .irp i, 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [kvm-unit-tests PATCH v2 6/7] lib: s390x: Enable reusability of VMs that were in PV mode
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 6/7] lib: s390x: Enable reusability of VMs that were in PV mode Janosch Frank
@ 2022-10-20 9:26 ` Claudio Imbrenda
0 siblings, 0 replies; 17+ messages in thread
From: Claudio Imbrenda @ 2022-10-20 9:26 UTC (permalink / raw)
To: Janosch Frank; +Cc: kvm, seiden, nrb, scgl, thuth
On Thu, 20 Oct 2022 09:00:08 +0000
Janosch Frank <frankja@linux.ibm.com> wrote:
> Convert the sblk to non-PV when the PV guest is destroyed.
>
> Early return in uv_init() instead of running into the assert. This is
> necessary since snippet_pv_init() will always call uv_init().
>
> Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
> ---
> lib/s390x/uv.c | 11 ++++++++++-
> 1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/lib/s390x/uv.c b/lib/s390x/uv.c
> index 0b6eb843..99775929 100644
> --- a/lib/s390x/uv.c
> +++ b/lib/s390x/uv.c
> @@ -76,7 +76,8 @@ void uv_init(void)
> int cc;
>
> /* Let's not do this twice */
> - assert(!initialized);
> + if (initialized)
> + return;
> /* Query is done on initialization but let's check anyway */
> assert(uvcb_qui.header.rc == 1 || uvcb_qui.header.rc == 0x100);
>
> @@ -188,6 +189,14 @@ void uv_destroy_guest(struct vm *vm)
> free_pages(vm->uv.conf_var_stor);
>
> free_pages((void *)(vm->uv.asce & PAGE_MASK));
> + memset(&vm->uv, 0, sizeof(vm->uv));
> +
> + /* Convert the sblk back to non-PV */
> + vm->save_area.guest.asce = stctg(1);
> + vm->sblk->sdf = 0;
> + vm->sblk->sidad = 0;
> + vm->sblk->pv_handle_cpu = 0;
> + vm->sblk->pv_handle_config = 0;
> }
>
> int uv_unpack(struct vm *vm, uint64_t addr, uint64_t len, uint64_t tweak)
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [kvm-unit-tests PATCH v2 7/7] lib: s390x: sie: Properly populate SCA
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 7/7] lib: s390x: sie: Properly populate SCA Janosch Frank
@ 2022-10-20 9:50 ` Claudio Imbrenda
0 siblings, 0 replies; 17+ messages in thread
From: Claudio Imbrenda @ 2022-10-20 9:50 UTC (permalink / raw)
To: Janosch Frank; +Cc: kvm, seiden, nrb, scgl, thuth
On Thu, 20 Oct 2022 09:00:09 +0000
Janosch Frank <frankja@linux.ibm.com> wrote:
> CPU0 is the only cpu that's being used but we should still mark it as
> online and set the SDA in the SCA.
>
> Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
> Reviewed-by: Nico Boehr <nrb@linux.ibm.com>
Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
> ---
> lib/s390x/sie.c | 8 ++++++++
> lib/s390x/sie.h | 35 ++++++++++++++++++++++++++++++++++-
> 2 files changed, 42 insertions(+), 1 deletion(-)
>
> diff --git a/lib/s390x/sie.c b/lib/s390x/sie.c
> index 6efad965..a71985b6 100644
> --- a/lib/s390x/sie.c
> +++ b/lib/s390x/sie.c
> @@ -9,6 +9,7 @@
> */
>
> #include <asm/barrier.h>
> +#include <bitops.h>
> #include <libcflat.h>
> #include <sie.h>
> #include <asm/page.h>
> @@ -72,6 +73,13 @@ void sie_guest_sca_create(struct vm *vm)
> vm->sblk->scaoh = ((uint64_t)vm->sca >> 32);
> vm->sblk->scaol = (uint64_t)vm->sca & ~0x3fU;
> vm->sblk->ecb2 |= ECB2_ESCA;
> +
> + /* Enable SIGP sense running interpretation */
> + vm->sblk->ecb |= ECB_SRSI;
> +
> + /* We assume that cpu 0 is always part of the vm */
> + vm->sca->mcn[0] = BIT(63);
> + vm->sca->cpu[0].sda = (uint64_t)vm->sblk;
> }
>
> /* Initializes the struct vm members like the SIE control block. */
> diff --git a/lib/s390x/sie.h b/lib/s390x/sie.h
> index 3e3605c9..a27a8401 100644
> --- a/lib/s390x/sie.h
> +++ b/lib/s390x/sie.h
> @@ -202,6 +202,39 @@ union {
> uint64_t pv_grregs[16]; /* 0x0380 */
> } __attribute__((packed));
>
> +union esca_sigp_ctrl {
> + uint16_t value;
> + struct {
> + uint8_t c : 1;
> + uint8_t reserved: 7;
> + uint8_t scn;
> + };
> +};
> +
> +struct esca_entry {
> + union esca_sigp_ctrl sigp_ctrl;
> + uint16_t reserved1[3];
> + uint64_t sda;
> + uint64_t reserved2[6];
> +};
> +
> +union ipte_control {
> + unsigned long val;
> + struct {
> + unsigned long k : 1;
> + unsigned long kh : 31;
> + unsigned long kg : 32;
> + };
> +};
> +
> +struct esca_block {
> + union ipte_control ipte_control;
> + uint64_t reserved1[7];
> + uint64_t mcn[4];
> + uint64_t reserved2[20];
> + struct esca_entry cpu[256];
> +};
> +
> struct vm_uv {
> uint64_t vm_handle;
> uint64_t vcpu_handle;
> @@ -230,7 +263,7 @@ struct vm_save_area {
> struct vm {
> struct kvm_s390_sie_block *sblk;
> struct vm_save_area save_area;
> - void *sca; /* System Control Area */
> + struct esca_block *sca; /* System Control Area */
> uint8_t *crycb; /* Crypto Control Block */
> struct vm_uv uv; /* PV UV information */
> /* Ptr to first guest page */
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [kvm-unit-tests PATCH v2 3/7] s390x: Add a linker script to assembly snippets
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 3/7] s390x: Add a linker script to assembly snippets Janosch Frank
@ 2022-10-20 10:02 ` Claudio Imbrenda
0 siblings, 0 replies; 17+ messages in thread
From: Claudio Imbrenda @ 2022-10-20 10:02 UTC (permalink / raw)
To: Janosch Frank; +Cc: kvm, seiden, nrb, scgl, thuth
On Thu, 20 Oct 2022 09:00:05 +0000
Janosch Frank <frankja@linux.ibm.com> wrote:
> A linker script has a few benefits:
> - We can easily define a lowcore and load the snippet from 0x0 instead
> of 0x4000 which makes asm snippets behave like c snippets
> - We can easily define an invalid PGM new PSW to ensure an exit on a
> guest PGM
> - We can remove a lot of the offset variables from lib/s390x/snippet.h
>
> As we gain another step and file extension by linking, a comment
> explains which file extensions are generated and why.
>
> Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
> ---
> lib/s390x/snippet.h | 14 +++++---------
> s390x/Makefile | 18 ++++++++++++++----
> s390x/mvpg-sie.c | 2 +-
> s390x/pv-diags.c | 6 +++---
> s390x/snippets/asm/flat.lds | 35 +++++++++++++++++++++++++++++++++++
> 5 files changed, 58 insertions(+), 17 deletions(-)
> create mode 100644 s390x/snippets/asm/flat.lds
>
> diff --git a/lib/s390x/snippet.h b/lib/s390x/snippet.h
> index b17b2a4c..fcd04081 100644
> --- a/lib/s390x/snippet.h
> +++ b/lib/s390x/snippet.h
> @@ -32,8 +32,6 @@
>
> #define SNIPPET_PV_TWEAK0 0x42UL
> #define SNIPPET_PV_TWEAK1 0UL
> -#define SNIPPET_OFF_C 0
> -#define SNIPPET_OFF_ASM 0x4000
>
>
> /*
> @@ -57,15 +55,14 @@ static const struct psw snippet_psw = {
> * @vm: VM that this function will populated, has to be initialized already
> * @gbin: Snippet gbin data pointer
> * @gbin_len: Length of the gbin data
> - * @off: Offset from guest absolute 0x0 where snippet is copied to
> */
> static inline void snippet_init(struct vm *vm, const char *gbin,
> - uint64_t gbin_len, uint64_t off)
> + uint64_t gbin_len)
> {
> uint64_t mso = vm->sblk->mso;
>
> /* Copy test image to guest memory */
> - memcpy((void *)mso + off, gbin, gbin_len);
> + memcpy((void *)mso, gbin, gbin_len);
>
> /* Setup guest PSW */
> vm->sblk->gpsw = snippet_psw;
> @@ -87,23 +84,22 @@ static inline void snippet_init(struct vm *vm, const char *gbin,
> * @hdr: Snippet SE header data pointer
> * @gbin_len: Length of the gbin data
> * @hdr_len: Length of the hdr data
> - * @off: Offset from guest absolute 0x0 where snippet is copied to
> */
> static inline void snippet_pv_init(struct vm *vm, const char *gbin,
> const char *hdr, uint64_t gbin_len,
> - uint64_t hdr_len, uint64_t off)
> + uint64_t hdr_len)
> {
> uint64_t tweak[2] = {SNIPPET_PV_TWEAK0, SNIPPET_PV_TWEAK1};
> uint64_t mso = vm->sblk->mso;
> int i;
>
> - snippet_init(vm, gbin, gbin_len, off);
> + snippet_init(vm, gbin, gbin_len);
>
> uv_create_guest(vm);
> uv_set_se_hdr(vm->uv.vm_handle, (void *)hdr, hdr_len);
>
> /* Unpack works on guest addresses so we only need off */
> - uv_unpack(vm, off, gbin_len, tweak[0]);
> + uv_unpack(vm, 0, gbin_len, tweak[0]);
> uv_verify_load(vm);
>
> /*
> diff --git a/s390x/Makefile b/s390x/Makefile
> index 3b175015..0eaa72f4 100644
> --- a/s390x/Makefile
> +++ b/s390x/Makefile
> @@ -124,6 +124,13 @@ else
> snippet-hdr-obj =
> endif
>
> +# Each snippet will generate the following files (in order): \
> + *.o is a snippet that has been compiled \
> + *.ol is a snippet that has been linked \
> + *.gbin is a snippet that has been converted to binary \
> + *.gobj is the final format after converting the binary into a elf file again, \
> + it will be linked into the tests
> +
> # the asm/c snippets %.o have additional generated files as dependencies
> $(SNIPPET_DIR)/asm/%.o: $(SNIPPET_DIR)/asm/%.S $(asm-offsets)
> $(CC) $(CFLAGS) -c -nostdlib -o $@ $<
> @@ -131,17 +138,20 @@ $(SNIPPET_DIR)/asm/%.o: $(SNIPPET_DIR)/asm/%.S $(asm-offsets)
> $(SNIPPET_DIR)/c/%.o: $(SNIPPET_DIR)/c/%.c $(asm-offsets)
> $(CC) $(CFLAGS) -c -nostdlib -o $@ $<
>
> -$(SNIPPET_DIR)/asm/%.gbin: $(SNIPPET_DIR)/asm/%.o
> - $(OBJCOPY) -O binary -j ".rodata" -j ".text" -j ".data" -j ".bss" --set-section-flags .bss=alloc,load,contents $(patsubst %.gbin,%.o,$@) $@
> +$(SNIPPET_DIR)/asm/%.ol: $(SNIPPET_DIR)/asm/%.o
> + $(CC) $(LDFLAGS) -o $@ -T $(SNIPPET_DIR)/asm/flat.lds $<
> +
> +$(SNIPPET_DIR)/asm/%.gbin: $(SNIPPET_DIR)/asm/%.ol
> + $(OBJCOPY) -O binary -j ".rodata" -j ".lowcore" -j ".text" -j ".data" -j ".bss" --set-section-flags .bss=alloc,load,contents $< $@
> truncate -s '%4096' $@
can't you do $(CC) and $(OBJCOPY) in one step like you do for C
snippets? then you don't need the .ol intermediate
>
> $(SNIPPET_DIR)/c/%.gbin: $(SNIPPET_DIR)/c/%.o $(snippet_lib) $(FLATLIBS)
> - $(CC) $(LDFLAGS) -o $@ -T $(SRCDIR)/s390x/snippets/c/flat.lds $< $(snippet_lib) $(FLATLIBS)
> + $(CC) $(LDFLAGS) -o $@ -T $(SNIPPET_DIR)/c/flat.lds $< $(snippet_lib) $(FLATLIBS)
> $(OBJCOPY) -O binary -j ".rodata" -j ".lowcore" -j ".text" -j ".data" -j ".bss" --set-section-flags .bss=alloc,load,contents $@ $@
> truncate -s '%4096' $@
>
> $(SNIPPET_DIR)/asm/%.hdr: $(SNIPPET_DIR)/asm/%.gbin $(HOST_KEY_DOCUMENT)
> - $(GEN_SE_HEADER) -k $(HOST_KEY_DOCUMENT) -c $<,0x4000,0x00000000000000420000000000000000 --psw-addr 0x4000 -o $@
> + $(GEN_SE_HEADER) -k $(HOST_KEY_DOCUMENT) -c $<,0x0,0x00000000000000420000000000000000 --psw-addr 0x4000 -o $@
>
> $(SNIPPET_DIR)/c/%.hdr: $(SNIPPET_DIR)/c/%.gbin $(HOST_KEY_DOCUMENT)
> $(GEN_SE_HEADER) -k $(HOST_KEY_DOCUMENT) -c $<,0x0,0x00000000000000420000000000000000 --psw-addr 0x4000 -o $@
> diff --git a/s390x/mvpg-sie.c b/s390x/mvpg-sie.c
> index 46a2edb6..17e209ad 100644
> --- a/s390x/mvpg-sie.c
> +++ b/s390x/mvpg-sie.c
> @@ -87,7 +87,7 @@ static void setup_guest(void)
>
> snippet_setup_guest(&vm, false);
> snippet_init(&vm, SNIPPET_NAME_START(c, mvpg_snippet),
> - SNIPPET_LEN(c, mvpg_snippet), SNIPPET_OFF_C);
> + SNIPPET_LEN(c, mvpg_snippet));
>
> /* Enable MVPG interpretation as we want to test KVM and not ourselves */
> vm.sblk->eca = ECA_MVPGI;
> diff --git a/s390x/pv-diags.c b/s390x/pv-diags.c
> index 9ced68c7..d472c994 100644
> --- a/s390x/pv-diags.c
> +++ b/s390x/pv-diags.c
> @@ -28,7 +28,7 @@ static void test_diag_500(void)
>
> snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_diag_500),
> SNIPPET_HDR_START(asm, snippet_pv_diag_500),
> - size_gbin, size_hdr, SNIPPET_OFF_ASM);
> + size_gbin, size_hdr);
>
> sie(&vm);
> report(vm.sblk->icptcode == ICPT_PV_INSTR && vm.sblk->ipa == 0x8302 &&
> @@ -83,7 +83,7 @@ static void test_diag_288(void)
>
> snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_diag_288),
> SNIPPET_HDR_START(asm, snippet_pv_diag_288),
> - size_gbin, size_hdr, SNIPPET_OFF_ASM);
> + size_gbin, size_hdr);
>
> sie(&vm);
> report(vm.sblk->icptcode == ICPT_PV_INSTR && vm.sblk->ipa == 0x8302 &&
> @@ -124,7 +124,7 @@ static void test_diag_yield(void)
>
> snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_diag_yield),
> SNIPPET_HDR_START(asm, snippet_pv_diag_yield),
> - size_gbin, size_hdr, SNIPPET_OFF_ASM);
> + size_gbin, size_hdr);
>
> /* 0x44 */
> report_prefix_push("0x44");
> diff --git a/s390x/snippets/asm/flat.lds b/s390x/snippets/asm/flat.lds
> new file mode 100644
> index 00000000..388d7d5d
> --- /dev/null
> +++ b/s390x/snippets/asm/flat.lds
> @@ -0,0 +1,35 @@
> +SECTIONS
> +{
> + .lowcore : {
> + /* Restart new PSW for booting via PSW restart. */
> + . = 0x1a0;
> + QUAD(0x0000000180000000)
> + QUAD(0x0000000000004000)
> + /*
> + * Invalid PGM new PSW so we hopefully get a code 8
> + * intercept on a PGM for PV snippets.
> + */
> + . = 0x1d0;
> + QUAD(0x0008000000000000)
> + QUAD(0x0000000000000001)
> + }
> + . = 0x4000;
> + .text : {
> + *(.text)
> + *(.text.*)
> + }
> + . = ALIGN(64K);
> + etext = .;
> + . = ALIGN(16);
> + .data : {
> + *(.data)
> + *(.data.rel*)
> + }
> + . = ALIGN(16);
> + .rodata : { *(.rodata) *(.rodata.*) }
> + . = ALIGN(16);
do you really need to ALIGN(16) right after ALIGN(64K) ?
> + __bss_start = .;
> + .bss : { *(.bss) }
> + __bss_end = .;
> + . = ALIGN(64K);
why align the end? and why 64K instead of e.g. 4K?
> +}
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [kvm-unit-tests PATCH v2 5/7] lib: s390x: Use a new asce for each PV guest
2022-10-20 9:25 ` Claudio Imbrenda
@ 2022-10-20 11:27 ` Janosch Frank
2022-10-20 11:46 ` Claudio Imbrenda
0 siblings, 1 reply; 17+ messages in thread
From: Janosch Frank @ 2022-10-20 11:27 UTC (permalink / raw)
To: Claudio Imbrenda; +Cc: kvm, seiden, nrb, scgl, thuth
On 10/20/22 11:25, Claudio Imbrenda wrote:
> On Thu, 20 Oct 2022 09:00:07 +0000
> Janosch Frank <frankja@linux.ibm.com> wrote:
>
>> Every PV guest needs its own ASCE so let's copy the topmost table
>> designated by CR1 to create a new ASCE for the PV guest. Before and
>> after SIE we now need to switch ASCEs to and from the PV guest / test
>> ASCE. The SIE assembly function does that automatically.
>>
>> Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
>> ---
>> lib/s390x/asm-offsets.c | 2 ++
>> lib/s390x/sie.c | 2 ++
>> lib/s390x/sie.h | 2 ++
>> lib/s390x/uv.c | 24 +++++++++++++++++++++++-
>> lib/s390x/uv.h | 5 ++---
>> s390x/cpu.S | 6 ++++++
>> 6 files changed, 37 insertions(+), 4 deletions(-)
>>
>> diff --git a/lib/s390x/asm-offsets.c b/lib/s390x/asm-offsets.c
>> index fbea3278..f612f327 100644
>> --- a/lib/s390x/asm-offsets.c
>> +++ b/lib/s390x/asm-offsets.c
>> @@ -75,9 +75,11 @@ int main(void)
>> OFFSET(SIE_SAVEAREA_HOST_GRS, vm_save_area, host.grs[0]);
>> OFFSET(SIE_SAVEAREA_HOST_FPRS, vm_save_area, host.fprs[0]);
>> OFFSET(SIE_SAVEAREA_HOST_FPC, vm_save_area, host.fpc);
>> + OFFSET(SIE_SAVEAREA_HOST_ASCE, vm_save_area, host.asce);
>> OFFSET(SIE_SAVEAREA_GUEST_GRS, vm_save_area, guest.grs[0]);
>> OFFSET(SIE_SAVEAREA_GUEST_FPRS, vm_save_area, guest.fprs[0]);
>> OFFSET(SIE_SAVEAREA_GUEST_FPC, vm_save_area, guest.fpc);
>> + OFFSET(SIE_SAVEAREA_GUEST_ASCE, vm_save_area, guest.asce);
>> OFFSET(STACK_FRAME_INT_BACKCHAIN, stack_frame_int, back_chain);
>> OFFSET(STACK_FRAME_INT_FPC, stack_frame_int, fpc);
>> OFFSET(STACK_FRAME_INT_FPRS, stack_frame_int, fprs);
>> diff --git a/lib/s390x/sie.c b/lib/s390x/sie.c
>> index 3fee3def..6efad965 100644
>> --- a/lib/s390x/sie.c
>> +++ b/lib/s390x/sie.c
>> @@ -85,6 +85,8 @@ void sie_guest_create(struct vm *vm, uint64_t guest_mem, uint64_t guest_mem_len)
>>
>> /* Guest memory chunks are always 1MB */
>> assert(!(guest_mem_len & ~HPAGE_MASK));
>> + /* For non-PV guests we re-use the host's ASCE for ease of use */
>> + vm->save_area.guest.asce = stctg(1);
>> /* Currently MSO/MSL is the easiest option */
>> vm->sblk->mso = (uint64_t)guest_mem;
>> vm->sblk->msl = (uint64_t)guest_mem + ((guest_mem_len - 1) & HPAGE_MASK);
>> diff --git a/lib/s390x/sie.h b/lib/s390x/sie.h
>> index 320c4218..3e3605c9 100644
>> --- a/lib/s390x/sie.h
>> +++ b/lib/s390x/sie.h
>> @@ -205,12 +205,14 @@ union {
>> struct vm_uv {
>> uint64_t vm_handle;
>> uint64_t vcpu_handle;
>> + uint64_t asce;
>> void *conf_base_stor;
>> void *conf_var_stor;
>> void *cpu_stor;
>> };
>>
>> struct vm_save_regs {
>> + uint64_t asce;
>> uint64_t grs[16];
>> uint64_t fprs[16];
>> uint32_t fpc;
>> diff --git a/lib/s390x/uv.c b/lib/s390x/uv.c
>> index 3b4cafa9..0b6eb843 100644
>> --- a/lib/s390x/uv.c
>> +++ b/lib/s390x/uv.c
>> @@ -90,6 +90,25 @@ void uv_init(void)
>> initialized = true;
>> }
>>
>> +/*
>> + * Create a new ASCE for the UV config because they can't be shared
>> + * for security reasons. We just simply copy the top most table into a
>> + * fresh set of allocated pages and use those pages as the asce.
>> + */
>> +static uint64_t create_asce(void)
>> +{
>> + void *pgd_new, *pgd_old;
>> + uint64_t asce = stctg(1);
>> +
>> + pgd_new = memalign_pages_flags(PAGE_SIZE, PAGE_SIZE * 4, 0);
>
> here you can use memalign_pages, since you are not using the flags
Sure
>
>> + pgd_old = (void *)(asce & PAGE_MASK);
>> +
>> + memcpy(pgd_new, pgd_old, PAGE_SIZE * 4);
>> +
>> + asce = __pa(pgd_new) | ASCE_DT_REGION1 | REGION_TABLE_LENGTH | ASCE_P;
>
> why not taking the flags from the old ASCE? what if we choose to use a
> different type of table?
>
> something like this:
>
> asce = _pa(pgd_new) | ASCE_P | (asce & ~PAGE_MASK);
I should at least preserve DT and TL but I'd opt to not copy over the
other bits. If someone wants to do funky ASCE stuff they now have the
possibility to simply change vm->save_area.guest.asce
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [kvm-unit-tests PATCH v2 5/7] lib: s390x: Use a new asce for each PV guest
2022-10-20 11:27 ` Janosch Frank
@ 2022-10-20 11:46 ` Claudio Imbrenda
0 siblings, 0 replies; 17+ messages in thread
From: Claudio Imbrenda @ 2022-10-20 11:46 UTC (permalink / raw)
To: Janosch Frank; +Cc: kvm, seiden, nrb, scgl, thuth
On Thu, 20 Oct 2022 13:27:22 +0200
Janosch Frank <frankja@linux.ibm.com> wrote:
> On 10/20/22 11:25, Claudio Imbrenda wrote:
> > On Thu, 20 Oct 2022 09:00:07 +0000
> > Janosch Frank <frankja@linux.ibm.com> wrote:
> >
> >> Every PV guest needs its own ASCE so let's copy the topmost table
> >> designated by CR1 to create a new ASCE for the PV guest. Before and
> >> after SIE we now need to switch ASCEs to and from the PV guest / test
> >> ASCE. The SIE assembly function does that automatically.
> >>
> >> Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
> >> ---
> >> lib/s390x/asm-offsets.c | 2 ++
> >> lib/s390x/sie.c | 2 ++
> >> lib/s390x/sie.h | 2 ++
> >> lib/s390x/uv.c | 24 +++++++++++++++++++++++-
> >> lib/s390x/uv.h | 5 ++---
> >> s390x/cpu.S | 6 ++++++
> >> 6 files changed, 37 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/lib/s390x/asm-offsets.c b/lib/s390x/asm-offsets.c
> >> index fbea3278..f612f327 100644
> >> --- a/lib/s390x/asm-offsets.c
> >> +++ b/lib/s390x/asm-offsets.c
> >> @@ -75,9 +75,11 @@ int main(void)
> >> OFFSET(SIE_SAVEAREA_HOST_GRS, vm_save_area, host.grs[0]);
> >> OFFSET(SIE_SAVEAREA_HOST_FPRS, vm_save_area, host.fprs[0]);
> >> OFFSET(SIE_SAVEAREA_HOST_FPC, vm_save_area, host.fpc);
> >> + OFFSET(SIE_SAVEAREA_HOST_ASCE, vm_save_area, host.asce);
> >> OFFSET(SIE_SAVEAREA_GUEST_GRS, vm_save_area, guest.grs[0]);
> >> OFFSET(SIE_SAVEAREA_GUEST_FPRS, vm_save_area, guest.fprs[0]);
> >> OFFSET(SIE_SAVEAREA_GUEST_FPC, vm_save_area, guest.fpc);
> >> + OFFSET(SIE_SAVEAREA_GUEST_ASCE, vm_save_area, guest.asce);
> >> OFFSET(STACK_FRAME_INT_BACKCHAIN, stack_frame_int, back_chain);
> >> OFFSET(STACK_FRAME_INT_FPC, stack_frame_int, fpc);
> >> OFFSET(STACK_FRAME_INT_FPRS, stack_frame_int, fprs);
> >> diff --git a/lib/s390x/sie.c b/lib/s390x/sie.c
> >> index 3fee3def..6efad965 100644
> >> --- a/lib/s390x/sie.c
> >> +++ b/lib/s390x/sie.c
> >> @@ -85,6 +85,8 @@ void sie_guest_create(struct vm *vm, uint64_t guest_mem, uint64_t guest_mem_len)
> >>
> >> /* Guest memory chunks are always 1MB */
> >> assert(!(guest_mem_len & ~HPAGE_MASK));
> >> + /* For non-PV guests we re-use the host's ASCE for ease of use */
> >> + vm->save_area.guest.asce = stctg(1);
> >> /* Currently MSO/MSL is the easiest option */
> >> vm->sblk->mso = (uint64_t)guest_mem;
> >> vm->sblk->msl = (uint64_t)guest_mem + ((guest_mem_len - 1) & HPAGE_MASK);
> >> diff --git a/lib/s390x/sie.h b/lib/s390x/sie.h
> >> index 320c4218..3e3605c9 100644
> >> --- a/lib/s390x/sie.h
> >> +++ b/lib/s390x/sie.h
> >> @@ -205,12 +205,14 @@ union {
> >> struct vm_uv {
> >> uint64_t vm_handle;
> >> uint64_t vcpu_handle;
> >> + uint64_t asce;
> >> void *conf_base_stor;
> >> void *conf_var_stor;
> >> void *cpu_stor;
> >> };
> >>
> >> struct vm_save_regs {
> >> + uint64_t asce;
> >> uint64_t grs[16];
> >> uint64_t fprs[16];
> >> uint32_t fpc;
> >> diff --git a/lib/s390x/uv.c b/lib/s390x/uv.c
> >> index 3b4cafa9..0b6eb843 100644
> >> --- a/lib/s390x/uv.c
> >> +++ b/lib/s390x/uv.c
> >> @@ -90,6 +90,25 @@ void uv_init(void)
> >> initialized = true;
> >> }
> >>
> >> +/*
> >> + * Create a new ASCE for the UV config because they can't be shared
> >> + * for security reasons. We just simply copy the top most table into a
> >> + * fresh set of allocated pages and use those pages as the asce.
> >> + */
> >> +static uint64_t create_asce(void)
> >> +{
> >> + void *pgd_new, *pgd_old;
> >> + uint64_t asce = stctg(1);
> >> +
> >> + pgd_new = memalign_pages_flags(PAGE_SIZE, PAGE_SIZE * 4, 0);
> >
> > here you can use memalign_pages, since you are not using the flags
>
> Sure
>
> >
> >> + pgd_old = (void *)(asce & PAGE_MASK);
> >> +
> >> + memcpy(pgd_new, pgd_old, PAGE_SIZE * 4);
> >> +
> >> + asce = __pa(pgd_new) | ASCE_DT_REGION1 | REGION_TABLE_LENGTH | ASCE_P;
> >
> > why not taking the flags from the old ASCE? what if we choose to use a
> > different type of table?
> >
> > something like this:
> >
> > asce = _pa(pgd_new) | ASCE_P | (asce & ~PAGE_MASK);
>
> I should at least preserve DT and TL but I'd opt to not copy over the
> other bits. If someone wants to do funky ASCE stuff they now have the
> possibility to simply change vm->save_area.guest.asce
that's ok
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2022-10-20 11:47 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-10-20 9:00 [kvm-unit-tests PATCH v2 0/7] s390x: PV fixups Janosch Frank
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 1/7] s390x: snippets: asm: Add a macro to write an exception PSW Janosch Frank
2022-10-20 9:14 ` Claudio Imbrenda
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 2/7] s390x: MAKEFILE: Use $< instead of pathsubst Janosch Frank
2022-10-20 9:14 ` Claudio Imbrenda
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 3/7] s390x: Add a linker script to assembly snippets Janosch Frank
2022-10-20 10:02 ` Claudio Imbrenda
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 4/7] lib: s390x: sie: Improve validity handling and make it vm specific Janosch Frank
2022-10-20 9:18 ` Claudio Imbrenda
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 5/7] lib: s390x: Use a new asce for each PV guest Janosch Frank
2022-10-20 9:25 ` Claudio Imbrenda
2022-10-20 11:27 ` Janosch Frank
2022-10-20 11:46 ` Claudio Imbrenda
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 6/7] lib: s390x: Enable reusability of VMs that were in PV mode Janosch Frank
2022-10-20 9:26 ` Claudio Imbrenda
2022-10-20 9:00 ` [kvm-unit-tests PATCH v2 7/7] lib: s390x: sie: Properly populate SCA Janosch Frank
2022-10-20 9:50 ` Claudio Imbrenda
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox