* [PATCH v1 00/10] KVM: s390: Misc bugfixes and improvements for 7.4
@ 2026-10-22 15:55 Claudio Imbrenda
2026-10-22 15:55 ` [PATCH v1 01/10] KVM: s390: Move some skey functions to headers Claudio Imbrenda
` (9 more replies)
0 siblings, 10 replies; 21+ messages in thread
From: Claudio Imbrenda @ 2026-10-22 15:55 UTC (permalink / raw)
To: kvm; +Cc: linux-s390, borntraeger, frankja, david, nrb, schlameuss, gra
Misc bugfixes and improvements.
Some are preparations for future patches, some are preventive fixes for
mostly theoretical issues, some are quality of life improvement (mostly
for the maintainers).
Claudio Imbrenda (10):
KVM: s390: Move some skey functions to headers
KVM: s390: Refactor and improve DAT tables handling
KVM: s390: Move prefix marker from PGSTEs to PTEs
KVM: s390: Automatically enable storage keys for UCONTROL VMs
KVM: s390: Replace ASCE usage with a struct with more flags
KVM: s390: Rework some helper functions
KVM: s390: Fix UCONTROL inconsistencies
KVM: s390: Improve unmapping of large ranges
KVM: s390: vsie: Fix pinning using the wrong flag
KVM: s390: Enable 1M pages by default
arch/s390/kvm/gmap/dat.c | 195 +++++++++++-------------
arch/s390/kvm/gmap/dat.h | 196 +++++++++++++++++-------
arch/s390/kvm/gmap/gmap.c | 281 +++++++++++++++++++++++------------
arch/s390/kvm/gmap/gmap.h | 38 +++--
arch/s390/kvm/gmap/kvm_mmu.c | 6 +-
arch/s390/kvm/s390/gaccess.c | 27 ++--
arch/s390/kvm/s390/priv.c | 15 +-
arch/s390/kvm/s390/pv.c | 11 +-
arch/s390/kvm/s390/s390.c | 46 +++---
arch/s390/kvm/s390/s390.h | 6 +-
arch/s390/kvm/s390/vsie.c | 135 +++++++----------
11 files changed, 539 insertions(+), 417 deletions(-)
--
2.56.0
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v1 01/10] KVM: s390: Move some skey functions to headers
2026-10-22 15:55 [PATCH v1 00/10] KVM: s390: Misc bugfixes and improvements for 7.4 Claudio Imbrenda
@ 2026-10-22 15:55 ` Claudio Imbrenda
2026-10-22 15:58 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 02/10] KVM: s390: Refactor and improve DAT tables handling Claudio Imbrenda
` (8 subsequent siblings)
9 siblings, 1 reply; 21+ messages in thread
From: Claudio Imbrenda @ 2026-10-22 15:55 UTC (permalink / raw)
To: kvm; +Cc: linux-s390, borntraeger, frankja, david, nrb, schlameuss, gra
Move some storage key handling functions to headers, since they will be
needed from outside, and they are small, so inlining is acceptable.
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
---
arch/s390/kvm/gmap/dat.c | 26 --------------------------
arch/s390/kvm/gmap/dat.h | 26 ++++++++++++++++++++++++++
2 files changed, 26 insertions(+), 26 deletions(-)
diff --git a/arch/s390/kvm/gmap/dat.c b/arch/s390/kvm/gmap/dat.c
index ff80d02c9f56..b9507daaf327 100644
--- a/arch/s390/kvm/gmap/dat.c
+++ b/arch/s390/kvm/gmap/dat.c
@@ -159,32 +159,6 @@ bool __must_check dat_crstep_xchg_atomic(union crste *crstep, union crste old, u
return cspg_crste(crstep, old, new);
}
-static void dat_set_storage_key_from_pgste(union pte pte, union pgste pgste)
-{
- union skey nkey = { .acc = pgste.acc, .fp = pgste.fp };
-
- page_set_storage_key(pte_origin(pte), nkey.skey, 0);
-}
-
-static void dat_move_storage_key(union pte old, union pte new)
-{
- page_set_storage_key(pte_origin(new), page_get_storage_key(pte_origin(old)), 1);
-}
-
-static union pgste dat_save_storage_key_into_pgste(union pte pte, union pgste pgste)
-{
- union skey skey;
-
- skey.skey = page_get_storage_key(pte_origin(pte));
-
- pgste.acc = skey.acc;
- pgste.fp = skey.fp;
- pgste.gr |= skey.r;
- pgste.gc |= skey.c;
-
- return pgste;
-}
-
union pgste __dat_ptep_xchg(union pte *ptep, union pgste pgste, union pte new, gfn_t gfn,
union asce asce, bool uses_skeys)
{
diff --git a/arch/s390/kvm/gmap/dat.h b/arch/s390/kvm/gmap/dat.h
index 90389d47ba4e..3167543bf2cc 100644
--- a/arch/s390/kvm/gmap/dat.h
+++ b/arch/s390/kvm/gmap/dat.h
@@ -988,4 +988,30 @@ static inline bool crste_is_ucas(union crste crste)
return is_pmd(crste) && crste.h.i && crste.h.fc0.tl == 1 && crste.h.fc == 0;
}
+static inline void dat_set_storage_key_from_pgste(union pte pte, union pgste pgste)
+{
+ union skey nkey = { .acc = pgste.acc, .fp = pgste.fp };
+
+ page_set_storage_key(pte_origin(pte), nkey.skey, 0);
+}
+
+static inline void dat_move_storage_key(union pte old, union pte new)
+{
+ page_set_storage_key(pte_origin(new), page_get_storage_key(pte_origin(old)), 1);
+}
+
+static inline union pgste dat_save_storage_key_into_pgste(union pte pte, union pgste pgste)
+{
+ union skey skey;
+
+ skey.skey = page_get_storage_key(pte_origin(pte));
+
+ pgste.acc = skey.acc;
+ pgste.fp = skey.fp;
+ pgste.gr |= skey.r;
+ pgste.gc |= skey.c;
+
+ return pgste;
+}
+
#endif /* ARCH_KVM_GMAP_DAT_H */
--
2.56.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v1 02/10] KVM: s390: Refactor and improve DAT tables handling
2026-10-22 15:55 [PATCH v1 00/10] KVM: s390: Misc bugfixes and improvements for 7.4 Claudio Imbrenda
2026-10-22 15:55 ` [PATCH v1 01/10] KVM: s390: Move some skey functions to headers Claudio Imbrenda
@ 2026-10-22 15:55 ` Claudio Imbrenda
2026-10-22 16:05 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 03/10] KVM: s390: Move prefix marker from PGSTEs to PTEs Claudio Imbrenda
` (7 subsequent siblings)
9 siblings, 1 reply; 21+ messages in thread
From: Claudio Imbrenda @ 2026-10-22 15:55 UTC (permalink / raw)
To: kvm; +Cc: linux-s390, borntraeger, frankja, david, nrb, schlameuss, gra
Refactor and improve functions to handle gmap DAT tables. Handle
machine_has_tlb_guest() inside the helper functions; the callers won't
have to worry about it anymore.
Add crdte_pte() to allow using CRDTE on ptes.
Fix idte_crste() to add the missing table-type bits.
Fixes: 5a74e3d93417 ("KVM: s390: KVM-specific bitfields and helper functions")
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
---
arch/s390/kvm/gmap/dat.h | 51 ++++++++++++++++++++++++++++++----------
1 file changed, 39 insertions(+), 12 deletions(-)
diff --git a/arch/s390/kvm/gmap/dat.h b/arch/s390/kvm/gmap/dat.h
index 3167543bf2cc..bf01af50e192 100644
--- a/arch/s390/kvm/gmap/dat.h
+++ b/arch/s390/kvm/gmap/dat.h
@@ -110,6 +110,9 @@ union pte {
#define _PAGES_PER_SEGMENT _PAGE_ENTRIES
#define _PAGES_PER_REGION3 (_PAGES_PER_SEGMENT * _CRST_ENTRIES)
+#define CRDTE_NODAT IPTE_NODAT
+#define CRDTE_GUEST_ASCE IPTE_GUEST_ASCE
+
/* Soft dirty, needed as macro for atomic operations on ptes */
#define _PAGE_SD 0x002
@@ -609,41 +612,65 @@ static inline bool crdte_crste(union crste *crstep, union crste old, union crste
{
unsigned long dtt = 0x10 | new.h.tt << 2;
void *table = crste_table_start(crstep);
+ unsigned long addr = gfn_to_gpa(gfn);
+
+ addr |= machine_has_tlb_guest() ? CRDTE_GUEST_ASCE : 0;
+ return crdte(old.val, new.val, table, dtt, addr, asce.val);
+}
- return crdte(old.val, new.val, table, dtt, gfn_to_gpa(gfn), asce.val);
+static inline bool crdte_pte(union pte *ptep, union pte old, union pte new, gfn_t gfn,
+ union asce asce, bool nt)
+{
+ void *table = pte_table_start(ptep);
+ unsigned long addr = gfn_to_gpa(gfn);
+
+ if (machine_has_tlb_guest())
+ addr |= CRDTE_GUEST_ASCE | (nt ? CRDTE_NODAT : 0);
+ return crdte(old.val, new.val, table, 0, addr, asce.val);
}
/**
* idte_crste() - invalidate a crste entry using idte
* @crstep: pointer to the crste to be invalidated
* @gfn: a gfn mapped by the crste
- * @opt: options for the idte instruction
* @asce: the asce
- * @local: whether the operation is cpu-local
*/
-static __always_inline void idte_crste(union crste *crstep, gfn_t gfn, unsigned long opt,
- union asce asce, int local)
+static __always_inline void idte_crste(union crste *crstep, gfn_t gfn, union asce asce)
{
unsigned long table_origin = __pa(crste_table_start(crstep));
unsigned long gaddr = gfn_to_gpa(gfn) & HPAGE_MASK;
- if (__builtin_constant_p(opt) && opt == 0) {
+ table_origin |= crstep->val & _REGION_ENTRY_TYPE_MASK;
+
+ if (!machine_has_tlb_guest()) {
/* flush without guest asce */
- asm volatile("idte %[table_origin],0,%[gaddr],%[local]"
+ asm volatile("idte %[table_origin],0,%[gaddr]"
: "+m" (*crstep)
- : [table_origin] "a" (table_origin), [gaddr] "a" (gaddr),
- [local] "i" (local)
+ : [table_origin] "a" (table_origin), [gaddr] "a" (gaddr)
: "cc");
} else {
/* flush with guest asce */
- asm volatile("idte %[table_origin],%[asce],%[gaddr_opt],%[local]"
+ asm volatile("idte %[table_origin],%[asce],%[gaddr_opt]"
: "+m" (*crstep)
- : [table_origin] "a" (table_origin), [gaddr_opt] "a" (gaddr | opt),
- [asce] "a" (asce.val), [local] "i" (local)
+ : [table_origin] "a" (table_origin),
+ [gaddr_opt] "a" (gaddr | IDTE_GUEST_ASCE),
+ [asce] "a" (asce.val)
: "cc");
}
}
+static __always_inline void ipte(gfn_t gfn, union pte *ptep, union asce asce, bool nt)
+{
+ unsigned long flags;
+
+ if (machine_has_tlb_guest()) {
+ flags = IPTE_GUEST_ASCE | (nt ? IPTE_NODAT : 0);
+ __ptep_ipte(gfn_to_gpa(gfn), (void *)ptep, flags, asce.val, IPTE_GLOBAL);
+ } else {
+ __ptep_ipte(gfn_to_gpa(gfn), (void *)ptep, 0, 0, IPTE_GLOBAL);
+ }
+}
+
static inline void dat_init_pgstes(struct page_table *pt, unsigned long val)
{
memset64((void *)pt->pgstes, val, PTRS_PER_PTE);
--
2.56.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v1 03/10] KVM: s390: Move prefix marker from PGSTEs to PTEs
2026-10-22 15:55 [PATCH v1 00/10] KVM: s390: Misc bugfixes and improvements for 7.4 Claudio Imbrenda
2026-10-22 15:55 ` [PATCH v1 01/10] KVM: s390: Move some skey functions to headers Claudio Imbrenda
2026-10-22 15:55 ` [PATCH v1 02/10] KVM: s390: Refactor and improve DAT tables handling Claudio Imbrenda
@ 2026-10-22 15:55 ` Claudio Imbrenda
2026-10-22 16:06 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 04/10] KVM: s390: Automatically enable storage keys for UCONTROL VMs Claudio Imbrenda
` (6 subsequent siblings)
9 siblings, 1 reply; 21+ messages in thread
From: Claudio Imbrenda @ 2026-10-22 15:55 UTC (permalink / raw)
To: kvm; +Cc: linux-s390, borntraeger, frankja, david, nrb, schlameuss, gra
Move the prefix marker from PGSTEs to PTEs; this is a preparation for
PGSTE-less guests.
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
---
arch/s390/kvm/gmap/dat.c | 11 ++++++-----
arch/s390/kvm/gmap/dat.h | 10 +++++-----
arch/s390/kvm/gmap/gmap.c | 7 ++++---
arch/s390/kvm/gmap/gmap.h | 4 ++--
arch/s390/kvm/s390/gaccess.c | 1 +
5 files changed, 18 insertions(+), 15 deletions(-)
diff --git a/arch/s390/kvm/gmap/dat.c b/arch/s390/kvm/gmap/dat.c
index b9507daaf327..48bb97ae4dd1 100644
--- a/arch/s390/kvm/gmap/dat.c
+++ b/arch/s390/kvm/gmap/dat.c
@@ -234,12 +234,12 @@ static int dat_split_ste(struct kvm_s390_mmu_cache *mc, union pmd *pmdp, gfn_t g
init.s.y = old.s.fc1.y;
init.s.sd = old.s.fc1.sd;
init.s.pr = old.s.fc1.pr;
+ init.s.prefix_notif = old.s.fc1.prefix_notif;
pgste_init.val = 0;
if (old.h.fc) {
for (i = 0; i < _PAGE_ENTRIES; i++)
pt->ptes[i].val = init.val | i * PAGE_SIZE;
/* No need to take locks as the page table is not installed yet. */
- pgste_init.prefix_notif = old.s.fc1.prefix_notif;
pgste_init.vsie_notif = old.s.fc1.vsie_notif;
pgste_init.vsie_gmem = old.s.fc1.vsie_notif;
pgste_init.pcl = uses_skeys && init.h.i;
@@ -1017,14 +1017,15 @@ static long dat_set_pn_crste(union crste *crstep, gfn_t gfn, gfn_t next, struct
static long dat_set_pn_pte(union pte *ptep, gfn_t gfn, gfn_t next, struct dat_walk *walk)
{
int *n = walk->priv;
- union pgste pgste;
- pgste = pgste_get_lock(ptep);
+ /*
+ * Updating a softbit while holding kvm->mmu_lock in write mode, no
+ * need to take the PCL lock.
+ */
if (!ptep->h.i && !ptep->h.p) {
- pgste.prefix_notif = 1;
+ __atomic64_or(_PAGE_PREFIX_NOTIF, &ptep->val);
*n += 1;
}
- pgste_set_unlock(ptep, pgste);
return 0;
}
diff --git a/arch/s390/kvm/gmap/dat.h b/arch/s390/kvm/gmap/dat.h
index bf01af50e192..27cc5abe6eaf 100644
--- a/arch/s390/kvm/gmap/dat.h
+++ b/arch/s390/kvm/gmap/dat.h
@@ -77,7 +77,7 @@ union pte {
unsigned long u : 1; /* Page unused */
unsigned long s : 1; /* Special */
unsigned long w : 1; /* Writable */
- unsigned long r : 1; /* Readable */
+ unsigned long prefix_notif:1; /* Guest prefix invalidation notification */
unsigned long d : 1; /* Dirty */
unsigned long y : 1; /* Young */
unsigned long sd: 1; /* Soft dirty */
@@ -113,8 +113,9 @@ union pte {
#define CRDTE_NODAT IPTE_NODAT
#define CRDTE_GUEST_ASCE IPTE_GUEST_ASCE
-/* Soft dirty, needed as macro for atomic operations on ptes */
-#define _PAGE_SD 0x002
+/* Needed as macro for atomic operations on ptes */
+#define _PAGE_PREFIX_NOTIF 0x010 /* Guest prefix invalidation notification */
+#define _PAGE_SD 0x002 /* Soft dirty */
/* Needed as macro to perform atomic operations */
#define PGSTE_PCL_BIT 0x0080000000000000UL /* PCL lock, HW bit */
@@ -147,10 +148,9 @@ union pgste {
unsigned long usage : 2;
unsigned long : 8;
unsigned long cmma_d : 1; /* Dirty flag for CMMA bits */
- unsigned long prefix_notif : 1; /* Guest prefix invalidation notification */
unsigned long vsie_notif : 1; /* Referenced in a shadow table */
unsigned long vsie_gmem : 1; /* Contains nested guest memory */
- unsigned long : 4;
+ unsigned long : 5;
unsigned long : 8;
};
struct {
diff --git a/arch/s390/kvm/gmap/gmap.c b/arch/s390/kvm/gmap/gmap.c
index 3f3fa864cc36..bf7f27d6be96 100644
--- a/arch/s390/kvm/gmap/gmap.c
+++ b/arch/s390/kvm/gmap/gmap.c
@@ -298,7 +298,7 @@ static long gmap_clear_young_pte(union pte *ptep, gfn_t gfn, gfn_t end, struct d
return 0;
pgste = pgste_get_lock(ptep);
- if (!pgste.prefix_notif || gmap_mkold_prefix(p->gmap, gfn, end)) {
+ if (!pte.s.prefix_notif || gmap_mkold_prefix(p->gmap, gfn, end)) {
new = pte;
new.h.i = 1;
new.s.y = 0;
@@ -307,7 +307,7 @@ static long gmap_clear_young_pte(union pte *ptep, gfn_t gfn, gfn_t end, struct d
new.s.d = 0;
new.h.p = 1;
- pgste.prefix_notif = 0;
+ new.s.prefix_notif = 0;
pgste = __dat_ptep_xchg(ptep, pgste, new, gfn, walk->asce, uses_skeys(p->gmap));
}
p->young = 1;
@@ -470,7 +470,7 @@ static union pgste __pte_test_and_clear_softdirty(union pte *ptep, union pgste p
* If this page contains one or more prefixes of vCPUS that are currently
* running, do not reset the protection, leave it marked as dirty.
*/
- if (!pgste.prefix_notif || gmap_mkold_prefix(gmap, gfn, gfn + 1)) {
+ if (!pte.s.prefix_notif || gmap_mkold_prefix(gmap, gfn, gfn + 1)) {
pte.h.p = 1;
pte.s.sd = 0;
pgste = gmap_ptep_xchg(gmap, ptep, pte, pgste, gfn);
@@ -707,6 +707,7 @@ static int _gmap_link(struct kvm_s390_mmu_cache *mc, struct gmap *gmap, int leve
pgste = pgste_get_lock(f->ptep);
oldpte = *f->ptep;
newpte = _pte(f->pfn, f->writable, f->write_attempt | oldpte.s.d, !f->page);
+ newpte.s.prefix_notif = oldpte.s.prefix_notif;
newpte.s.sd = oldpte.s.sd;
oldpte.s.sd = 0;
if (oldpte.val == _PTE_EMPTY.val || oldpte.h.pfra == f->pfn) {
diff --git a/arch/s390/kvm/gmap/gmap.h b/arch/s390/kvm/gmap/gmap.h
index 8f47f29c89fd..4042e1d4e77c 100644
--- a/arch/s390/kvm/gmap/gmap.h
+++ b/arch/s390/kvm/gmap/gmap.h
@@ -232,8 +232,8 @@ static inline union pgste _gmap_ptep_xchg(struct gmap *gmap, union pte *ptep, un
else
lockdep_assert_not_held(&gmap->children_lock);
- if (pgste.prefix_notif && (newpte.h.p || newpte.h.i)) {
- pgste.prefix_notif = 0;
+ if (ptep->s.prefix_notif && (newpte.h.p || newpte.h.i)) {
+ newpte.s.prefix_notif = 0;
gmap_unmap_prefix(gmap, gfn, gfn + 1);
}
if (pte_needs_unshadow(*ptep, newpte, pgste)) {
diff --git a/arch/s390/kvm/s390/gaccess.c b/arch/s390/kvm/s390/gaccess.c
index 2d27dcb68002..8dc5b93ef01c 100644
--- a/arch/s390/kvm/s390/gaccess.c
+++ b/arch/s390/kvm/s390/gaccess.c
@@ -1435,6 +1435,7 @@ static int _do_shadow_pte(struct kvm_s390_mmu_cache *mc, struct gmap *sg, gpa_t
newpte.s.d |= ptep_h->s.d;
newpte.s.sd |= ptep_h->s.sd;
newpte.h.p &= ptep_h->h.p;
+ newpte.s.prefix_notif = ptep_h->s.prefix_notif;
if (!newpte.h.p && !f->writable) {
rc = -EOPNOTSUPP;
} else {
--
2.56.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v1 04/10] KVM: s390: Automatically enable storage keys for UCONTROL VMs
2026-10-22 15:55 [PATCH v1 00/10] KVM: s390: Misc bugfixes and improvements for 7.4 Claudio Imbrenda
` (2 preceding siblings ...)
2026-10-22 15:55 ` [PATCH v1 03/10] KVM: s390: Move prefix marker from PGSTEs to PTEs Claudio Imbrenda
@ 2026-10-22 15:55 ` Claudio Imbrenda
2026-10-22 16:13 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 05/10] KVM: s390: Replace ASCE usage with a struct with more flags Claudio Imbrenda
` (5 subsequent siblings)
9 siblings, 1 reply; 21+ messages in thread
From: Claudio Imbrenda @ 2026-10-22 15:55 UTC (permalink / raw)
To: kvm; +Cc: linux-s390, borntraeger, frankja, david, nrb, schlameuss, gra
Automatically enable storage keys for UCONTROL VMs upon creation. There
is no point in leaving it to the lazy initialization, as storage keys
will almost surely be used very early in the boot process of a UCONTROL
guest. Furthermore, if storage keys initialization fails, it will
prevent the VM from starting, instead of failing at runtime.
Opportunistically improve error handling if the fake memslot cannot be
initialized properly.
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
---
arch/s390/kvm/s390/s390.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/arch/s390/kvm/s390/s390.c b/arch/s390/kvm/s390/s390.c
index e4912db67adc..dafcd104d316 100644
--- a/arch/s390/kvm/s390/s390.c
+++ b/arch/s390/kvm/s390/s390.c
@@ -3346,10 +3346,15 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
.flags = 0,
};
+ rc = gmap_enable_skeys(kvm->arch.gmap);
+ if (rc)
+ goto out_err;
/* one flat fake memslot covering the whole address-space */
mutex_lock(&kvm->slots_lock);
- KVM_BUG_ON(kvm_set_internal_memslot(kvm, &fake_memslot), kvm);
+ rc = kvm_set_internal_memslot(kvm, &fake_memslot);
mutex_unlock(&kvm->slots_lock);
+ if (KVM_BUG_ON(rc, kvm))
+ goto out_err;
set_bit(GMAP_FLAG_IS_UCONTROL, &kvm->arch.gmap->flags);
} else {
struct crst_table *table = dereference_asce(kvm->arch.gmap->asce);
@@ -3369,6 +3374,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
return 0;
out_err:
+ if (kvm->arch.gmap)
+ kvm->arch.gmap = gmap_put(kvm->arch.gmap);
free_page((unsigned long)kvm->arch.sie_page2);
debug_unregister(kvm->arch.dbf);
sca_dispose(kvm);
--
2.56.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v1 05/10] KVM: s390: Replace ASCE usage with a struct with more flags
2026-10-22 15:55 [PATCH v1 00/10] KVM: s390: Misc bugfixes and improvements for 7.4 Claudio Imbrenda
` (3 preceding siblings ...)
2026-10-22 15:55 ` [PATCH v1 04/10] KVM: s390: Automatically enable storage keys for UCONTROL VMs Claudio Imbrenda
@ 2026-10-22 15:55 ` Claudio Imbrenda
2026-10-22 16:09 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 06/10] KVM: s390: Rework some helper functions Claudio Imbrenda
` (4 subsequent siblings)
9 siblings, 1 reply; 21+ messages in thread
From: Claudio Imbrenda @ 2026-10-22 15:55 UTC (permalink / raw)
To: kvm; +Cc: linux-s390, borntraeger, frankja, david, nrb, schlameuss, gra
Replace usage of simple ASCEs with a struct that contains the ASCE, a
pointer to the table, and some flags.
This way, there will be no need to manually pass the various flags in
addition to the ASCE where needed. Walking the DAT tables will also not
need to first convert the physical address to virtual.
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
---
arch/s390/kvm/gmap/dat.c | 158 ++++++++++++++++++-----------------
arch/s390/kvm/gmap/dat.h | 93 ++++++++++++---------
arch/s390/kvm/gmap/gmap.c | 114 +++++++++++++------------
arch/s390/kvm/gmap/gmap.h | 27 ++----
arch/s390/kvm/gmap/kvm_mmu.c | 6 +-
arch/s390/kvm/s390/gaccess.c | 26 +++---
arch/s390/kvm/s390/priv.c | 15 ++--
arch/s390/kvm/s390/pv.c | 11 +--
arch/s390/kvm/s390/s390.c | 35 ++++----
arch/s390/kvm/s390/vsie.c | 3 +-
10 files changed, 252 insertions(+), 236 deletions(-)
diff --git a/arch/s390/kvm/gmap/dat.c b/arch/s390/kvm/gmap/dat.c
index 48bb97ae4dd1..807a4ef84a07 100644
--- a/arch/s390/kvm/gmap/dat.c
+++ b/arch/s390/kvm/gmap/dat.c
@@ -102,34 +102,37 @@ void dat_free_level(struct crst_table *table, bool owns_ptes)
dat_free_crst(table);
}
-int dat_set_asce_limit(struct kvm_s390_mmu_cache *mc, union asce *asce, int newtype)
+int dat_set_asce_limit(struct kvm_s390_mmu_cache *mc, struct dat_tables *datt, int newtype)
{
struct crst_table *table;
union crste crste;
- while (asce->dt > newtype) {
- table = dereference_asce(*asce);
+ while (datt->asce.dt > newtype) {
+ table = datt->table;
crste = table->crstes[0];
if (crste.h.fc)
return 0;
if (!crste.h.i) {
- asce->rsto = crste.h.fc0.to;
+ datt->asce.rsto = crste.h.fc0.to;
dat_free_crst(table);
+ datt->table = phys_to_virt(datt->asce.val & _ASCE_ORIGIN);
} else {
crste.h.tt--;
crst_table_init((void *)table, crste.val);
}
- asce->dt--;
+ datt->asce.dt--;
}
- while (asce->dt < newtype) {
- crste = _crste_fc0(asce->rsto, asce->dt + 1);
+ while (datt->asce.dt < newtype) {
+ crste = _crste_fc0(datt->asce.rsto, datt->asce.dt + 1);
table = dat_alloc_crst_noinit(mc);
if (!table)
return -ENOMEM;
crst_table_init((void *)table, _CRSTE_HOLE(crste.h.tt).val);
table->crstes[0] = crste;
- asce->rsto = __pa(table) >> PAGE_SHIFT;
- asce->dt++;
+
+ datt->asce.rsto = __pa(table) >> PAGE_SHIFT;
+ datt->asce.dt++;
+ datt->table = table;
}
return 0;
}
@@ -160,7 +163,7 @@ bool __must_check dat_crstep_xchg_atomic(union crste *crstep, union crste old, u
}
union pgste __dat_ptep_xchg(union pte *ptep, union pgste pgste, union pte new, gfn_t gfn,
- union asce asce, bool uses_skeys)
+ struct dat_tables datt)
{
union pte old = READ_ONCE(*ptep);
@@ -174,12 +177,13 @@ union pgste __dat_ptep_xchg(union pte *ptep, union pgste pgste, union pte new, g
unsigned long opts = IPTE_GUEST_ASCE | (pgste.nodat ? IPTE_NODAT : 0);
if (machine_has_tlb_guest())
- __ptep_ipte(gfn_to_gpa(gfn), (void *)ptep, opts, asce.val, IPTE_GLOBAL);
+ __ptep_ipte(gfn_to_gpa(gfn), (void *)ptep, opts,
+ datt.asce.val, IPTE_GLOBAL);
else
__ptep_ipte(gfn_to_gpa(gfn), (void *)ptep, 0, 0, IPTE_GLOBAL);
}
- if (uses_skeys) {
+ if (datt.uses_skeys) {
if (old.h.i && !new.h.i)
/* Invalid to valid: restore storage keys from PGSTE. */
dat_set_storage_key_from_pgste(new, pgste);
@@ -205,7 +209,7 @@ union pgste __dat_ptep_xchg(union pte *ptep, union pgste pgste, union pte new, g
* Return: 0 in case of success, -ENOMEM if running out of memory.
*/
static int dat_split_ste(struct kvm_s390_mmu_cache *mc, union pmd *pmdp, gfn_t gfn,
- union asce asce, bool uses_skeys)
+ struct dat_tables datt)
{
union pgste pgste_init;
struct page_table *pt;
@@ -242,13 +246,13 @@ static int dat_split_ste(struct kvm_s390_mmu_cache *mc, union pmd *pmdp, gfn_t g
/* No need to take locks as the page table is not installed yet. */
pgste_init.vsie_notif = old.s.fc1.vsie_notif;
pgste_init.vsie_gmem = old.s.fc1.vsie_notif;
- pgste_init.pcl = uses_skeys && init.h.i;
+ pgste_init.pcl = datt.uses_skeys && init.h.i;
dat_init_pgstes(pt, pgste_init.val);
} else {
dat_init_page_table(pt, init.val, 0);
}
- if (dat_pmdp_xchg_atomic(pmdp, old, new, gfn, asce)) {
+ if (dat_pmdp_xchg_atomic(pmdp, old, new, gfn, datt.asce)) {
if (!pgste_init.pcl)
return 0;
for (i = 0; i < _PAGE_ENTRIES; i++) {
@@ -274,7 +278,7 @@ static int dat_split_ste(struct kvm_s390_mmu_cache *mc, union pmd *pmdp, gfn_t g
* Return: %0 in case of success, %-ENOMEM if running out of memory.
*/
static int dat_split_crste(struct kvm_s390_mmu_cache *mc, union crste *crstep,
- gfn_t gfn, union asce asce, bool uses_skeys)
+ gfn_t gfn, struct dat_tables datt)
{
struct crst_table *table;
union crste old, new, init;
@@ -282,7 +286,7 @@ static int dat_split_crste(struct kvm_s390_mmu_cache *mc, union crste *crstep,
old = READ_ONCE(*crstep);
if (is_pmd(old))
- return dat_split_ste(mc, &crstep->pmd, gfn, asce, uses_skeys);
+ return dat_split_ste(mc, &crstep->pmd, gfn, datt);
BUG_ON(!mc);
@@ -307,7 +311,7 @@ static int dat_split_crste(struct kvm_s390_mmu_cache *mc, union crste *crstep,
} else {
crst_table_init((void *)table, init.val);
}
- if (dat_crstep_xchg_atomic(crstep, old, new, gfn, asce))
+ if (dat_crstep_xchg_atomic(crstep, old, new, gfn, datt.asce))
return 0;
old = READ_ONCE(*crstep);
}
@@ -321,7 +325,7 @@ static int dat_split_crste(struct kvm_s390_mmu_cache *mc, union crste *crstep,
* @mc: Cache to use to allocate dat tables, if needed; can be NULL if neither
* %DAT_WALK_SPLIT or %DAT_WALK_ALLOC is specified in @flags.
* @gfn: Guest frame.
- * @asce: The ASCE of the address space.
+ * @datt: The DAT tables of the address space.
* @flags: Flags from WALK_* macros.
* @walk_level: Level to walk to, from LEVEL_* macros.
* @last: Will be filled the last visited non-pte DAT entry.
@@ -337,7 +341,6 @@ static int dat_split_crste(struct kvm_s390_mmu_cache *mc, union crste *crstep,
* * %DAT_WALK_ANY: return successfully even if the requested level could not be reached
* * %DAT_WALK_CONTINUE: walk to the requested level with the specified flags, and then try to
* continue walking to ptes with only DAT_WALK_ANY
- * * %DAT_WALK_USES_SKEYS: storage keys are in use
*
* Context: called with kvm->mmu_lock held.
*
@@ -350,12 +353,11 @@ static int dat_split_crste(struct kvm_s390_mmu_cache *mc, union crste *crstep,
* * %-ENOENT if the requested level could not be reached for other reasons
* * %-ENOMEM if running out of memory while allocating or splitting a table
*/
-int dat_entry_walk(struct kvm_s390_mmu_cache *mc, gfn_t gfn, union asce asce, int flags,
+int dat_entry_walk(struct kvm_s390_mmu_cache *mc, gfn_t gfn, struct dat_tables datt, int flags,
int walk_level, union crste **last, union pte **ptepp)
{
union vaddress vaddr = { .addr = gfn_to_gpa(gfn) };
bool continue_anyway = flags & DAT_WALK_CONTINUE;
- bool uses_skeys = flags & DAT_WALK_USES_SKEYS;
bool ign_holes = flags & DAT_WALK_IGN_HOLES;
bool allocate = flags & DAT_WALK_ALLOC;
bool split = flags & DAT_WALK_SPLIT;
@@ -368,15 +370,15 @@ int dat_entry_walk(struct kvm_s390_mmu_cache *mc, gfn_t gfn, union asce asce, in
*last = NULL;
*ptepp = NULL;
- if (WARN_ON_ONCE(unlikely(!asce.val)))
+ if (WARN_ON_ONCE(unlikely(!datt.asce.val)))
return -EINVAL;
- if (WARN_ON_ONCE(unlikely(walk_level > asce.dt)))
+ if (WARN_ON_ONCE(unlikely(walk_level > datt.asce.dt)))
return -EINVAL;
- if (!asce_contains_gfn(asce, gfn))
+ if (!dat_tables_contain_gfn(datt, gfn))
return PGM_ADDRESSING;
- table = dereference_asce(asce);
- if (asce.dt >= ASCE_TYPE_REGION1) {
+ table = datt.table;
+ if (datt.asce.dt >= ASCE_TYPE_REGION1) {
*last = table->crstes + vaddr.rfx;
entry = READ_ONCE(**last);
if (WARN_ON_ONCE(entry.h.tt != TABLE_TYPE_REGION1))
@@ -388,7 +390,7 @@ int dat_entry_walk(struct kvm_s390_mmu_cache *mc, gfn_t gfn, union asce asce, in
if (entry.pgd.h.i) {
if (!allocate)
return any ? 0 : -ENOENT;
- rc = dat_split_crste(mc, *last, gfn, asce, uses_skeys);
+ rc = dat_split_crste(mc, *last, gfn, datt);
if (rc)
return rc;
entry = READ_ONCE(**last);
@@ -396,7 +398,7 @@ int dat_entry_walk(struct kvm_s390_mmu_cache *mc, gfn_t gfn, union asce asce, in
table = dereference_crste(entry.pgd);
}
- if (asce.dt >= ASCE_TYPE_REGION2) {
+ if (datt.asce.dt >= ASCE_TYPE_REGION2) {
*last = table->crstes + vaddr.rsx;
entry = READ_ONCE(**last);
if (WARN_ON_ONCE(entry.h.tt != TABLE_TYPE_REGION2))
@@ -408,7 +410,7 @@ int dat_entry_walk(struct kvm_s390_mmu_cache *mc, gfn_t gfn, union asce asce, in
if (entry.p4d.h.i) {
if (!allocate)
return any ? 0 : -ENOENT;
- rc = dat_split_crste(mc, *last, gfn, asce, uses_skeys);
+ rc = dat_split_crste(mc, *last, gfn, datt);
if (rc)
return rc;
entry = READ_ONCE(**last);
@@ -416,7 +418,7 @@ int dat_entry_walk(struct kvm_s390_mmu_cache *mc, gfn_t gfn, union asce asce, in
table = dereference_crste(entry.p4d);
}
- if (asce.dt >= ASCE_TYPE_REGION3) {
+ if (datt.asce.dt >= ASCE_TYPE_REGION3) {
*last = table->crstes + vaddr.rtx;
entry = READ_ONCE(**last);
if (WARN_ON_ONCE(entry.h.tt != TABLE_TYPE_REGION3))
@@ -433,7 +435,7 @@ int dat_entry_walk(struct kvm_s390_mmu_cache *mc, gfn_t gfn, union asce asce, in
if (entry.pud.h.i && !entry.pud.h.fc) {
if (!allocate)
return any ? 0 : -ENOENT;
- rc = dat_split_crste(mc, *last, gfn, asce, uses_skeys);
+ rc = dat_split_crste(mc, *last, gfn, datt);
if (rc)
return rc;
entry = READ_ONCE(**last);
@@ -441,7 +443,7 @@ int dat_entry_walk(struct kvm_s390_mmu_cache *mc, gfn_t gfn, union asce asce, in
if (walk_level <= TABLE_TYPE_SEGMENT && entry.pud.h.fc) {
if (!split)
return -EFBIG;
- rc = dat_split_crste(mc, *last, gfn, asce, uses_skeys);
+ rc = dat_split_crste(mc, *last, gfn, datt);
if (rc)
return rc;
entry = READ_ONCE(**last);
@@ -465,7 +467,7 @@ int dat_entry_walk(struct kvm_s390_mmu_cache *mc, gfn_t gfn, union asce asce, in
if (entry.pmd.h.i && !entry.pmd.h.fc) {
if (!allocate)
return any ? 0 : -ENOENT;
- rc = dat_split_ste(mc, &(*last)->pmd, gfn, asce, uses_skeys);
+ rc = dat_split_ste(mc, &(*last)->pmd, gfn, datt);
if (rc)
return rc;
entry = READ_ONCE(**last);
@@ -473,7 +475,7 @@ int dat_entry_walk(struct kvm_s390_mmu_cache *mc, gfn_t gfn, union asce asce, in
if (walk_level <= TABLE_TYPE_PAGE_TABLE && entry.pmd.h.fc) {
if (!split)
return -EFBIG;
- rc = dat_split_ste(mc, &(*last)->pmd, gfn, asce, uses_skeys);
+ rc = dat_split_ste(mc, &(*last)->pmd, gfn, datt);
if (rc)
return rc;
entry = READ_ONCE(**last);
@@ -555,7 +557,7 @@ static long dat_crste_walk_range(gfn_t start, gfn_t end, struct crst_table *tabl
* _dat_walk_gfn_range() - Walk DAT tables.
* @start: The first guest page frame to walk.
* @end: The guest page frame immediately after the last one to walk.
- * @asce: The ASCE of the guest mapping.
+ * @datt: The DAT tables of the guest mapping.
* @ops: The gmap_walk_ops that will be used to perform the walk.
* @flags: Flags from WALK_* (currently only WALK_IGN_HOLES is supported).
* @priv: Will be passed as-is to the callbacks.
@@ -566,29 +568,28 @@ static long dat_crste_walk_range(gfn_t start, gfn_t end, struct crst_table *tabl
* given ASCE unless the DAT_WALK_IGN_HOLES flag is specified,
* otherwise it returns whatever the callbacks return.
*/
-long _dat_walk_gfn_range(gfn_t start, gfn_t end, union asce asce,
+long _dat_walk_gfn_range(gfn_t start, gfn_t end, struct dat_tables datt,
const struct dat_walk_ops *ops, int flags, void *priv)
{
- struct crst_table *table = dereference_asce(asce);
struct dat_walk walk = {
.ops = ops,
- .asce = asce,
+ .datt = datt,
.priv = priv,
.flags = flags,
.start = start,
.end = end,
};
- if (WARN_ON_ONCE(unlikely(!asce.val)))
+ if (WARN_ON_ONCE(unlikely(!datt.asce.val)))
return -EINVAL;
- if (!asce_contains_gfn(asce, start))
+ if (!dat_tables_contain_gfn(datt, start))
return (flags & DAT_WALK_IGN_HOLES) ? 0 : -EFAULT;
- return dat_crste_walk_range(start, min(end, asce_end(asce)), table, &walk);
+ return dat_crste_walk_range(start, min(end, dat_tables_end(datt)), datt.table, &walk);
}
#if KVM_S390_MANAGES_S390_GUEST
-int dat_get_storage_key(union asce asce, gfn_t gfn, union skey *skey)
+int dat_get_storage_key(struct dat_tables datt, gfn_t gfn, union skey *skey)
{
union crste *crstep;
union pgste pgste;
@@ -597,7 +598,7 @@ int dat_get_storage_key(union asce asce, gfn_t gfn, union skey *skey)
again:
skey->skey = 0;
- rc = dat_entry_walk(NULL, gfn, asce, DAT_WALK_ANY, TABLE_TYPE_PAGE_TABLE, &crstep, &ptep);
+ rc = dat_entry_walk(NULL, gfn, datt, DAT_WALK_ANY, TABLE_TYPE_PAGE_TABLE, &crstep, &ptep);
if (rc)
return rc;
@@ -631,7 +632,7 @@ static void dat_update_ptep_sd(union pgste old, union pgste pgste, union pte *pt
__atomic64_or(_PAGE_SD, &ptep->val);
}
-int dat_set_storage_key(struct kvm_s390_mmu_cache *mc, union asce asce, gfn_t gfn,
+int dat_set_storage_key(struct kvm_s390_mmu_cache *mc, struct dat_tables datt, gfn_t gfn,
union skey skey, bool nq)
{
union pgste pgste, old;
@@ -640,7 +641,7 @@ int dat_set_storage_key(struct kvm_s390_mmu_cache *mc, union asce asce, gfn_t gf
int rc;
again:
- rc = dat_entry_walk(mc, gfn, asce, DAT_WALK_LEAF_ALLOC, TABLE_TYPE_PAGE_TABLE,
+ rc = dat_entry_walk(mc, gfn, datt, DAT_WALK_LEAF_ALLOC, TABLE_TYPE_PAGE_TABLE,
&crstep, &ptep);
if (rc)
return rc;
@@ -693,7 +694,7 @@ static bool page_cond_set_storage_key(phys_addr_t paddr, union skey skey, union
return true;
}
-int dat_cond_set_storage_key(struct kvm_s390_mmu_cache *mmc, union asce asce, gfn_t gfn,
+int dat_cond_set_storage_key(struct kvm_s390_mmu_cache *mmc, struct dat_tables datt, gfn_t gfn,
union skey skey, union skey *oldkey, bool nq, bool mr, bool mc)
{
union pgste pgste, old;
@@ -703,7 +704,7 @@ int dat_cond_set_storage_key(struct kvm_s390_mmu_cache *mmc, union asce asce, gf
int rc;
again:
- rc = dat_entry_walk(mmc, gfn, asce, DAT_WALK_LEAF_ALLOC, TABLE_TYPE_PAGE_TABLE,
+ rc = dat_entry_walk(mmc, gfn, datt, DAT_WALK_LEAF_ALLOC, TABLE_TYPE_PAGE_TABLE,
&crstep, &ptep);
if (rc)
return rc;
@@ -751,7 +752,7 @@ int dat_cond_set_storage_key(struct kvm_s390_mmu_cache *mmc, union asce asce, gf
return rc;
}
-int dat_reset_reference_bit(union asce asce, gfn_t gfn, union skey *skey)
+int dat_reset_reference_bit(struct dat_tables datt, gfn_t gfn, union skey *skey)
{
union pgste pgste, old;
union crste *crstep;
@@ -760,7 +761,7 @@ int dat_reset_reference_bit(union asce asce, gfn_t gfn, union skey *skey)
skey->skey = 0;
again:
- rc = dat_entry_walk(NULL, gfn, asce, DAT_WALK_ANY, TABLE_TYPE_PAGE_TABLE, &crstep, &ptep);
+ rc = dat_entry_walk(NULL, gfn, datt, DAT_WALK_ANY, TABLE_TYPE_PAGE_TABLE, &crstep, &ptep);
if (rc)
return rc;
@@ -828,7 +829,7 @@ static long dat_reset_skeys_crste(union crste *crstep, gfn_t gfn, gfn_t next, st
return 0;
}
-long dat_reset_skeys(union asce asce, gfn_t start)
+long dat_reset_skeys(struct dat_tables datt, gfn_t start)
{
const struct dat_walk_ops ops = {
.pte_entry = dat_reset_skeys_pte,
@@ -836,7 +837,8 @@ long dat_reset_skeys(union asce asce, gfn_t start)
.pud_entry = dat_reset_skeys_crste,
};
- return _dat_walk_gfn_range(start, asce_end(asce), asce, &ops, DAT_WALK_IGN_HOLES, NULL);
+ return _dat_walk_gfn_range(start, dat_tables_end(datt), datt, &ops,
+ DAT_WALK_IGN_HOLES, NULL);
}
#endif /* KVM_S390_MANAGES_S390_GUEST */
@@ -852,7 +854,7 @@ static long _dat_slot_pte(union pte *ptep, gfn_t gfn, gfn_t next, struct dat_wal
return 0;
pgste = pgste_get_lock(ptep);
- pgste = __dat_ptep_xchg(ptep, pgste, new_pte, gfn, walk->asce, false);
+ pgste = __dat_ptep_xchg(ptep, pgste, new_pte, gfn, walk->datt);
pgste.cmma_d = 0;
pgste_set_unlock(ptep, pgste);
@@ -872,7 +874,7 @@ static long _dat_slot_crste(union crste *crstep, gfn_t gfn, gfn_t next, struct d
/* This table entry needs to be updated. */
if (walk->start <= gfn && walk->end >= next) {
- if (!dat_crstep_xchg_atomic(crstep, crste, new_crste, gfn, walk->asce))
+ if (!dat_crstep_xchg_atomic(crstep, crste, new_crste, gfn, walk->datt.asce))
return -EINVAL;
/* A lower level table was present, needs to be freed. */
if (!crste.h.fc && !crste.h.i) {
@@ -889,7 +891,7 @@ static long _dat_slot_crste(union crste *crstep, gfn_t gfn, gfn_t next, struct d
return 0;
/* Split (install a lower level table), and handle things there. */
if (mc)
- return dat_split_crste(mc, crstep, gfn, walk->asce, false);
+ return dat_split_crste(mc, crstep, gfn, walk->datt);
/* A large page should never cross memslots boundaries */
return -EINVAL;
}
@@ -899,9 +901,9 @@ static const struct dat_walk_ops dat_slot_ops = {
.crste_ops = { _dat_slot_crste, _dat_slot_crste, _dat_slot_crste, _dat_slot_crste, },
};
-int dat_set_slot(struct kvm_s390_mmu_cache *mc, union asce asce, gfn_t start, gfn_t end)
+int dat_set_slot(struct kvm_s390_mmu_cache *mc, struct dat_tables datt, gfn_t start, gfn_t end)
{
- return _dat_walk_gfn_range(start, end, asce, &dat_slot_ops,
+ return _dat_walk_gfn_range(start, end, datt, &dat_slot_ops,
DAT_WALK_IGN_HOLES | DAT_WALK_ANY, mc);
}
@@ -982,7 +984,7 @@ static const struct dat_walk_ops test_age_ops = {
/**
* dat_test_age_gfn() - Test young.
- * @asce: The ASCE whose address range is to be tested.
+ * @datt: The DAT tables whose address range is to be tested.
* @start: The first guest frame of the range to check.
* @end: The guest frame after the last in the range.
*
@@ -990,9 +992,9 @@ static const struct dat_walk_ops test_age_ops = {
*
* Return: %true if any page in the given range is young, otherwise %false.
*/
-bool dat_test_age_gfn(union asce asce, gfn_t start, gfn_t end)
+bool dat_test_age_gfn(struct dat_tables datt, gfn_t start, gfn_t end)
{
- return _dat_walk_gfn_range(start, end, asce, &test_age_ops, 0, NULL) > 0;
+ return _dat_walk_gfn_range(start, end, datt, &test_age_ops, 0, NULL) > 0;
}
#if KVM_S390_MANAGES_S390_GUEST
@@ -1009,7 +1011,7 @@ static long dat_set_pn_crste(union crste *crstep, gfn_t gfn, gfn_t next, struct
break;
newcrste = oldcrste;
newcrste.s.fc1.prefix_notif = 1;
- } while (!dat_crstep_xchg_atomic(crstep, oldcrste, newcrste, gfn, walk->asce));
+ } while (!dat_crstep_xchg_atomic(crstep, oldcrste, newcrste, gfn, walk->datt.asce));
*n = 2;
return 0;
}
@@ -1029,7 +1031,7 @@ static long dat_set_pn_pte(union pte *ptep, gfn_t gfn, gfn_t next, struct dat_wa
return 0;
}
-int dat_set_prefix_notif_bit(union asce asce, gfn_t gfn)
+int dat_set_prefix_notif_bit(struct dat_tables datt, gfn_t gfn)
{
static const struct dat_walk_ops ops = {
.pte_entry = dat_set_pn_pte,
@@ -1039,7 +1041,7 @@ int dat_set_prefix_notif_bit(union asce asce, gfn_t gfn)
int n = 0;
- _dat_walk_gfn_range(gfn, gfn + 2, asce, &ops, DAT_WALK_IGN_HOLES, &n);
+ _dat_walk_gfn_range(gfn, gfn + 2, datt, &ops, DAT_WALK_IGN_HOLES, &n);
if (n != 2)
return -EAGAIN;
return 0;
@@ -1047,7 +1049,7 @@ int dat_set_prefix_notif_bit(union asce asce, gfn_t gfn)
/**
* dat_perform_essa() - Perform ESSA actions on the PGSTE.
- * @asce: The asce to operate on.
+ * @datt: The DAT tables to operate on.
* @gfn: The guest page frame to operate on.
* @orc: The specific action to perform, see the ESSA_SET_* macros.
* @state: The storage attributes to be returned to the guest.
@@ -1060,14 +1062,15 @@ int dat_set_prefix_notif_bit(union asce asce, gfn_t gfn)
* * %0 if the page state has been altered, but the page is not to be added to the CBRL
* * %-1 if the page state has not been altered and the page is not to be added to the CBRL
*/
-int dat_perform_essa(union asce asce, gfn_t gfn, int orc, union essa_state *state, bool *dirty)
+int dat_perform_essa(struct dat_tables datt, gfn_t gfn, int orc,
+ union essa_state *state, bool *dirty)
{
union crste *crstep;
union pgste pgste;
union pte *ptep;
int res = 0;
- if (dat_entry_walk(NULL, gfn, asce, 0, TABLE_TYPE_PAGE_TABLE, &crstep, &ptep)) {
+ if (dat_entry_walk(NULL, gfn, datt, 0, TABLE_TYPE_PAGE_TABLE, &crstep, &ptep)) {
*state = (union essa_state) { .exception = 1 };
return -1;
}
@@ -1155,13 +1158,13 @@ static long dat_reset_cmma_pte(union pte *ptep, gfn_t gfn, gfn_t next, struct da
return 0;
}
-long dat_reset_cmma(union asce asce, gfn_t start)
+long dat_reset_cmma(struct dat_tables datt, gfn_t start)
{
const struct dat_walk_ops dat_reset_cmma_ops = {
.pte_entry = dat_reset_cmma_pte,
};
- return _dat_walk_gfn_range(start, asce_end(asce), asce, &dat_reset_cmma_ops,
+ return _dat_walk_gfn_range(start, dat_tables_end(datt), datt, &dat_reset_cmma_ops,
DAT_WALK_IGN_HOLES, NULL);
}
@@ -1195,7 +1198,7 @@ static long __dat_peek_cmma_crste(union crste *crstep, gfn_t gfn, gfn_t next, st
return 0;
}
-int dat_peek_cmma(gfn_t start, union asce asce, unsigned int *count, u8 *values)
+int dat_peek_cmma(gfn_t start, struct dat_tables datt, unsigned int *count, u8 *values)
{
const struct dat_walk_ops ops = {
.pte_entry = __dat_peek_cmma_pte,
@@ -1207,7 +1210,7 @@ int dat_peek_cmma(gfn_t start, union asce asce, unsigned int *count, u8 *values)
struct dat_get_cmma_state state = { .values = values, };
int rc;
- rc = _dat_walk_gfn_range(start, start + *count, asce, &ops, DAT_WALK_DEFAULT, &state);
+ rc = _dat_walk_gfn_range(start, start + *count, datt, &ops, DAT_WALK_DEFAULT, &state);
*count = state.end >= start ? state.end - start : 0;
/* Return success if at least one value was saved, otherwise an error. */
return (rc == -EFAULT && *count > 0) ? 0 : rc;
@@ -1241,7 +1244,8 @@ static long __dat_get_cmma_pte(union pte *ptep, gfn_t gfn, gfn_t next, struct da
return 0;
}
-int dat_get_cmma(union asce asce, gfn_t *start, unsigned int *count, u8 *values, atomic64_t *rem)
+int dat_get_cmma(struct dat_tables datt, gfn_t *start, unsigned int *count,
+ u8 *values, atomic64_t *rem)
{
const struct dat_walk_ops ops = { .pte_entry = __dat_get_cmma_pte, };
struct dat_get_cmma_state state = {
@@ -1251,10 +1255,10 @@ int dat_get_cmma(union asce asce, gfn_t *start, unsigned int *count, u8 *values,
.start = -1,
};
- _dat_walk_gfn_range(*start, asce_end(asce), asce, &ops, DAT_WALK_IGN_HOLES, &state);
+ _dat_walk_gfn_range(*start, dat_tables_end(datt), datt, &ops, DAT_WALK_IGN_HOLES, &state);
/* If no dirty pages were found, wrap around and continue searching */
if (*start && state.start == -1)
- _dat_walk_gfn_range(0, *start, asce, &ops, DAT_WALK_IGN_HOLES, &state);
+ _dat_walk_gfn_range(0, *start, datt, &ops, DAT_WALK_IGN_HOLES, &state);
if (state.start == -1) {
*count = 0;
@@ -1289,7 +1293,7 @@ static long __dat_set_cmma_pte(union pte *ptep, gfn_t gfn, gfn_t next, struct da
/**
* dat_set_cmma_bits() - Set CMMA bits for a range of guest pages.
* @mc: Cache used for allocations.
- * @asce: The ASCE of the guest.
+ * @datt: The DAT tables of the guest.
* @gfn: The guest frame of the fist page whose CMMA bits are to set.
* @count: How many pages need to be processed.
* @mask: Which PGSTE bits should be set.
@@ -1304,7 +1308,7 @@ static long __dat_set_cmma_pte(union pte *ptep, gfn_t gfn, gfn_t next, struct da
*
* Return: %0 in case of success, a negative error value otherwise.
*/
-int dat_set_cmma_bits(struct kvm_s390_mmu_cache *mc, union asce asce, gfn_t gfn,
+int dat_set_cmma_bits(struct kvm_s390_mmu_cache *mc, struct dat_tables datt, gfn_t gfn,
unsigned long count, unsigned long mask, const uint8_t *bits)
{
const struct dat_walk_ops ops = { .pte_entry = __dat_set_cmma_pte, };
@@ -1315,11 +1319,11 @@ int dat_set_cmma_bits(struct kvm_s390_mmu_cache *mc, union asce asce, gfn_t gfn,
int rc;
for (cur = ALIGN_DOWN(gfn, _PAGE_ENTRIES); cur < gfn + count; cur += _PAGE_ENTRIES) {
- rc = dat_entry_walk(mc, cur, asce, DAT_WALK_ALLOC, TABLE_TYPE_PAGE_TABLE,
+ rc = dat_entry_walk(mc, cur, datt, DAT_WALK_ALLOC, TABLE_TYPE_PAGE_TABLE,
&crstep, &ptep);
if (rc)
return rc;
}
- return _dat_walk_gfn_range(gfn, gfn + count, asce, &ops, DAT_WALK_IGN_HOLES, &state);
+ return _dat_walk_gfn_range(gfn, gfn + count, datt, &ops, DAT_WALK_IGN_HOLES, &state);
}
#endif /* KVM_S390_MANAGES_S390_GUEST */
diff --git a/arch/s390/kvm/gmap/dat.h b/arch/s390/kvm/gmap/dat.h
index 27cc5abe6eaf..5dceb397fc1b 100644
--- a/arch/s390/kvm/gmap/dat.h
+++ b/arch/s390/kvm/gmap/dat.h
@@ -27,9 +27,6 @@
/* For consistency */
#define KVM_S390_CMMA_SIZE_MAX ((u32)KVM_S390_SKEYS_MAX)
-#define _ASCE(x) ((union asce) { .val = (x), })
-#define NULL_ASCE _ASCE(0)
-
enum {
_DAT_TOKEN_NONE = 0,
_DAT_TOKEN_PIC,
@@ -55,7 +52,6 @@ enum {
#define TABLE_TYPE_PAGE_TABLE -1
enum dat_walk_flags {
- DAT_WALK_USES_SKEYS = 0x40,
DAT_WALK_CONTINUE = 0x20,
DAT_WALK_IGN_HOLES = 0x10,
DAT_WALK_SPLIT = 0x08,
@@ -353,6 +349,15 @@ struct page_table {
static_assert(sizeof(struct crst_table) == _CRST_TABLE_SIZE);
static_assert(sizeof(struct page_table) == PAGE_SIZE);
+struct dat_tables {
+ union asce asce;
+ struct crst_table *table;
+ bool has_pgstes; /* Need to use the PCL lock */
+ bool owns_ptes; /* The ptes are owned by this instance */
+ bool uses_skeys; /* Storage keys are in use */
+ bool uses_cmma; /* CMMA is in use */
+};
+
struct dat_walk;
typedef long (*dat_walk_op)(union crste *crste, gfn_t gfn, gfn_t next, struct dat_walk *w);
@@ -374,7 +379,7 @@ struct dat_walk {
const struct dat_walk_ops *ops;
union crste *last;
union pte *last_pte;
- union asce asce;
+ struct dat_tables datt;
gfn_t start;
gfn_t end;
int flags;
@@ -523,52 +528,53 @@ struct guest_fault {
#define PTVAL_VMADDR MKPTVAL(8, 6)
union pgste __must_check __dat_ptep_xchg(union pte *ptep, union pgste pgste, union pte new,
- gfn_t gfn, union asce asce, bool uses_skeys);
+ gfn_t gfn, struct dat_tables datt);
bool dat_crstep_xchg_atomic(union crste *crstep, union crste old, union crste new, gfn_t gfn,
union asce asce);
-void dat_crstep_xchg(union crste *crstep, union crste new, gfn_t gfn, union asce asce);
-long _dat_walk_gfn_range(gfn_t start, gfn_t end, union asce asce,
+long _dat_walk_gfn_range(gfn_t start, gfn_t end, struct dat_tables datt,
const struct dat_walk_ops *ops, int flags, void *priv);
-int dat_entry_walk(struct kvm_s390_mmu_cache *mc, gfn_t gfn, union asce asce, int flags,
+int dat_entry_walk(struct kvm_s390_mmu_cache *mc, gfn_t gfn, struct dat_tables datt, int flags,
int walk_level, union crste **last, union pte **ptepp);
void dat_free_level(struct crst_table *table, bool owns_ptes);
struct crst_table *dat_alloc_crst_sleepable(unsigned long init);
-int dat_set_asce_limit(struct kvm_s390_mmu_cache *mc, union asce *asce, int newtype);
+int dat_set_asce_limit(struct kvm_s390_mmu_cache *mc, struct dat_tables *datt, int newtype);
#if KVM_S390_MANAGES_S390_GUEST
-int dat_get_storage_key(union asce asce, gfn_t gfn, union skey *skey);
-int dat_set_storage_key(struct kvm_s390_mmu_cache *mc, union asce asce, gfn_t gfn,
+int dat_get_storage_key(struct dat_tables datt, gfn_t gfn, union skey *skey);
+int dat_set_storage_key(struct kvm_s390_mmu_cache *mc, struct dat_tables datt, gfn_t gfn,
union skey skey, bool nq);
-int dat_cond_set_storage_key(struct kvm_s390_mmu_cache *mmc, union asce asce, gfn_t gfn,
+int dat_cond_set_storage_key(struct kvm_s390_mmu_cache *mmc, struct dat_tables datt, gfn_t gfn,
union skey skey, union skey *oldkey, bool nq, bool mr, bool mc);
-int dat_reset_reference_bit(union asce asce, gfn_t gfn, union skey *skey);
-long dat_reset_skeys(union asce asce, gfn_t start);
+int dat_reset_reference_bit(struct dat_tables datt, gfn_t gfn, union skey *skey);
+long dat_reset_skeys(struct dat_tables datt, gfn_t start);
#endif /* KVM_S390_MANAGES_S390_GUEST */
unsigned long dat_get_ptval(struct page_table *table, struct ptval_param param);
void dat_set_ptval(struct page_table *table, struct ptval_param param, unsigned long val);
-int dat_set_slot(struct kvm_s390_mmu_cache *mc, union asce asce, gfn_t start, gfn_t end);
+int dat_set_slot(struct kvm_s390_mmu_cache *mc, struct dat_tables datt, gfn_t start, gfn_t end);
#if KVM_S390_MANAGES_S390_GUEST
-int dat_set_prefix_notif_bit(union asce asce, gfn_t gfn);
+int dat_set_prefix_notif_bit(struct dat_tables datt, gfn_t gfn);
#else
-static inline int dat_set_prefix_notif_bit(union asce asce, gfn_t gfn)
+static inline int dat_set_prefix_notif_bit(struct dat_tables datt, gfn_t gfn)
{
return 0;
}
#endif /* KVM_S390_MANAGES_S390_GUEST */
-bool dat_test_age_gfn(union asce asce, gfn_t start, gfn_t end);
+bool dat_test_age_gfn(struct dat_tables datt, gfn_t start, gfn_t end);
#if KVM_S390_MANAGES_S390_GUEST
-int dat_perform_essa(union asce asce, gfn_t gfn, int orc, union essa_state *state, bool *dirty);
-long dat_reset_cmma(union asce asce, gfn_t start_gfn);
-int dat_peek_cmma(gfn_t start, union asce asce, unsigned int *count, u8 *values);
-int dat_get_cmma(union asce asce, gfn_t *start, unsigned int *count, u8 *values, atomic64_t *rem);
-int dat_set_cmma_bits(struct kvm_s390_mmu_cache *mc, union asce asce, gfn_t gfn,
+int dat_perform_essa(struct dat_tables datt, gfn_t gfn, int orc,
+ union essa_state *state, bool *dirty);
+long dat_reset_cmma(struct dat_tables datt, gfn_t start_gfn);
+int dat_peek_cmma(gfn_t start, struct dat_tables datt, unsigned int *count, u8 *values);
+int dat_get_cmma(struct dat_tables datt, gfn_t *start, unsigned int *count,
+ u8 *values, atomic64_t *rem);
+int dat_set_cmma_bits(struct kvm_s390_mmu_cache *mc, struct dat_tables datt, gfn_t gfn,
unsigned long count, unsigned long mask, const uint8_t *bits);
#endif /* KVM_S390_MANAGES_S390_GUEST */
@@ -597,6 +603,11 @@ static inline struct vsie_rmap *kvm_s390_mmu_cache_alloc_rmap(struct kvm_s390_mm
return kzalloc_obj(struct vsie_rmap, GFP_KVM_S390_MMU_CACHE);
}
+static inline bool has_pgstes(struct dat_tables datt)
+{
+ return KVM_S390_MANAGES_S390_GUEST && datt.has_pgstes;
+}
+
static inline struct crst_table *crste_table_start(union crste *crstep)
{
return (struct crst_table *)ALIGN_DOWN((unsigned long)crstep, _CRST_TABLE_SIZE);
@@ -688,6 +699,11 @@ static inline gfn_t asce_end(union asce asce)
return 1ULL << ((asce.dt + 1) * 11 + _SEGMENT_SHIFT - PAGE_SHIFT);
}
+static inline gfn_t dat_tables_end(struct dat_tables datt)
+{
+ return asce_end(datt.asce);
+}
+
#define _CRSTE(x) ((union crste) { .val = _Generic((x), \
union pgd : (x).val, \
union p4d : (x).val, \
@@ -714,6 +730,11 @@ static inline bool asce_contains_gfn(union asce asce, gfn_t gfn)
return gfn < asce_end(asce);
}
+static inline bool dat_tables_contain_gfn(struct dat_tables datt, gfn_t gfn)
+{
+ return asce_contains_gfn(datt.asce, gfn);
+}
+
static inline bool is_pmd(union crste crste)
{
return crste.h.tt == TABLE_TYPE_SEGMENT;
@@ -879,11 +900,6 @@ static inline struct crst_table *_dereference_crste(union crste crste)
union pgd : _dereference_crste(_CRSTE(x)), \
union crste : _dereference_crste(_CRSTE(x))))
-static inline struct crst_table *dereference_asce(union asce asce)
-{
- return phys_to_virt(asce.val & _ASCE_ORIGIN);
-}
-
static inline void asce_flush_tlb(union asce asce)
{
__tlb_flush_idte(asce.val);
@@ -920,19 +936,18 @@ static inline void pgste_set_unlock(union pte *ptep, union pgste pgste)
WRITE_ONCE(*pgste_of(ptep), pgste);
}
-static inline void dat_ptep_xchg(union pte *ptep, union pte new, gfn_t gfn, union asce asce,
- bool has_skeys)
+static inline void dat_ptep_xchg(union pte *ptep, union pte new, gfn_t gfn, struct dat_tables datt)
{
union pgste pgste;
pgste = pgste_get_lock(ptep);
- pgste = __dat_ptep_xchg(ptep, pgste, new, gfn, asce, has_skeys);
+ pgste = __dat_ptep_xchg(ptep, pgste, new, gfn, datt);
pgste_set_unlock(ptep, pgste);
}
-static inline void dat_ptep_clear(union pte *ptep, gfn_t gfn, union asce asce, bool has_skeys)
+static inline void dat_ptep_clear(union pte *ptep, gfn_t gfn, struct dat_tables datt)
{
- dat_ptep_xchg(ptep, _PTE_EMPTY, gfn, asce, has_skeys);
+ dat_ptep_xchg(ptep, _PTE_EMPTY, gfn, datt);
}
static inline void dat_free_pt(struct page_table *pt)
@@ -999,15 +1014,15 @@ static inline int get_level(union crste *crstep, union pte *ptep)
return ptep ? TABLE_TYPE_PAGE_TABLE : crstep->h.tt;
}
-static inline int dat_delete_slot(union asce asce, gfn_t start, unsigned long npages)
+static inline int dat_delete_slot(struct dat_tables datt, gfn_t start, unsigned long npages)
{
- return dat_set_slot(NULL, asce, start, start + npages);
+ return dat_set_slot(NULL, datt, start, start + npages);
}
-static inline int dat_create_slot(struct kvm_s390_mmu_cache *mc, union asce asce, gfn_t start,
- unsigned long npages)
+static inline int dat_create_slot(struct kvm_s390_mmu_cache *mc, struct dat_tables datt,
+ gfn_t start, unsigned long npages)
{
- return dat_set_slot(mc, asce, start, start + npages);
+ return dat_set_slot(mc, datt, start, start + npages);
}
static inline bool crste_is_ucas(union crste crste)
diff --git a/arch/s390/kvm/gmap/gmap.c b/arch/s390/kvm/gmap/gmap.c
index bf7f27d6be96..cb8f247cff60 100644
--- a/arch/s390/kvm/gmap/gmap.c
+++ b/arch/s390/kvm/gmap/gmap.c
@@ -46,8 +46,8 @@ static int gmap_limit_to_type(gfn_t limit)
*/
struct gmap *gmap_new(struct kvm *kvm, gfn_t limit)
{
+ struct gmap *gmap __free(kfree) = NULL;
struct crst_table *table;
- struct gmap *gmap;
int type;
type = gmap_limit_to_type(limit);
@@ -64,27 +64,27 @@ struct gmap *gmap_new(struct kvm *kvm, gfn_t limit)
refcount_set(&gmap->refcount, 1);
table = dat_alloc_crst_sleepable(_CRSTE_EMPTY(type).val);
- if (!table) {
- kfree(gmap);
+ if (!table)
return NULL;
- }
- gmap->asce.val = __pa(table);
- gmap->asce.dt = type;
- gmap->asce.tl = _ASCE_TABLE_LENGTH;
- gmap->asce.x = 1;
- gmap->asce.p = 1;
- gmap->asce.s = 1;
+ gmap->datt.table = table;
+ gmap->datt.asce.val = __pa(table);
+ gmap->datt.asce.dt = type;
+ gmap->datt.asce.tl = _ASCE_TABLE_LENGTH;
+ gmap->datt.asce.x = 1;
+ gmap->datt.asce.p = 1;
+ gmap->datt.asce.s = 1;
gmap->kvm = kvm;
- set_bit(GMAP_FLAG_OWNS_PAGETABLES, &gmap->flags);
+ gmap->datt.owns_ptes = true;
+ gmap->datt.has_pgstes = KVM_S390_MANAGES_S390_GUEST && !(sclp.has_kss && cpu_has_edat2());
- return gmap;
+ return_ptr(gmap);
}
static void gmap_add_child(struct gmap *parent, struct gmap *child)
{
KVM_BUG_ON(is_ucontrol(parent) && parent->parent, parent->kvm);
- KVM_BUG_ON(is_ucontrol(parent) && !owns_page_tables(parent), parent->kvm);
+ KVM_BUG_ON(is_ucontrol(parent) && !parent->datt.owns_ptes, parent->kvm);
KVM_BUG_ON(!refcount_read(&child->refcount), parent->kvm);
lockdep_assert_held(&parent->children_lock);
@@ -105,8 +105,11 @@ static void gmap_add_child(struct gmap *parent, struct gmap *child)
else
clear_bit(GMAP_FLAG_ALLOW_HPAGE_2G, &child->flags);
- if (kvm_is_ucontrol(parent->kvm))
- clear_bit(GMAP_FLAG_OWNS_PAGETABLES, &child->flags);
+ if (kvm_is_ucontrol(parent->kvm)) {
+ child->datt.owns_ptes = false;
+ child->datt.uses_skeys = true;
+ }
+ child->datt.has_pgstes = true;
list_add(&child->list, &parent->children);
}
@@ -139,7 +142,7 @@ int gmap_set_limit(struct gmap *gmap, gfn_t limit)
if (rc)
return rc;
scoped_guard(write_lock, &gmap->kvm->mmu_lock)
- rc = dat_set_asce_limit(mc, &gmap->asce, type);
+ rc = dat_set_asce_limit(mc, &gmap->datt, type);
} while (rc == -ENOMEM);
return 0;
@@ -196,15 +199,15 @@ void gmap_dispose(struct gmap *gmap)
/* No VSIE shadow block is allowed to use this gmap */
KVM_BUG_ON(!list_empty(&gmap->scb_users), gmap->kvm);
/* The ASCE must be valid */
- KVM_BUG_ON(!gmap->asce.val, gmap->kvm);
+ KVM_BUG_ON(!gmap->datt.asce.val, gmap->kvm);
/* The refcount must be 0 */
KVM_BUG_ON(refcount_read(&gmap->refcount), gmap->kvm);
/* Flush tlb of all gmaps */
- asce_flush_tlb(gmap->asce);
+ asce_flush_tlb(gmap->datt.asce);
/* Free all DAT tables. */
- dat_free_level(dereference_asce(gmap->asce), owns_page_tables(gmap));
+ dat_free_level(gmap->datt.table, gmap->datt.owns_ptes);
/* Free additional data for a shadow gmap */
if (is_shadow(gmap))
@@ -232,21 +235,25 @@ void gmap_dispose(struct gmap *gmap)
int s390_replace_asce(struct gmap *gmap)
{
struct crst_table *table;
- union asce asce;
/* Replacing segment type ASCEs would cause serious issues */
- if (gmap->asce.dt == ASCE_TYPE_SEGMENT)
+ if (gmap->datt.asce.dt == ASCE_TYPE_SEGMENT)
return -EINVAL;
table = dat_alloc_crst_sleepable(0);
if (!table)
return -ENOMEM;
- memcpy(table, dereference_asce(gmap->asce), sizeof(*table));
/* Set new table origin while preserving existing ASCE control bits */
- asce = gmap->asce;
- asce.rsto = virt_to_pfn(table);
- WRITE_ONCE(gmap->asce, asce);
+ scoped_guard(write_lock, &gmap->kvm->mmu_lock) {
+ memcpy(table, gmap->datt.table, sizeof(*table));
+ if (gmap->datt.asce.dt == ASCE_TYPE_SEGMENT) {
+ dat_free_crst(table);
+ return -EINVAL;
+ }
+ gmap->datt.asce.rsto = virt_to_pfn(table);
+ gmap->datt.table = table;
+ }
return 0;
}
@@ -308,7 +315,7 @@ static long gmap_clear_young_pte(union pte *ptep, gfn_t gfn, gfn_t end, struct d
new.h.p = 1;
new.s.prefix_notif = 0;
- pgste = __dat_ptep_xchg(ptep, pgste, new, gfn, walk->asce, uses_skeys(p->gmap));
+ pgste = __dat_ptep_xchg(ptep, pgste, new, gfn, walk->datt);
}
p->young = 1;
pgste_set_unlock(ptep, pgste);
@@ -338,7 +345,7 @@ static long gmap_clear_young_crste(union crste *crstep, gfn_t gfn, gfn_t end, st
folio_set_dirty(phys_to_folio(crste_origin_large(crste)));
new.s.fc1.d = 0;
new.h.p = 1;
- } while (!dat_crstep_xchg_atomic(crstep, crste, new, gfn, walk->asce));
+ } while (!dat_crstep_xchg_atomic(crstep, crste, new, gfn, walk->datt.asce));
priv->young = 1;
return 0;
@@ -365,7 +372,7 @@ bool gmap_age_gfn(struct gmap *gmap, gfn_t start, gfn_t end)
.young = false,
};
- _dat_walk_gfn_range(start, end, gmap->asce, &ops, 0, &priv);
+ _dat_walk_gfn_range(start, end, gmap->datt, &ops, 0, &priv);
return priv.young;
}
@@ -454,7 +461,7 @@ bool gmap_unmap_gfn_range(struct gmap *gmap, struct kvm_memory_slot *slot, gfn_t
lockdep_assert_held_write(&gmap->kvm->mmu_lock);
- _dat_walk_gfn_range(start, end, gmap->asce, &ops, 0, &priv);
+ _dat_walk_gfn_range(start, end, gmap->datt, &ops, 0, &priv);
return false;
}
@@ -536,7 +543,7 @@ void gmap_sync_dirty_log(struct gmap *gmap, gfn_t start, gfn_t end)
lockdep_assert_held(&gmap->kvm->mmu_lock);
- _dat_walk_gfn_range(start, end, gmap->asce, &walk_ops, 0, gmap);
+ _dat_walk_gfn_range(start, end, gmap->datt, &walk_ops, 0, gmap);
}
static int gmap_handle_minor_crste_fault(struct gmap *gmap, struct guest_fault *f)
@@ -618,7 +625,7 @@ int gmap_try_fixup_minor(struct gmap *gmap, struct guest_fault *fault)
lockdep_assert_held(&gmap->kvm->mmu_lock);
- rc = dat_entry_walk(NULL, fault->gfn, gmap->asce, DAT_WALK_LEAF, TABLE_TYPE_PAGE_TABLE,
+ rc = dat_entry_walk(NULL, fault->gfn, gmap->datt, DAT_WALK_LEAF, TABLE_TYPE_PAGE_TABLE,
&fault->crstep, &fault->ptep);
/* If a PTE or a leaf CRSTE could not be reached, slow path. */
if (rc)
@@ -692,7 +699,7 @@ static int _gmap_link(struct kvm_s390_mmu_cache *mc, struct gmap *gmap, int leve
union pgste pgste;
int rc = 0;
- rc = dat_entry_walk(mc, f->gfn, gmap->asce, DAT_WALK_ALLOC_CONTINUE, level,
+ rc = dat_entry_walk(mc, f->gfn, gmap->datt, DAT_WALK_ALLOC_CONTINUE, level,
&f->crstep, &f->ptep);
if (rc == -ENOMEM)
return rc;
@@ -766,10 +773,10 @@ static int gmap_ucas_map_one(struct kvm_s390_mmu_cache *mc, struct gmap *gmap,
int rc;
if (force_alloc)
- rc = dat_entry_walk(mc, p_gfn, gmap->parent->asce, DAT_WALK_ALLOC,
+ rc = dat_entry_walk(mc, p_gfn, gmap->parent->datt, DAT_WALK_ALLOC,
TABLE_TYPE_PAGE_TABLE, &crstep, &ptep);
else
- rc = dat_entry_walk(mc, p_gfn, gmap->parent->asce, DAT_WALK_ALLOC_CONTINUE,
+ rc = dat_entry_walk(mc, p_gfn, gmap->parent->datt, DAT_WALK_ALLOC_CONTINUE,
TABLE_TYPE_SEGMENT, &crstep, &ptep);
if (rc)
return rc;
@@ -782,7 +789,7 @@ static int gmap_ucas_map_one(struct kvm_s390_mmu_cache *mc, struct gmap *gmap,
dat_set_ptval(pt, PTVAL_VMADDR, p_gfn >> (_SEGMENT_SHIFT - PAGE_SHIFT));
newcrste = _crste_fc0(virt_to_pfn(pt), TABLE_TYPE_SEGMENT);
}
- rc = dat_entry_walk(mc, c_gfn, gmap->asce, DAT_WALK_ALLOC, TABLE_TYPE_SEGMENT,
+ rc = dat_entry_walk(mc, c_gfn, gmap->datt, DAT_WALK_ALLOC, TABLE_TYPE_SEGMENT,
&crstep, &ptep);
if (rc)
return rc;
@@ -790,7 +797,7 @@ static int gmap_ucas_map_one(struct kvm_s390_mmu_cache *mc, struct gmap *gmap,
oldcrste = READ_ONCE(*crstep);
if (oldcrste.val == newcrste.val)
break;
- } while (!dat_crstep_xchg_atomic(crstep, oldcrste, newcrste, c_gfn, gmap->asce));
+ } while (!dat_crstep_xchg_atomic(crstep, oldcrste, newcrste, c_gfn, gmap->datt.asce));
return 0;
}
@@ -799,7 +806,7 @@ static int gmap_ucas_translate_simple(struct gmap *gmap, gpa_t *gaddr, union crs
union pte *ptep;
int rc;
- rc = dat_entry_walk(NULL, gpa_to_gfn(*gaddr), gmap->asce, DAT_WALK_CONTINUE,
+ rc = dat_entry_walk(NULL, gpa_to_gfn(*gaddr), gmap->datt, DAT_WALK_CONTINUE,
TABLE_TYPE_SEGMENT, crstepp, &ptep);
if (rc || (!ptep && !crste_is_ucas(**crstepp)))
return -EREMOTE;
@@ -893,10 +900,11 @@ static void gmap_ucas_unmap_one(struct gmap *gmap, gfn_t c_gfn)
union pte *ptep;
int rc;
- rc = dat_entry_walk(NULL, c_gfn, gmap->asce, 0, TABLE_TYPE_SEGMENT, &crstep, &ptep);
+ rc = dat_entry_walk(NULL, c_gfn, gmap->datt, 0, TABLE_TYPE_SEGMENT, &crstep, &ptep);
if (rc)
return;
- while (!dat_crstep_xchg_atomic(crstep, READ_ONCE(*crstep), _PMD_EMPTY, c_gfn, gmap->asce))
+ while (!dat_crstep_xchg_atomic(crstep, READ_ONCE(*crstep), _PMD_EMPTY,
+ c_gfn, gmap->datt.asce))
;
}
@@ -921,7 +929,7 @@ static long _gmap_split_crste(union crste *crstep, gfn_t gfn, gfn_t next, struct
gmap_unmap_prefix(gmap, gfn, next);
if (crste.s.fc1.vsie_notif)
gmap_handle_vsie_unshadow_event(gmap, gfn);
- if (dat_crstep_xchg_atomic(crstep, crste, newcrste, gfn, walk->asce))
+ if (dat_crstep_xchg_atomic(crstep, crste, newcrste, gfn, walk->datt.asce))
break;
crste = READ_ONCE(*crstep);
}
@@ -942,7 +950,7 @@ void gmap_split_huge_pages(struct gmap *gmap)
do {
scoped_guard(read_lock, &gmap->kvm->mmu_lock)
- start = _dat_walk_gfn_range(start, asce_end(gmap->asce), gmap->asce,
+ start = _dat_walk_gfn_range(start, dat_tables_end(gmap->datt), gmap->datt,
&ops, DAT_WALK_IGN_HOLES, gmap);
} while (start);
}
@@ -957,16 +965,17 @@ static int _gmap_enable_skeys(struct gmap *gmap)
if (uses_skeys(gmap))
return 0;
- set_bit(GMAP_FLAG_USES_SKEYS, &gmap->flags);
+ gmap->datt.uses_skeys = true;
+ gmap->datt.has_pgstes = true;
rc = gmap_helper_disable_cow_sharing();
if (rc) {
- clear_bit(GMAP_FLAG_USES_SKEYS, &gmap->flags);
+ gmap->datt.uses_skeys = false;
return rc;
}
do {
scoped_guard(write_lock, &gmap->kvm->mmu_lock)
- start = dat_reset_skeys(gmap->asce, start);
+ start = dat_reset_skeys(gmap->datt, start);
} while (start);
return 0;
}
@@ -1021,7 +1030,7 @@ int gmap_pv_destroy_range(struct gmap *gmap, gfn_t start, gfn_t end, bool interr
do {
scoped_guard(read_lock, &gmap->kvm->mmu_lock)
- start = _dat_walk_gfn_range(start, end, gmap->asce, &ops,
+ start = _dat_walk_gfn_range(start, end, gmap->datt, &ops,
DAT_WALK_IGN_HOLES, NULL);
if (interruptible && fatal_signal_pending(current))
return -EINTR;
@@ -1073,14 +1082,13 @@ int gmap_protect_rmap(struct kvm_s390_mmu_cache *mc, struct gmap *sg, gfn_t p_gf
union pgste pgste;
union pte *ptep;
union pte pte;
- int flags, rc;
+ int rc;
if (KVM_BUG_ON(!is_shadow(sg) || level <= TABLE_TYPE_PAGE_TABLE, sg->kvm))
return -EINVAL;
lockdep_assert_held(&sg->parent->children_lock);
- flags = DAT_WALK_SPLIT_ALLOC | (uses_skeys(sg->parent) ? DAT_WALK_USES_SKEYS : 0);
- rc = dat_entry_walk(mc, p_gfn, sg->parent->asce, flags,
+ rc = dat_entry_walk(mc, p_gfn, sg->parent->datt, DAT_WALK_SPLIT_ALLOC,
TABLE_TYPE_PAGE_TABLE, &crstep, &ptep);
if (rc)
return rc;
@@ -1141,8 +1149,8 @@ void _gmap_set_cmma_all(struct gmap *gmap, bool dirty)
do {
scoped_guard(read_lock, &gmap->kvm->mmu_lock)
- gfn = _dat_walk_gfn_range(gfn, asce_end(gmap->asce), gmap->asce, &ops,
- DAT_WALK_IGN_HOLES,
+ gfn = _dat_walk_gfn_range(gfn, dat_tables_end(gmap->datt), gmap->datt,
+ &ops, DAT_WALK_IGN_HOLES,
&gmap->kvm->arch.cmma_dirty_pages);
} while (gfn);
}
@@ -1160,15 +1168,15 @@ static void gmap_unshadow_level(struct gmap *sg, gfn_t r_gfn, int level)
align = 1UL << (11 * level + _SEGMENT_SHIFT);
kvm_s390_vsie_gmap_notifier(sg, ALIGN_DOWN(gaddr, align), ALIGN(gaddr + 1, align));
sg->invalidated = true;
- if (dat_entry_walk(NULL, r_gfn, sg->asce, 0, level, &crstep, &ptep))
+ if (dat_entry_walk(NULL, r_gfn, sg->datt, 0, level, &crstep, &ptep))
return;
if (ptep) {
if (READ_ONCE(*ptep).val != _PTE_EMPTY.val)
- dat_ptep_xchg(ptep, _PTE_EMPTY, r_gfn, sg->asce, uses_skeys(sg));
+ dat_ptep_xchg(ptep, _PTE_EMPTY, r_gfn, sg->datt);
return;
}
- crste = dat_crstep_clear_atomic(crstep, r_gfn, sg->asce);
+ crste = dat_crstep_clear_atomic(crstep, r_gfn, sg->datt.asce);
if (crste_leaf(crste) || crste.h.i)
return;
if (is_pmd(crste))
diff --git a/arch/s390/kvm/gmap/gmap.h b/arch/s390/kvm/gmap/gmap.h
index 4042e1d4e77c..c40a49b30618 100644
--- a/arch/s390/kvm/gmap/gmap.h
+++ b/arch/s390/kvm/gmap/gmap.h
@@ -18,28 +18,20 @@
* enum gmap_flags - Flags of a gmap.
*
* @GMAP_FLAG_SHADOW: The gmap is a vsie shadow gmap.
- * @GMAP_FLAG_OWNS_PAGETABLES: The gmap owns all dat levels; normally 1, is 0
- * only for ucontrol per-cpu gmaps, since they
- * share the page tables with the main gmap.
* @GMAP_FLAG_IS_UCONTROL: The gmap is ucontrol (main gmap or per-cpu gmap).
* @GMAP_FLAG_ALLOW_HPAGE_1M: 1M hugepages are allowed for this gmap,
* independently of the page size used by userspace.
* @GMAP_FLAG_ALLOW_HPAGE_2G: 2G hugepages are allowed for this gmap,
* independently of the page size used by userspace.
* @GMAP_FLAG_PFAULT_ENABLED: Pfault is enabled for the gmap.
- * @GMAP_FLAG_USES_SKEYS: If the guest uses storage keys.
- * @GMAP_FLAG_USES_CMM: Whether the guest uses CMMA.
* @GMAP_FLAG_EXPORT_ON_UNMAP: Whether to export guest pages when unmapping.
*/
enum gmap_flags {
GMAP_FLAG_SHADOW = 0,
- GMAP_FLAG_OWNS_PAGETABLES,
GMAP_FLAG_IS_UCONTROL,
GMAP_FLAG_ALLOW_HPAGE_1M,
GMAP_FLAG_ALLOW_HPAGE_2G,
GMAP_FLAG_PFAULT_ENABLED,
- GMAP_FLAG_USES_SKEYS,
- GMAP_FLAG_USES_CMM,
GMAP_FLAG_EXPORT_ON_UNMAP,
};
@@ -49,7 +41,7 @@ enum gmap_flags {
* @flags: GMAP_FLAG_* flags.
* @edat_level: The edat level of this shadow gmap.
* @kvm: The vm.
- * @asce: The ASCE used by this gmap.
+ * @datt: The DAT tables used by this gmap.
* @list: List head used in children gmaps for the children gmap list.
* @children_lock: Protects children and scb_users.
* @children: List of child gmaps of this gmap.
@@ -64,7 +56,7 @@ struct gmap {
unsigned char edat_level;
bool invalidated;
struct kvm *kvm;
- union asce asce;
+ struct dat_tables datt;
struct list_head list;
spinlock_t children_lock; /* Protects: children, scb_users */
struct list_head children;
@@ -116,12 +108,12 @@ void gmap_split_huge_pages(struct gmap *gmap);
static inline bool uses_skeys(struct gmap *gmap)
{
- return test_bit(GMAP_FLAG_USES_SKEYS, &gmap->flags);
+ return has_pgstes(gmap->datt) && gmap->datt.uses_skeys;
}
static inline bool uses_cmm(struct gmap *gmap)
{
- return test_bit(GMAP_FLAG_USES_CMM, &gmap->flags);
+ return has_pgstes(gmap->datt) && gmap->datt.uses_cmma;
}
static inline bool pfault_enabled(struct gmap *gmap)
@@ -139,11 +131,6 @@ static inline bool is_shadow(struct gmap *gmap)
return test_bit(GMAP_FLAG_SHADOW, &gmap->flags);
}
-static inline bool owns_page_tables(struct gmap *gmap)
-{
- return test_bit(GMAP_FLAG_OWNS_PAGETABLES, &gmap->flags);
-}
-
static inline struct gmap *gmap_put(struct gmap *gmap)
{
if (refcount_dec_and_test(&gmap->refcount))
@@ -247,7 +234,7 @@ static inline union pgste _gmap_ptep_xchg(struct gmap *gmap, union pte *ptep, un
if (!ptep->s.d && newpte.s.d && !newpte.s.s)
SetPageDirty(pfn_to_page(newpte.h.pfra));
pgste.zero = 0;
- return __dat_ptep_xchg(ptep, pgste, newpte, gfn, gmap->asce, uses_skeys(gmap));
+ return __dat_ptep_xchg(ptep, pgste, newpte, gfn, gmap->datt);
}
static inline union pgste gmap_ptep_xchg(struct gmap *gmap, union pte *ptep, union pte newpte,
@@ -305,7 +292,7 @@ static inline bool __must_check _gmap_crstep_xchg_atomic(struct gmap *gmap, unio
gmap_handle_vsie_unshadow_event(gmap, gfn);
else
_gmap_handle_vsie_unshadow_event(gmap, gfn);
- if (!dat_crstep_xchg_atomic(crstep, oldcrste, newcrste, gfn, gmap->asce))
+ if (!dat_crstep_xchg_atomic(crstep, oldcrste, newcrste, gfn, gmap->datt.asce))
return false;
/*
* Return false even if the swap was successful, as it only
@@ -319,7 +306,7 @@ static inline bool __must_check _gmap_crstep_xchg_atomic(struct gmap *gmap, unio
}
if (!oldcrste.s.fc1.d && newcrste.s.fc1.d && !newcrste.s.fc1.s)
SetPageDirty(phys_to_page(crste_origin_large(newcrste)));
- return dat_crstep_xchg_atomic(crstep, oldcrste, newcrste, gfn, gmap->asce);
+ return dat_crstep_xchg_atomic(crstep, oldcrste, newcrste, gfn, gmap->datt.asce);
}
static inline bool __must_check gmap_crstep_xchg_atomic(struct gmap *gmap, union crste *crstep,
diff --git a/arch/s390/kvm/gmap/kvm_mmu.c b/arch/s390/kvm/gmap/kvm_mmu.c
index c2ffb5e59ec6..784c4aae5995 100644
--- a/arch/s390/kvm/gmap/kvm_mmu.c
+++ b/arch/s390/kvm/gmap/kvm_mmu.c
@@ -72,7 +72,7 @@ int s390_kvm_mmu_prepare_memory_region(struct kvm *kvm,
return -EINVAL;
if ((new->base_gfn + new->npages) * PAGE_SIZE > kvm->arch.mem_limit)
return -EINVAL;
- if (!asce_contains_gfn(kvm->arch.gmap->asce, new->base_gfn + new->npages - 1))
+ if (!dat_tables_contain_gfn(kvm->arch.gmap->datt, new->base_gfn + new->npages - 1))
return -EINVAL;
}
@@ -102,9 +102,9 @@ int s390_kvm_mmu_prepare_memory_region(struct kvm *kvm,
scoped_guard(write_lock, &kvm->mmu_lock) {
kvm_s390_update_cmma_dirty(kvm, old);
if (change == KVM_MR_DELETE || change == KVM_MR_MOVE)
- rc = dat_delete_slot(kvm->arch.gmap->asce, old->base_gfn, old->npages);
+ rc = dat_delete_slot(kvm->arch.gmap->datt, old->base_gfn, old->npages);
if (!rc && (change == KVM_MR_MOVE || change == KVM_MR_CREATE))
- rc = dat_create_slot(mc, kvm->arch.gmap->asce, new->base_gfn, new->npages);
+ rc = dat_create_slot(mc, kvm->arch.gmap->datt, new->base_gfn, new->npages);
}
/*
* Can only be triggered if dat_{create,delete}_slot() found an
diff --git a/arch/s390/kvm/s390/gaccess.c b/arch/s390/kvm/s390/gaccess.c
index 8dc5b93ef01c..eff16f9edfad 100644
--- a/arch/s390/kvm/s390/gaccess.c
+++ b/arch/s390/kvm/s390/gaccess.c
@@ -653,7 +653,7 @@ static int vm_check_access_key_gpa(struct kvm *kvm, u8 access_key,
int r;
scoped_guard(read_lock, &kvm->mmu_lock)
- r = dat_get_storage_key(kvm->arch.gmap->asce, gpa_to_gfn(gpa), &storage_key);
+ r = dat_get_storage_key(kvm->arch.gmap->datt, gpa_to_gfn(gpa), &storage_key);
if (r)
return r;
if (access_key == 0 || storage_key.acc == access_key)
@@ -712,7 +712,7 @@ static int vcpu_check_access_key_gpa(struct kvm_vcpu *vcpu, u8 access_key,
* assume that this cannot fail
*/
scoped_guard(read_lock, &vcpu->kvm->mmu_lock)
- r = dat_get_storage_key(vcpu->arch.gmap->asce, gpa_to_gfn(gpa), &storage_key);
+ r = dat_get_storage_key(vcpu->arch.gmap->datt, gpa_to_gfn(gpa), &storage_key);
if (r)
return r;
/* access key matches storage key -> allow */
@@ -1286,7 +1286,7 @@ static int walk_guest_tables(struct gmap *sg, unsigned long saddr, struct pgtwal
return -EAGAIN;
kvm = parent->kvm;
WARN_ON(!kvm);
- asce = sg->guest_asce;
+ asce.val = sg->guest_asce.val;
entries = get_entries(w);
w->level = LEVEL_MEM;
@@ -1452,7 +1452,7 @@ static int _do_shadow_pte(struct kvm_s390_mmu_cache *mc, struct gmap *sg, gpa_t
newpte = _pte(f->pfn, 0, !p, 0);
if (!pgste_get_trylock(ptep, &pgste))
return -EAGAIN;
- pgste = __dat_ptep_xchg(ptep, pgste, newpte, gpa_to_gfn(raddr), sg->asce, uses_skeys(sg));
+ pgste = __dat_ptep_xchg(ptep, pgste, newpte, gpa_to_gfn(raddr), sg->datt);
pgste_set_unlock(ptep, pgste);
return 0;
@@ -1495,7 +1495,7 @@ static int _do_shadow_crste(struct kvm_s390_mmu_cache *mc, struct gmap *sg, gpa_
return -EAGAIN;
newcrste = _crste_fc1(f->pfn, oldcrste.h.tt, 0, !p);
- while (!dat_crstep_xchg_atomic(table, READ_ONCE(*table), newcrste, r_gfn, sg->asce))
+ while (!dat_crstep_xchg_atomic(table, READ_ONCE(*table), newcrste, r_gfn, sg->datt.asce))
;
return 0;
}
@@ -1504,9 +1504,9 @@ static int _gaccess_do_shadow(struct kvm_s390_mmu_cache *mc, struct gmap *sg,
unsigned long saddr, struct pgtwalk *w)
{
struct guest_fault *entries;
- int flags, i, hl, gl, l, rc;
union crste *table, *host;
union pte *ptep, *ptep_h;
+ int i, hl, gl, l, rc;
lockdep_assert_held(&sg->kvm->mmu_lock);
lockdep_assert_held(&sg->parent->children_lock);
@@ -1515,8 +1515,8 @@ static int _gaccess_do_shadow(struct kvm_s390_mmu_cache *mc, struct gmap *sg,
ptep_h = NULL;
ptep = NULL;
- rc = dat_entry_walk(NULL, gpa_to_gfn(saddr), sg->asce, DAT_WALK_ANY, TABLE_TYPE_PAGE_TABLE,
- &table, &ptep);
+ rc = dat_entry_walk(NULL, gpa_to_gfn(saddr), sg->datt, DAT_WALK_ANY,
+ TABLE_TYPE_PAGE_TABLE, &table, &ptep);
if (rc)
return rc;
@@ -1547,7 +1547,7 @@ static int _gaccess_do_shadow(struct kvm_s390_mmu_cache *mc, struct gmap *sg,
return -EAGAIN;
}
- rc = dat_entry_walk(NULL, entries[LEVEL_MEM].gfn, sg->parent->asce, DAT_WALK_LEAF,
+ rc = dat_entry_walk(NULL, entries[LEVEL_MEM].gfn, sg->parent->datt, DAT_WALK_LEAF,
TABLE_TYPE_PAGE_TABLE, &host, &ptep_h);
if (rc)
return rc;
@@ -1557,16 +1557,16 @@ static int _gaccess_do_shadow(struct kvm_s390_mmu_cache *mc, struct gmap *sg,
l = min3(gl, hl, w->level);
real_address_space:
- flags = DAT_WALK_SPLIT_ALLOC | (uses_skeys(sg->parent) ? DAT_WALK_USES_SKEYS : 0);
/* If necessary, create the shadow mapping */
if (l < gl) {
- rc = dat_entry_walk(mc, gpa_to_gfn(saddr), sg->asce, flags, l, &table, &ptep);
+ rc = dat_entry_walk(mc, gpa_to_gfn(saddr), sg->datt, DAT_WALK_SPLIT_ALLOC, l,
+ &table, &ptep);
if (rc)
return rc;
}
if (l < hl) {
- rc = dat_entry_walk(mc, entries[LEVEL_MEM].gfn, sg->parent->asce,
- flags, l, &host, &ptep_h);
+ rc = dat_entry_walk(mc, entries[LEVEL_MEM].gfn, sg->parent->datt,
+ DAT_WALK_SPLIT_ALLOC, l, &host, &ptep_h);
if (rc)
return rc;
}
diff --git a/arch/s390/kvm/s390/priv.c b/arch/s390/kvm/s390/priv.c
index b3cb2c2c3aa7..1d9e81f6429b 100644
--- a/arch/s390/kvm/s390/priv.c
+++ b/arch/s390/kvm/s390/priv.c
@@ -276,7 +276,7 @@ static int handle_iske(struct kvm_vcpu *vcpu)
gaddr = kvm_s390_logical_to_effective(vcpu, gaddr);
gaddr = kvm_s390_real_to_abs(vcpu, gaddr);
scoped_guard(read_lock, &vcpu->kvm->mmu_lock)
- rc = dat_get_storage_key(vcpu->arch.gmap->asce, gpa_to_gfn(gaddr), &key);
+ rc = dat_get_storage_key(vcpu->arch.gmap->datt, gpa_to_gfn(gaddr), &key);
if (rc > 0)
return kvm_s390_inject_program_int(vcpu, rc);
if (rc < 0)
@@ -308,7 +308,7 @@ static int handle_rrbe(struct kvm_vcpu *vcpu)
gaddr = kvm_s390_logical_to_effective(vcpu, gaddr);
gaddr = kvm_s390_real_to_abs(vcpu, gaddr);
scoped_guard(read_lock, &vcpu->kvm->mmu_lock)
- rc = dat_reset_reference_bit(vcpu->arch.gmap->asce, gpa_to_gfn(gaddr), &skey);
+ rc = dat_reset_reference_bit(vcpu->arch.gmap->datt, gpa_to_gfn(gaddr), &skey);
if (rc > 0)
return kvm_s390_inject_program_int(vcpu, rc);
if (rc < 0)
@@ -360,7 +360,7 @@ static int handle_sske(struct kvm_vcpu *vcpu)
while (start != end) {
scoped_guard(read_lock, &vcpu->kvm->mmu_lock) {
- rc = dat_cond_set_storage_key(vcpu->arch.mc, vcpu->arch.gmap->asce,
+ rc = dat_cond_set_storage_key(vcpu->arch.mc, vcpu->arch.gmap->datt,
gpa_to_gfn(start), key, &oldkey,
m3 & SSKE_NQ, m3 & SSKE_MR, m3 & SSKE_MC);
}
@@ -1118,7 +1118,7 @@ static int handle_pfmf(struct kvm_vcpu *vcpu)
if (rc)
return rc;
scoped_guard(read_lock, &vcpu->kvm->mmu_lock) {
- rc = dat_cond_set_storage_key(vcpu->arch.mc, vcpu->arch.gmap->asce,
+ rc = dat_cond_set_storage_key(vcpu->arch.mc, vcpu->arch.gmap->datt,
gpa_to_gfn(start), key,
NULL, nq, mr, mc);
}
@@ -1167,7 +1167,7 @@ static inline int __do_essa(struct kvm_vcpu *vcpu, const int orc)
gfn = vcpu->run->s.regs.gprs[r2] >> PAGE_SHIFT;
entries = (vcpu->arch.sie_block->cbrlo & ~PAGE_MASK) >> 3;
- nappended = dat_perform_essa(vcpu->arch.gmap->asce, gfn, orc, &state, &dirtied);
+ nappended = dat_perform_essa(vcpu->arch.gmap->datt, gfn, orc, &state, &dirtied);
vcpu->run->s.regs.gprs[r1] = state.val;
if (nappended < 0)
return 0;
@@ -1199,7 +1199,7 @@ static void _essa_clear_cbrl(struct kvm_vcpu *vcpu, unsigned long *cbrl, int len
lockdep_assert_held(&vcpu->kvm->mmu_lock);
for (i = 0; i < len; i++) {
- if (dat_entry_walk(NULL, gpa_to_gfn(cbrl[i]), vcpu->arch.gmap->asce,
+ if (dat_entry_walk(NULL, gpa_to_gfn(cbrl[i]), vcpu->arch.gmap->datt,
0, TABLE_TYPE_PAGE_TABLE, &crstep, &ptep))
continue;
if (!ptep || ptep->s.pr)
@@ -1238,6 +1238,7 @@ static int handle_essa(struct kvm_vcpu *vcpu)
return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
if (!READ_ONCE(vcpu->kvm->arch.migration_mode)) {
+ WRITE_ONCE(vcpu->arch.gmap->datt.has_pgstes, true);
/*
* CMMA is enabled in the KVM settings, but is disabled in
* the SIE block and in the mm_context, and we are not doing
@@ -1247,7 +1248,7 @@ static int handle_essa(struct kvm_vcpu *vcpu)
* value really needs to be written to; if the value is
* already correct, we do nothing and avoid the lock.
*/
- set_bit(GMAP_FLAG_USES_CMM, &vcpu->arch.gmap->flags);
+ WRITE_ONCE(vcpu->arch.gmap->datt.uses_cmma, true);
/*
* If we are here, we are supposed to have CMMA enabled in
* the SIE block. Enabling CMMA works on a per-CPU basis,
diff --git a/arch/s390/kvm/s390/pv.c b/arch/s390/kvm/s390/pv.c
index b18abd0e29ef..d565b02e47b1 100644
--- a/arch/s390/kvm/s390/pv.c
+++ b/arch/s390/kvm/s390/pv.c
@@ -487,7 +487,7 @@ int kvm_s390_pv_set_aside(struct kvm *kvm, u16 *rc, u16 *rrc)
return -EINVAL;
/* Guest with segment type ASCE, refuse to destroy asynchronously */
- if (kvm->arch.gmap->asce.dt == TABLE_TYPE_SEGMENT)
+ if (kvm->arch.gmap->datt.asce.dt == TABLE_TYPE_SEGMENT)
return -EINVAL;
priv = kzalloc_obj(*priv);
@@ -500,7 +500,7 @@ int kvm_s390_pv_set_aside(struct kvm *kvm, u16 *rc, u16 *rrc)
priv->stor_var = kvm->arch.pv.stor_var;
priv->stor_base = kvm->arch.pv.stor_base;
priv->handle = kvm_s390_pv_get_handle(kvm);
- priv->old_gmap_table = (unsigned long)dereference_asce(kvm->arch.gmap->asce);
+ priv->old_gmap_table = (unsigned long)kvm->arch.gmap->datt.table;
if (s390_replace_asce(kvm->arch.gmap))
res = -ENOMEM;
}
@@ -626,7 +626,8 @@ int kvm_s390_pv_deinit_cleanup_all(struct kvm *kvm, u16 *rc, u16 *rrc)
* cleanup has been performed.
*/
if (need_zap && mmget_not_zero(kvm->mm)) {
- gmap_pv_destroy_range(kvm->arch.gmap, 0, asce_end(kvm->arch.gmap->asce), false);
+ gmap_pv_destroy_range(kvm->arch.gmap, 0, dat_tables_end(kvm->arch.gmap->datt),
+ false);
mmput(kvm->mm);
}
@@ -664,7 +665,7 @@ int kvm_s390_pv_deinit_aside_vm(struct kvm *kvm, u16 *rc, u16 *rrc)
return -EINVAL;
/* When a fatal signal is received, stop immediately */
- if (gmap_pv_destroy_range(kvm->arch.gmap, 0, asce_end(kvm->arch.gmap->asce), true))
+ if (gmap_pv_destroy_range(kvm->arch.gmap, 0, dat_tables_end(kvm->arch.gmap->datt), true))
goto done;
if (kvm_s390_pv_dispose_one_leftover(kvm, p, rc, rrc))
ret = -EIO;
@@ -737,7 +738,7 @@ int kvm_s390_pv_init_vm(struct kvm *kvm, u16 *rc, u16 *rrc)
/* Inputs */
uvcb.guest_stor_origin = 0; /* MSO is 0 for KVM */
uvcb.guest_stor_len = kvm->arch.pv.guest_len;
- uvcb.guest_asce = kvm->arch.gmap->asce.val;
+ uvcb.guest_asce = kvm->arch.gmap->datt.asce.val;
uvcb.guest_sca = virt_to_phys(kvm->arch.sca);
uvcb.conf_base_stor_origin =
virt_to_phys((void *)kvm->arch.pv.stor_base);
diff --git a/arch/s390/kvm/s390/s390.c b/arch/s390/kvm/s390/s390.c
index dafcd104d316..280310870dac 100644
--- a/arch/s390/kvm/s390/s390.c
+++ b/arch/s390/kvm/s390/s390.c
@@ -563,25 +563,23 @@ static int kvm_s390_keyop(struct kvm_s390_mmu_cache *mc, struct kvm *kvm, int op
unsigned long addr, union skey skey)
{
gfn_t gfn = gpa_to_gfn(addr);
- union asce asce;
int r;
guard(read_lock)(&kvm->mmu_lock);
- asce = kvm->arch.gmap->asce;
switch (op) {
case KVM_S390_KEYOP_SSKE:
- r = dat_cond_set_storage_key(mc, asce, gfn, skey, &skey, 0, 0, 0);
+ r = dat_cond_set_storage_key(mc, kvm->arch.gmap->datt, gfn, skey, &skey, 0, 0, 0);
if (r == 0 || r == 1)
return skey.skey;
break;
case KVM_S390_KEYOP_ISKE:
- r = dat_get_storage_key(asce, gfn, &skey);
+ r = dat_get_storage_key(kvm->arch.gmap->datt, gfn, &skey);
if (!r)
return skey.skey;
break;
case KVM_S390_KEYOP_RRBE:
- r = dat_reset_reference_bit(asce, gfn, &skey);
+ r = dat_reset_reference_bit(kvm->arch.gmap->datt, gfn, &skey);
if (!r)
return skey.skey;
break;
@@ -1001,7 +999,7 @@ static int kvm_s390_set_mem_control(struct kvm *kvm, struct kvm_device_attr *att
VM_EVENT(kvm, 3, "%s", "RESET: CMMA states");
do {
scoped_guard(read_lock, &kvm->mmu_lock)
- start_gfn = dat_reset_cmma(kvm->arch.gmap->asce, start_gfn);
+ start_gfn = dat_reset_cmma(kvm->arch.gmap->datt, start_gfn);
} while (start_gfn);
ret = 0;
break;
@@ -1048,7 +1046,7 @@ static int kvm_s390_set_mem_control(struct kvm *kvm, struct kvm_device_attr *att
if (ret)
break;
VM_EVENT(kvm, 3, "SET: max guest address: %lu", new_limit);
- VM_EVENT(kvm, 3, "New guest asce: 0x%p", (void *)kvm->arch.gmap->asce.val);
+ VM_EVENT(kvm, 3, "New guest asce: 0x%p", (void *)kvm->arch.gmap->datt.asce.val);
break;
}
default:
@@ -2174,7 +2172,7 @@ static int kvm_s390_get_skeys(struct kvm *kvm, struct kvm_s390_skeys *args)
scoped_guard(read_lock, &kvm->mmu_lock) {
for (i = 0; i < args->count; i++) {
- r = dat_get_storage_key(kvm->arch.gmap->asce,
+ r = dat_get_storage_key(kvm->arch.gmap->datt,
args->start_gfn + i, keys + i);
if (r)
break;
@@ -2241,7 +2239,7 @@ static int kvm_s390_set_skeys(struct kvm *kvm, struct kvm_s390_skeys *args)
break;
scoped_guard(read_lock, &kvm->mmu_lock) {
for (i = 0 ; i < args->count; i++) {
- r = dat_set_storage_key(mc, kvm->arch.gmap->asce,
+ r = dat_set_storage_key(mc, kvm->arch.gmap->datt,
args->start_gfn + i, keys[i], 0);
if (r)
break;
@@ -2295,10 +2293,10 @@ static int kvm_s390_get_cmma_bits(struct kvm *kvm,
scoped_guard(read_lock, &kvm->mmu_lock) {
if (peek)
- ret = dat_peek_cmma(args->start_gfn, kvm->arch.gmap->asce, &args->count,
+ ret = dat_peek_cmma(args->start_gfn, kvm->arch.gmap->datt, &args->count,
values);
else
- ret = dat_get_cmma(kvm->arch.gmap->asce, &args->start_gfn, &args->count,
+ ret = dat_get_cmma(kvm->arch.gmap->datt, &args->start_gfn, &args->count,
values, &kvm->arch.cmma_dirty_pages);
}
@@ -2354,12 +2352,13 @@ static int kvm_s390_set_cmma_bits(struct kvm *kvm,
if (r)
return r;
scoped_guard(read_lock, &kvm->mmu_lock) {
- r = dat_set_cmma_bits(mc, kvm->arch.gmap->asce, args->start_gfn,
+ r = dat_set_cmma_bits(mc, kvm->arch.gmap->datt, args->start_gfn,
args->count, args->mask, bits);
}
} while (r == -ENOMEM);
- set_bit(GMAP_FLAG_USES_CMM, &kvm->arch.gmap->flags);
+ WRITE_ONCE(kvm->arch.gmap->datt.has_pgstes, true);
+ WRITE_ONCE(kvm->arch.gmap->datt.uses_cmma, true);
return r <= 0 ? r : -EFAULT;
}
@@ -3357,7 +3356,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
goto out_err;
set_bit(GMAP_FLAG_IS_UCONTROL, &kvm->arch.gmap->flags);
} else {
- struct crst_table *table = dereference_asce(kvm->arch.gmap->asce);
+ struct crst_table *table = kvm->arch.gmap->datt.table;
crst_table_init((void *)table, _CRSTE_HOLE(table->crstes[0].h.tt).val);
}
@@ -4335,7 +4334,7 @@ static int kvm_s390_fixup_prefix(struct kvm_vcpu *vcpu)
return rc;
scoped_guard(write_lock, &vcpu->kvm->mmu_lock)
- rc = dat_set_prefix_notif_bit(vcpu->kvm->arch.gmap->asce, gfn);
+ rc = dat_set_prefix_notif_bit(vcpu->kvm->arch.gmap->datt, gfn);
return rc;
}
@@ -4813,7 +4812,7 @@ static int __vcpu_run(struct kvm_vcpu *vcpu)
sie_return = kvm_s390_enter_exit_sie(vcpu->arch.sie_block,
vcpu->run->s.regs.gprs,
- vcpu->arch.gmap->asce.val);
+ vcpu->arch.gmap->datt.asce.val);
__enable_cpu_timer_accounting(vcpu);
guest_timing_exit_irqoff();
@@ -5776,7 +5775,7 @@ void kvm_s390_update_cmma_dirty(struct kvm *kvm, const struct kvm_memory_slot *o
if (kvm->arch.migration_mode && kvm->arch.use_cmma && old) {
_dat_walk_gfn_range(old->base_gfn, old->base_gfn + old->npages,
- kvm->arch.gmap->asce, &ops, DAT_WALK_IGN_HOLES,
+ kvm->arch.gmap->datt, &ops, DAT_WALK_IGN_HOLES,
&kvm->arch.cmma_dirty_pages);
}
}
@@ -5842,7 +5841,7 @@ long kvm_arch_vcpu_pre_fault_memory(struct kvm_vcpu *vcpu, struct kvm_pre_fault_
bool kvm_test_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range)
{
scoped_guard(read_lock, &kvm->mmu_lock)
- return dat_test_age_gfn(kvm->arch.gmap->asce, range->start, range->end);
+ return dat_test_age_gfn(kvm->arch.gmap->datt, range->start, range->end);
}
/**
diff --git a/arch/s390/kvm/s390/vsie.c b/arch/s390/kvm/s390/vsie.c
index 5926689fa226..3b5c322bf8b4 100644
--- a/arch/s390/kvm/s390/vsie.c
+++ b/arch/s390/kvm/s390/vsie.c
@@ -1232,7 +1232,8 @@ static int do_vsie_run(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page, struc
goto xfer_to_guest_mode_check;
}
guest_timing_enter_irqoff();
- sie_return = kvm_s390_enter_exit_sie(scb_s, vcpu->run->s.regs.gprs, sg->asce.val);
+ sie_return = kvm_s390_enter_exit_sie(scb_s, vcpu->run->s.regs.gprs,
+ sg->datt.asce.val);
guest_timing_exit_irqoff();
local_irq_enable();
}
--
2.56.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v1 06/10] KVM: s390: Rework some helper functions
2026-10-22 15:55 [PATCH v1 00/10] KVM: s390: Misc bugfixes and improvements for 7.4 Claudio Imbrenda
` (4 preceding siblings ...)
2026-10-22 15:55 ` [PATCH v1 05/10] KVM: s390: Replace ASCE usage with a struct with more flags Claudio Imbrenda
@ 2026-10-22 15:55 ` Claudio Imbrenda
2026-10-22 16:00 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 07/10] KVM: s390: Fix UCONTROL inconsistencies Claudio Imbrenda
` (3 subsequent siblings)
9 siblings, 1 reply; 21+ messages in thread
From: Claudio Imbrenda @ 2026-10-22 15:55 UTC (permalink / raw)
To: kvm; +Cc: linux-s390, borntraeger, frankja, david, nrb, schlameuss, gra
Introduce export_on_unmap() to make the code more readable
Move the #ifdef inside is_ucontrol(), make kvm_is_ucontrol() into a
wrapper around is_ucontrol(). This way the compiler can remove more
code when CONFIG_KVM_S390_UCONTROL is disabled.
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
---
arch/s390/kvm/gmap/gmap.c | 2 +-
arch/s390/kvm/gmap/gmap.h | 9 +++++++++
arch/s390/kvm/s390/s390.h | 6 +-----
3 files changed, 11 insertions(+), 6 deletions(-)
diff --git a/arch/s390/kvm/gmap/gmap.c b/arch/s390/kvm/gmap/gmap.c
index cb8f247cff60..c23da6fad92a 100644
--- a/arch/s390/kvm/gmap/gmap.c
+++ b/arch/s390/kvm/gmap/gmap.c
@@ -414,7 +414,7 @@ static long _gmap_unmap_crste(union crste *crstep, gfn_t gfn, gfn_t next, struct
if (!old.h.fc)
return 0;
- if (old.s.fc1.pr && test_bit(GMAP_FLAG_EXPORT_ON_UNMAP, &priv->gmap->flags))
+ if (old.s.fc1.pr && export_on_unmap(priv->gmap))
folio = phys_to_folio(crste_origin_large(old));
/*
* No races should happen because kvm->mmu_lock is held in write mode,
diff --git a/arch/s390/kvm/gmap/gmap.h b/arch/s390/kvm/gmap/gmap.h
index c40a49b30618..6824507706ed 100644
--- a/arch/s390/kvm/gmap/gmap.h
+++ b/arch/s390/kvm/gmap/gmap.h
@@ -123,7 +123,16 @@ static inline bool pfault_enabled(struct gmap *gmap)
static inline bool is_ucontrol(struct gmap *gmap)
{
+#ifdef CONFIG_KVM_S390_UCONTROL
return test_bit(GMAP_FLAG_IS_UCONTROL, &gmap->flags);
+#else
+ return 0;
+#endif
+}
+
+static inline bool export_on_unmap(struct gmap *gmap)
+{
+ return test_bit(GMAP_FLAG_EXPORT_ON_UNMAP, &gmap->flags);
}
static inline bool is_shadow(struct gmap *gmap)
diff --git a/arch/s390/kvm/s390/s390.h b/arch/s390/kvm/s390/s390.h
index aa0d1d062f8d..daf3b0d004a4 100644
--- a/arch/s390/kvm/s390/s390.h
+++ b/arch/s390/kvm/s390/s390.h
@@ -115,11 +115,7 @@ static inline int is_vcpu_idle(struct kvm_vcpu *vcpu)
static inline int kvm_is_ucontrol(struct kvm *kvm)
{
-#ifdef CONFIG_KVM_S390_UCONTROL
- return test_bit(GMAP_FLAG_IS_UCONTROL, &kvm->arch.gmap->flags);
-#else
- return 0;
-#endif
+ return is_ucontrol(kvm->arch.gmap);
}
#define GUEST_PREFIX_SHIFT 12
--
2.56.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v1 07/10] KVM: s390: Fix UCONTROL inconsistencies
2026-10-22 15:55 [PATCH v1 00/10] KVM: s390: Misc bugfixes and improvements for 7.4 Claudio Imbrenda
` (5 preceding siblings ...)
2026-10-22 15:55 ` [PATCH v1 06/10] KVM: s390: Rework some helper functions Claudio Imbrenda
@ 2026-10-22 15:55 ` Claudio Imbrenda
2026-10-22 17:00 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 08/10] KVM: s390: Improve unmapping of large ranges Claudio Imbrenda
` (2 subsequent siblings)
9 siblings, 1 reply; 21+ messages in thread
From: Claudio Imbrenda @ 2026-10-22 15:55 UTC (permalink / raw)
To: kvm; +Cc: linux-s390, borntraeger, frankja, david, nrb, schlameuss, gra
For UCONTROL VMs, when a vCPU segment entry is mapped but unallocated,
the tl field was set to 1. This collides with the present bit, which
means that other parts of gmap would mistake the entry for a present
one.
Refactor the code to use a different combination of flags to indicate a
mapped but unallocated segment.
Fixes: a2c17f9270cc ("KVM: s390: New gmap code")
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
---
arch/s390/kvm/gmap/dat.h | 16 +++++++++++-
arch/s390/kvm/gmap/gmap.c | 54 +++++++++++++++++++--------------------
2 files changed, 41 insertions(+), 29 deletions(-)
diff --git a/arch/s390/kvm/gmap/dat.h b/arch/s390/kvm/gmap/dat.h
index 5dceb397fc1b..5b03a21c180d 100644
--- a/arch/s390/kvm/gmap/dat.h
+++ b/arch/s390/kvm/gmap/dat.h
@@ -105,6 +105,8 @@ union pte {
#define _REGION3_FR_MASK (_REGION3_MASK >> PAGE_SHIFT)
#define _PAGES_PER_SEGMENT _PAGE_ENTRIES
#define _PAGES_PER_REGION3 (_PAGES_PER_SEGMENT * _CRST_ENTRIES)
+#define _UCAS_ENTRY_LENGTH 0
+#define _UCAS_ENTRY_OFFSET 3
#define CRDTE_NODAT IPTE_NODAT
#define CRDTE_GUEST_ASCE IPTE_GUEST_ASCE
@@ -429,6 +431,17 @@ static inline union crste _crste_fc0(kvm_pfn_t pfn, int tt)
return res;
}
+static inline union crste _crste_ucas(kvm_pfn_t pfn)
+{
+ union crste res = { .val = PFN_PHYS(pfn) };
+
+ res.h.i = 1;
+ res.h.tt = TABLE_TYPE_SEGMENT;
+ res.h.fc0.tl = _UCAS_ENTRY_LENGTH;
+ res.h.fc0.tf = _UCAS_ENTRY_OFFSET;
+ return res;
+}
+
/**
* _crste() - Useful constructor for union crste with FC=1
* @pfn: the pfn this pte should point to.
@@ -1027,7 +1040,8 @@ static inline int dat_create_slot(struct kvm_s390_mmu_cache *mc, struct dat_tabl
static inline bool crste_is_ucas(union crste crste)
{
- return is_pmd(crste) && crste.h.i && crste.h.fc0.tl == 1 && crste.h.fc == 0;
+ return is_pmd(crste) && crste.h.i && !crste.h.fc &&
+ crste.h.fc0.tl == _UCAS_ENTRY_LENGTH && crste.h.fc0.tf == _UCAS_ENTRY_OFFSET;
}
static inline void dat_set_storage_key_from_pgste(union pte pte, union pgste pgste)
diff --git a/arch/s390/kvm/gmap/gmap.c b/arch/s390/kvm/gmap/gmap.c
index c23da6fad92a..c10681f882c2 100644
--- a/arch/s390/kvm/gmap/gmap.c
+++ b/arch/s390/kvm/gmap/gmap.c
@@ -781,16 +781,14 @@ static int gmap_ucas_map_one(struct kvm_s390_mmu_cache *mc, struct gmap *gmap,
if (rc)
return rc;
if (!ptep) {
- newcrste = _crste_fc0(p_gfn, TABLE_TYPE_SEGMENT);
- newcrste.h.i = 1;
- newcrste.h.fc0.tl = 1;
+ newcrste = _crste_ucas(p_gfn);
} else {
pt = pte_table_start(ptep);
dat_set_ptval(pt, PTVAL_VMADDR, p_gfn >> (_SEGMENT_SHIFT - PAGE_SHIFT));
newcrste = _crste_fc0(virt_to_pfn(pt), TABLE_TYPE_SEGMENT);
}
- rc = dat_entry_walk(mc, c_gfn, gmap->datt, DAT_WALK_ALLOC, TABLE_TYPE_SEGMENT,
- &crstep, &ptep);
+ rc = dat_entry_walk(mc, c_gfn, gmap->datt, DAT_WALK_ALLOC | DAT_WALK_IGN_HOLES,
+ TABLE_TYPE_SEGMENT, &crstep, &ptep);
if (rc)
return rc;
do {
@@ -806,8 +804,9 @@ static int gmap_ucas_translate_simple(struct gmap *gmap, gpa_t *gaddr, union crs
union pte *ptep;
int rc;
- rc = dat_entry_walk(NULL, gpa_to_gfn(*gaddr), gmap->datt, DAT_WALK_CONTINUE,
- TABLE_TYPE_SEGMENT, crstepp, &ptep);
+ rc = dat_entry_walk(NULL, gpa_to_gfn(*gaddr), gmap->datt,
+ DAT_WALK_IGN_HOLES | DAT_WALK_ANY,
+ TABLE_TYPE_PAGE_TABLE, crstepp, &ptep);
if (rc || (!ptep && !crste_is_ucas(**crstepp)))
return -EREMOTE;
if (!ptep)
@@ -844,26 +843,24 @@ int gmap_ucas_translate(struct kvm_s390_mmu_cache *mc, struct gmap *gmap, gpa_t
if (rc <= 0)
return rc;
}
- do {
- scoped_guard(write_lock, &gmap->kvm->mmu_lock) {
- rc = gmap_ucas_translate_simple(gmap, gaddr, &crstep);
- if (rc <= 0)
- return rc;
- translated_address = (*gaddr & ~_SEGMENT_MASK) |
- (crstep->val & _SEGMENT_MASK);
- rc = gmap_ucas_map_one(mc, gmap, gpa_to_gfn(translated_address), gfn, true);
- }
- if (!rc) {
- *gaddr = translated_address;
- return 0;
- }
- if (rc != -ENOMEM)
- return -EREMOTE;
- rc = kvm_s390_mmu_cache_topup(mc);
- if (rc)
+ rc = kvm_s390_mmu_cache_topup(mc);
+ if (rc)
+ return rc;
+
+ scoped_guard(write_lock, &gmap->kvm->mmu_lock) {
+ rc = gmap_ucas_translate_simple(gmap, gaddr, &crstep);
+ if (rc <= 0)
return rc;
- } while (1);
- return 0;
+ translated_address = (*gaddr & ~_SEGMENT_MASK) |
+ (crstep->val & _SEGMENT_MASK);
+ rc = gmap_ucas_map_one(mc, gmap, gpa_to_gfn(translated_address), gfn, true);
+ }
+
+ if (!rc)
+ *gaddr = translated_address;
+ if (!rc || rc == -ENOMEM)
+ return rc;
+ return -EREMOTE;
}
int gmap_ucas_map(struct gmap *gmap, gfn_t p_gfn, gfn_t c_gfn, unsigned long count)
@@ -900,8 +897,9 @@ static void gmap_ucas_unmap_one(struct gmap *gmap, gfn_t c_gfn)
union pte *ptep;
int rc;
- rc = dat_entry_walk(NULL, c_gfn, gmap->datt, 0, TABLE_TYPE_SEGMENT, &crstep, &ptep);
- if (rc)
+ rc = dat_entry_walk(NULL, c_gfn, gmap->datt, DAT_WALK_IGN_HOLES,
+ TABLE_TYPE_SEGMENT, &crstep, &ptep);
+ if (rc || !is_pmd(*crstep))
return;
while (!dat_crstep_xchg_atomic(crstep, READ_ONCE(*crstep), _PMD_EMPTY,
c_gfn, gmap->datt.asce))
--
2.56.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v1 08/10] KVM: s390: Improve unmapping of large ranges
2026-10-22 15:55 [PATCH v1 00/10] KVM: s390: Misc bugfixes and improvements for 7.4 Claudio Imbrenda
` (6 preceding siblings ...)
2026-10-22 15:55 ` [PATCH v1 07/10] KVM: s390: Fix UCONTROL inconsistencies Claudio Imbrenda
@ 2026-10-22 15:55 ` Claudio Imbrenda
2026-10-22 16:53 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 09/10] KVM: s390: vsie: Fix pinning using the wrong flag Claudio Imbrenda
2026-10-22 15:55 ` [PATCH v1 10/10] KVM: s390: Enable 1M pages by default Claudio Imbrenda
9 siblings, 1 reply; 21+ messages in thread
From: Claudio Imbrenda @ 2026-10-22 15:55 UTC (permalink / raw)
To: kvm; +Cc: linux-s390, borntraeger, frankja, david, nrb, schlameuss, gra
When unmapping guest ranges, each individual leaf DAT entry is
invalidated. When unmapping large ranges, this results in a very large
amount of invalidations, which is time consuming. Moreover, since the
unmap callback is called by KVM while holding a spinlock, this also
leads to stall warnings.
Improve the situation by invalidating DAT entries higher in the tree,
instead of each leaf individually.
For small invalidations (up to 1 segment = 1M of guest memory), and
special cases (like protected VMs) regardless of the size, the old and
slow invalidation is performed; all leaf entries are invalidated one by
one.
For large invalidations (larger than 1M of guest memory), the new
strategy is used:
- Invalidate prefixes on the whole range, extending start and end to
1M granularity
- Iterate through the page table pointed to by each pmd and unshadow
entries as needed
- Detatch (invalidate) the pmd
- Save storage keys as needed
- Invalidate each (now detached) pte by simply overwriting it
- Reattach the old pmd to point to the updated page table
Fixes: a2c17f9270cc ("KVM: s390: New gmap code")
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
---
arch/s390/kvm/gmap/gmap.c | 110 ++++++++++++++++++++++++++++++++++----
1 file changed, 99 insertions(+), 11 deletions(-)
diff --git a/arch/s390/kvm/gmap/gmap.c b/arch/s390/kvm/gmap/gmap.c
index c10681f882c2..6700a87a420a 100644
--- a/arch/s390/kvm/gmap/gmap.c
+++ b/arch/s390/kvm/gmap/gmap.c
@@ -437,17 +437,8 @@ static long _gmap_unmap_crste(union crste *crstep, gfn_t gfn, gfn_t next, struct
return 0;
}
-/**
- * gmap_unmap_gfn_range() - Unmap a range of guest addresses.
- * @gmap: The gmap to act on.
- * @slot: The memslot in which the range is located.
- * @start: The first gfn to unmap.
- * @end: The gfn after the last one to unmap.
- *
- * Context: Called with the kvm mmu write lock held.
- * Return: false
- */
-bool gmap_unmap_gfn_range(struct gmap *gmap, struct kvm_memory_slot *slot, gfn_t start, gfn_t end)
+static bool gmap_unmap_gfn_range_slow(struct gmap *gmap, struct kvm_memory_slot *slot,
+ gfn_t start, gfn_t end)
{
const struct dat_walk_ops ops = {
.pte_entry = _gmap_unmap_pte,
@@ -465,6 +456,103 @@ bool gmap_unmap_gfn_range(struct gmap *gmap, struct kvm_memory_slot *slot, gfn_t
return false;
}
+static long _gmap_unmap_pud_fast(union crste *crstep, gfn_t gfn, gfn_t next, struct dat_walk *w)
+{
+ union crste old = *crstep;
+
+ /* We only care about large puds */
+ if (!old.h.fc)
+ return 0;
+ if (!gfn)
+ gmap_unmap_prefix(w->priv, 0, next);
+ if (old.s.fc1.vsie_notif)
+ gmap_handle_vsie_unshadow_event(w->priv, gfn);
+
+ idte_crste(crstep, gfn, w->datt.asce);
+ *crstep = _CRSTE_EMPTY(old.h.tt);
+
+ return 0;
+}
+
+static long _gmap_unmap_pmd_fast(union crste *crstep, gfn_t gfn, gfn_t next, struct dat_walk *w)
+{
+ union crste old = *crstep;
+ struct page_table *pt;
+ gfn_t first, last;
+ unsigned int i;
+
+ if (old.h.fc) {
+ if (old.s.fc1.vsie_notif)
+ gmap_handle_vsie_unshadow_event(w->priv, gfn);
+
+ idte_crste(crstep, gfn, w->datt.asce);
+ WRITE_ONCE(*crstep, _CRSTE_EMPTY(old.h.tt));
+
+ return 0;
+ }
+
+ if (old.h.i)
+ return 0;
+
+ pt = dereference_pmd(old.pmd);
+ first = max(gfn, w->start) & (PTRS_PER_PTE - 1);
+ last = (min(next, w->end) - 1) & (PTRS_PER_PTE - 1);
+
+ for (i = first; i <= last; i++)
+ if (pte_needs_unshadow(pt->ptes[i], _PTE_EMPTY, pt->pgstes[i]))
+ gmap_handle_vsie_unshadow_event(w->priv, gfn + i);
+
+ /* Detach the page table */
+ idte_crste(crstep, gfn, w->datt.asce);
+
+ for (i = first; i <= last; i++) {
+ if (!pt->ptes[i].h.i && w->datt.uses_skeys)
+ pt->pgstes[i] = dat_save_storage_key_into_pgste(pt->ptes[i], pt->pgstes[i]);
+ pt->ptes[i] = _PTE_EMPTY;
+ pt->pgstes[i].vsie_notif = 0;
+ pt->pgstes[i].vsie_gmem = 0;
+ }
+
+ /* Reattach the modified page table */
+ WRITE_ONCE(*crstep, old);
+
+ return 0;
+}
+
+static bool gmap_unmap_gfn_range_fast(struct gmap *gmap, gfn_t start, gfn_t end)
+{
+ const struct dat_walk_ops ops = {
+ .pud_entry = _gmap_unmap_pud_fast,
+ .pmd_entry = _gmap_unmap_pmd_fast,
+ };
+ bool failed;
+
+ lockdep_assert_held_write(&gmap->kvm->mmu_lock);
+
+ gmap_unmap_prefix(gmap, ALIGN_DOWN(start, _PAGE_ENTRIES), ALIGN(end, _PAGE_ENTRIES));
+
+ failed = _dat_walk_gfn_range(start, end, gmap->datt, &ops, DAT_WALK_IGN_HOLES, gmap);
+ KVM_BUG_ON(failed, gmap->kvm);
+ return false;
+}
+
+/**
+ * gmap_unmap_gfn_range() - Unmap a range of guest addresses.
+ * @gmap: The gmap to act on.
+ * @slot: The memslot in which the range is located.
+ * @start: The first gfn to unmap.
+ * @end: The gfn after the last one to unmap.
+ *
+ * Context: Called with the kvm mmu write lock held.
+ * Return: false
+ */
+bool gmap_unmap_gfn_range(struct gmap *gmap, struct kvm_memory_slot *slot, gfn_t start, gfn_t end)
+{
+ if ((end - start <= _PAGES_PER_SEGMENT) || export_on_unmap(gmap) || is_ucontrol(gmap))
+ return gmap_unmap_gfn_range_slow(gmap, slot, start, end);
+ return gmap_unmap_gfn_range_fast(gmap, start, end);
+}
+
static union pgste __pte_test_and_clear_softdirty(union pte *ptep, union pgste pgste, gfn_t gfn,
struct gmap *gmap)
{
--
2.56.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v1 09/10] KVM: s390: vsie: Fix pinning using the wrong flag
2026-10-22 15:55 [PATCH v1 00/10] KVM: s390: Misc bugfixes and improvements for 7.4 Claudio Imbrenda
` (7 preceding siblings ...)
2026-10-22 15:55 ` [PATCH v1 08/10] KVM: s390: Improve unmapping of large ranges Claudio Imbrenda
@ 2026-10-22 15:55 ` Claudio Imbrenda
2026-10-22 16:02 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 10/10] KVM: s390: Enable 1M pages by default Claudio Imbrenda
9 siblings, 1 reply; 21+ messages in thread
From: Claudio Imbrenda @ 2026-10-22 15:55 UTC (permalink / raw)
To: kvm; +Cc: linux-s390, borntraeger, frankja, david, nrb, schlameuss, gra
When pinning guest memory using pin_blocks(), gfn_to_page() was used,
which under the hood uses FOLL_GET, which is the wrong type of flag for
memory that is accessed directly by hardware.
Fix by using kvm_vcpu_{,un}map(), which uses FOLL_PIN and is the
correct thing to do.
While technically a bugfix, the previous behaviour can only lead to
guest corruption in extremely convoluted test scenarios that cannot
happen in real/production systems.
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
---
arch/s390/kvm/s390/vsie.c | 132 ++++++++++++++------------------------
1 file changed, 48 insertions(+), 84 deletions(-)
diff --git a/arch/s390/kvm/s390/vsie.c b/arch/s390/kvm/s390/vsie.c
index 3b5c322bf8b4..d2f788bfc956 100644
--- a/arch/s390/kvm/s390/vsie.c
+++ b/arch/s390/kvm/s390/vsie.c
@@ -49,20 +49,21 @@ struct vsie_page {
/* address of the last reported fault to guest2 */
unsigned long fault_addr; /* 0x0228 */
/* calculated guest addresses of satellite control blocks */
- gpa_t sca_gpa; /* 0x0230 */
- gpa_t itdba_gpa; /* 0x0238 */
- gpa_t gvrd_gpa; /* 0x0240 */
- gpa_t riccbd_gpa; /* 0x0248 */
- gpa_t sdnx_gpa; /* 0x0250 */
+ struct kvm_host_map sca_map; /* 0x0230 */
+ struct kvm_host_map itdba_map; /* 0x0260 */
+ struct kvm_host_map gvrd_map; /* 0x0290 */
+ struct kvm_host_map riccbd_map; /* 0x02c0 */
+ struct kvm_host_map sdnx_map; /* 0x02f0 */
+ struct kvm_host_map scb_map; /* 0x0320 */
/*
* guest address of the original SCB. Remains set for free vsie
* pages, so we can properly look them up in our addr_to_page
* radix tree.
*/
- gpa_t scb_gpa; /* 0x0258 */
+ gpa_t scb_gpa; /* 0x0350 */
/* the shadow gmap in use by the vsie_page */
- struct gmap_cache gmap_cache; /* 0x0260 */
- __u8 reserved[0x06f8 - 0x0278]; /* 0x0278 */
+ struct gmap_cache gmap_cache; /* 0x0358 */
+ __u8 reserved[0x06f8 - 0x0370]; /* 0x0370 */
struct kvm_s390_crypto_cb crycb; /* 0x06f8 */
__u8 fac[8 + S390_ARCH_FAC_LIST_SIZE_BYTE];/* 0x07f8 */
};
@@ -681,72 +682,45 @@ static int map_prefix(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page, struct
}
/*
- * Pin the guest page given by gpa and set hpa to the pinned host address.
+ * pin_guest_page() - Pin the given guest page and return its hpa.
+ * @vcpu: the vCPU
+ * @gpa: the gpa to pin
+ * @map: the map representing this pinning.
+ *
* Will always be pinned writable.
*
- * Returns: - 0 on success
- * - -EINVAL if the gpa is not valid guest storage
+ * Returns: - 0 if the gpa is not valid guest storage,
+ * - The host hpa of the pinned page in case of success
*/
-static int pin_guest_page(struct kvm *kvm, gpa_t gpa, hpa_t *hpa)
+static hpa_t pin_guest_page(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *map)
{
- struct page *page;
-
- page = gfn_to_page(kvm, gpa_to_gfn(gpa));
- if (!page)
- return -EINVAL;
- *hpa = (hpa_t)page_to_phys(page) + (gpa & ~PAGE_MASK);
- return 0;
-}
-
-/* Unpins a page previously pinned via pin_guest_page, marking it as dirty. */
-static void unpin_guest_page(struct kvm *kvm, gpa_t gpa, hpa_t hpa)
-{
- kvm_release_page_dirty(pfn_to_page(phys_to_pfn(hpa)));
- /* mark the page always as dirty for migration */
- mark_page_dirty(kvm, gpa_to_gfn(gpa));
+ if (kvm_vcpu_map(vcpu, gpa_to_gfn(gpa), map))
+ return 0;
+ if (!kvm_vcpu_mapped(map))
+ return 0;
+ return (hpa_t)PFN_PHYS(map->pfn) + (gpa & ~PAGE_MASK);
}
/* unpin all blocks previously pinned by pin_blocks(), marking them dirty */
static void unpin_blocks(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)
{
struct kvm_s390_sie_block *scb_s = &vsie_page->scb_s;
- hpa_t hpa;
- hpa = (u64) scb_s->scaoh << 32 | scb_s->scaol;
- if (hpa) {
- unpin_guest_page(vcpu->kvm, vsie_page->sca_gpa, hpa);
- vsie_page->sca_gpa = 0;
- scb_s->scaol = 0;
- scb_s->scaoh = 0;
- }
+ kvm_vcpu_unmap(vcpu, &vsie_page->sca_map);
+ scb_s->scaol = 0;
+ scb_s->scaoh = 0;
- hpa = scb_s->itdba;
- if (hpa) {
- unpin_guest_page(vcpu->kvm, vsie_page->itdba_gpa, hpa);
- vsie_page->itdba_gpa = 0;
- scb_s->itdba = 0;
- }
+ kvm_vcpu_unmap(vcpu, &vsie_page->itdba_map);
+ scb_s->itdba = 0;
- hpa = scb_s->gvrd;
- if (hpa) {
- unpin_guest_page(vcpu->kvm, vsie_page->gvrd_gpa, hpa);
- vsie_page->gvrd_gpa = 0;
- scb_s->gvrd = 0;
- }
+ kvm_vcpu_unmap(vcpu, &vsie_page->gvrd_map);
+ scb_s->gvrd = 0;
- hpa = scb_s->riccbd;
- if (hpa) {
- unpin_guest_page(vcpu->kvm, vsie_page->riccbd_gpa, hpa);
- vsie_page->riccbd_gpa = 0;
- scb_s->riccbd = 0;
- }
+ kvm_vcpu_unmap(vcpu, &vsie_page->riccbd_map);
+ scb_s->riccbd = 0;
- hpa = scb_s->sdnxo;
- if (hpa) {
- unpin_guest_page(vcpu->kvm, vsie_page->sdnx_gpa, hpa);
- vsie_page->sdnx_gpa = 0;
- scb_s->sdnxo = 0;
- }
+ kvm_vcpu_unmap(vcpu, &vsie_page->sdnx_map);
+ scb_s->sdnxo = 0;
}
/*
@@ -783,13 +757,12 @@ static int pin_blocks(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)
((gpa + offsetof(struct bsca_block, cpu[0]) - 1) & PAGE_MASK))
rc = set_validity_icpt(scb_s, 0x003bU);
if (!rc) {
- rc = pin_guest_page(vcpu->kvm, gpa, &hpa);
- if (rc)
+ hpa = pin_guest_page(vcpu, gpa, &vsie_page->sca_map);
+ if (!hpa)
rc = set_validity_icpt(scb_s, 0x0034U);
}
- if (rc)
+ if (rc || !hpa)
goto unpin;
- vsie_page->sca_gpa = gpa;
scb_s->scaoh = (u32)((u64)hpa >> 32);
scb_s->scaol = (u32)(u64)hpa;
}
@@ -801,12 +774,11 @@ static int pin_blocks(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)
goto unpin;
}
/* 256 bytes cannot cross page boundaries */
- rc = pin_guest_page(vcpu->kvm, gpa, &hpa);
- if (rc) {
+ hpa = pin_guest_page(vcpu, gpa, &vsie_page->itdba_map);
+ if (!hpa) {
rc = set_validity_icpt(scb_s, 0x0080U);
goto unpin;
}
- vsie_page->itdba_gpa = gpa;
scb_s->itdba = hpa;
}
@@ -820,12 +792,11 @@ static int pin_blocks(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)
* 512 bytes vector registers cannot cross page boundaries
* if this block gets bigger, we have to shadow it.
*/
- rc = pin_guest_page(vcpu->kvm, gpa, &hpa);
- if (rc) {
+ hpa = pin_guest_page(vcpu, gpa, &vsie_page->gvrd_map);
+ if (!hpa) {
rc = set_validity_icpt(scb_s, 0x1310U);
goto unpin;
}
- vsie_page->gvrd_gpa = gpa;
scb_s->gvrd = hpa;
}
@@ -836,13 +807,12 @@ static int pin_blocks(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)
goto unpin;
}
/* 64 bytes cannot cross page boundaries */
- rc = pin_guest_page(vcpu->kvm, gpa, &hpa);
- if (rc) {
+ hpa = pin_guest_page(vcpu, gpa, &vsie_page->riccbd_map);
+ if (!hpa) {
rc = set_validity_icpt(scb_s, 0x0043U);
goto unpin;
}
/* Validity 0x0044 will be checked by SIE */
- vsie_page->riccbd_gpa = gpa;
scb_s->riccbd = hpa;
}
if (((scb_s->ecb & ECB_GS) && !(scb_s->ecd & ECD_HOSTREGMGMT)) ||
@@ -866,12 +836,11 @@ static int pin_blocks(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)
/* Due to alignment rules (checked above) this cannot
* cross page boundaries
*/
- rc = pin_guest_page(vcpu->kvm, gpa, &hpa);
- if (rc) {
+ hpa = pin_guest_page(vcpu, gpa, &vsie_page->sdnx_map);
+ if (!hpa) {
rc = set_validity_icpt(scb_s, 0x10b0U);
goto unpin;
}
- vsie_page->sdnx_gpa = gpa;
scb_s->sdnxo = hpa | sdnxc;
}
return 0;
@@ -884,10 +853,7 @@ static int pin_blocks(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)
static void unpin_scb(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page,
gpa_t gpa)
{
- hpa_t hpa = virt_to_phys(vsie_page->scb_o);
-
- if (hpa)
- unpin_guest_page(vcpu->kvm, gpa, hpa);
+ kvm_vcpu_unmap(vcpu, &vsie_page->scb_map);
vsie_page->scb_o = NULL;
}
@@ -901,12 +867,10 @@ static int pin_scb(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page,
gpa_t gpa)
{
hpa_t hpa;
- int rc;
- rc = pin_guest_page(vcpu->kvm, gpa, &hpa);
- if (rc) {
- rc = kvm_s390_inject_program_int(vcpu, PGM_ADDRESSING);
- WARN_ON_ONCE(rc);
+ hpa = pin_guest_page(vcpu, gpa, &vsie_page->scb_map);
+ if (!hpa) {
+ WARN_ON_ONCE(kvm_s390_inject_program_int(vcpu, PGM_ADDRESSING));
return 1;
}
vsie_page->scb_o = phys_to_virt(hpa);
--
2.56.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v1 10/10] KVM: s390: Enable 1M pages by default
2026-10-22 15:55 [PATCH v1 00/10] KVM: s390: Misc bugfixes and improvements for 7.4 Claudio Imbrenda
` (8 preceding siblings ...)
2026-10-22 15:55 ` [PATCH v1 09/10] KVM: s390: vsie: Fix pinning using the wrong flag Claudio Imbrenda
@ 2026-10-22 15:55 ` Claudio Imbrenda
2026-10-22 16:11 ` sashiko-bot
9 siblings, 1 reply; 21+ messages in thread
From: Claudio Imbrenda @ 2026-10-22 15:55 UTC (permalink / raw)
To: kvm; +Cc: linux-s390, borntraeger, frankja, david, nrb, schlameuss, gra
When userspace uses hugepages (e.g. hugetlbfs or THP), use hugepages
for the guest mapping by default. Until now the hpage parameter had to
be set manually when loading kvm.
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
---
arch/s390/kvm/s390/s390.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/s390/kvm/s390/s390.c b/arch/s390/kvm/s390/s390.c
index 280310870dac..71d79b4437ac 100644
--- a/arch/s390/kvm/s390/s390.c
+++ b/arch/s390/kvm/s390/s390.c
@@ -212,7 +212,7 @@ module_param(nested, int, S_IRUGO);
MODULE_PARM_DESC(nested, "Nested virtualization support");
/* allow 1m huge page guest backing */
-static int hpage;
+static int hpage = 1;
module_param(hpage, int, 0444);
MODULE_PARM_DESC(hpage, "1m huge page backing support");
--
2.56.0
^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [PATCH v1 01/10] KVM: s390: Move some skey functions to headers
2026-10-22 15:55 ` [PATCH v1 01/10] KVM: s390: Move some skey functions to headers Claudio Imbrenda
@ 2026-10-22 15:58 ` sashiko-bot
0 siblings, 0 replies; 21+ messages in thread
From: sashiko-bot @ 2026-10-22 15:58 UTC (permalink / raw)
To: Claudio Imbrenda
Cc: linux-s390, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
Christian Borntraeger
> Move some storage key handling functions to headers, since they will be
> needed from outside, and they are small, so inlining is acceptable.
>
> Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261022155531.204593-1-imbrenda@linux.ibm.com?part=1
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v1 06/10] KVM: s390: Rework some helper functions
2026-10-22 15:55 ` [PATCH v1 06/10] KVM: s390: Rework some helper functions Claudio Imbrenda
@ 2026-10-22 16:00 ` sashiko-bot
0 siblings, 0 replies; 21+ messages in thread
From: sashiko-bot @ 2026-10-22 16:00 UTC (permalink / raw)
To: Claudio Imbrenda
Cc: linux-s390, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
Christian Borntraeger
> Introduce export_on_unmap() to make the code more readable
>
> Move the #ifdef inside is_ucontrol(), make kvm_is_ucontrol() into a
> wrapper around is_ucontrol(). This way the compiler can remove more
> code when CONFIG_KVM_S390_UCONTROL is disabled.
>
> Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261022155531.204593-1-imbrenda@linux.ibm.com?part=6
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v1 09/10] KVM: s390: vsie: Fix pinning using the wrong flag
2026-10-22 15:55 ` [PATCH v1 09/10] KVM: s390: vsie: Fix pinning using the wrong flag Claudio Imbrenda
@ 2026-10-22 16:02 ` sashiko-bot
0 siblings, 0 replies; 21+ messages in thread
From: sashiko-bot @ 2026-10-22 16:02 UTC (permalink / raw)
To: Claudio Imbrenda
Cc: linux-s390, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
Christian Borntraeger
> When pinning guest memory using pin_blocks(), gfn_to_page() was used,
> which under the hood uses FOLL_GET, which is the wrong type of flag for
> memory that is accessed directly by hardware.
>
> Fix by using kvm_vcpu_{,un}map(), which uses FOLL_PIN and is the
> correct thing to do.
>
> While technically a bugfix, the previous behaviour can only lead to
> guest corruption in extremely convoluted test scenarios that cannot
> happen in real/production systems.
>
> Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261022155531.204593-1-imbrenda@linux.ibm.com?part=9
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v1 02/10] KVM: s390: Refactor and improve DAT tables handling
2026-10-22 15:55 ` [PATCH v1 02/10] KVM: s390: Refactor and improve DAT tables handling Claudio Imbrenda
@ 2026-10-22 16:05 ` sashiko-bot
0 siblings, 0 replies; 21+ messages in thread
From: sashiko-bot @ 2026-10-22 16:05 UTC (permalink / raw)
To: Claudio Imbrenda
Cc: linux-s390, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
Christian Borntraeger
> Refactor and improve functions to handle gmap DAT tables. Handle
> machine_has_tlb_guest() inside the helper functions; the callers won't
> have to worry about it anymore.
>
> Add crdte_pte() to allow using CRDTE on ptes.
>
> Fix idte_crste() to add the missing table-type bits.
>
> Fixes: 5a74e3d93417 ("KVM: s390: KVM-specific bitfields and helper functions")
> Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261022155531.204593-1-imbrenda@linux.ibm.com?part=2
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v1 03/10] KVM: s390: Move prefix marker from PGSTEs to PTEs
2026-10-22 15:55 ` [PATCH v1 03/10] KVM: s390: Move prefix marker from PGSTEs to PTEs Claudio Imbrenda
@ 2026-10-22 16:06 ` sashiko-bot
0 siblings, 0 replies; 21+ messages in thread
From: sashiko-bot @ 2026-10-22 16:06 UTC (permalink / raw)
To: Claudio Imbrenda
Cc: linux-s390, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
Christian Borntraeger
> Move the prefix marker from PGSTEs to PTEs; this is a preparation for
> PGSTE-less guests.
>
> Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261022155531.204593-1-imbrenda@linux.ibm.com?part=3
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v1 05/10] KVM: s390: Replace ASCE usage with a struct with more flags
2026-10-22 15:55 ` [PATCH v1 05/10] KVM: s390: Replace ASCE usage with a struct with more flags Claudio Imbrenda
@ 2026-10-22 16:09 ` sashiko-bot
0 siblings, 0 replies; 21+ messages in thread
From: sashiko-bot @ 2026-10-22 16:09 UTC (permalink / raw)
To: Claudio Imbrenda
Cc: linux-s390, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
Christian Borntraeger
> Replace usage of simple ASCEs with a struct that contains the ASCE, a
> pointer to the table, and some flags.
>
> This way, there will be no need to manually pass the various flags in
> addition to the ASCE where needed. Walking the DAT tables will also not
> need to first convert the physical address to virtual.
>
> Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261022155531.204593-1-imbrenda@linux.ibm.com?part=5
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v1 10/10] KVM: s390: Enable 1M pages by default
2026-10-22 15:55 ` [PATCH v1 10/10] KVM: s390: Enable 1M pages by default Claudio Imbrenda
@ 2026-10-22 16:11 ` sashiko-bot
0 siblings, 0 replies; 21+ messages in thread
From: sashiko-bot @ 2026-10-22 16:11 UTC (permalink / raw)
To: Claudio Imbrenda, kvm
Cc: linux-s390, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
Christian Borntraeger
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] KVM: s390: missing kernel-parameters.txt update for kvm.hpage default change
--
commit fecb3f4de4d4ce9912b0cacb3dd61c76a521ab9f
Author: Claudio Imbrenda <imbrenda@linux.ibm.com>
KVM: s390: Enable 1M pages by default
This commit changes the default value of the kvm.hpage module parameter on
s390 from 0 to 1 so that 1m hugepages are used by default for guest mappings
when userspace uses hugepages.
> diff --git a/arch/s390/kvm/s390/s390.c b/arch/s390/kvm/s390/s390.c
> index 280310870dac3..71d79b4437ac0 100644
> --- a/arch/s390/kvm/s390/s390.c
> +++ b/arch/s390/kvm/s390/s390.c
> @@ -212,7 +212,7 @@ module_param(nested, int, S_IRUGO);
> MODULE_PARM_DESC(nested, "Nested virtualization support");
>
> /* allow 1m huge page guest backing */
> -static int hpage;
> +static int hpage = 1;
> module_param(hpage, int, 0444);
> MODULE_PARM_DESC(hpage, "1m huge page backing support");
>
[Severity: Low]
In arch/s390/kvm/s390/s390.c, hpage is exposed as a kernel command-line and
module parameter (kvm.hpage=) via module_param(hpage, int, 0444), and its
default value is changed here from 0 to 1.
Should this change also update Documentation/admin-guide/kernel-parameters.txt
to document kvm.hpage= (its format, behavior, default value, and s390
architecture applicability), as well as the s390x nested virtualization
section in Documentation/virt/kvm/x86/running-nested-guests.rst?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261022155531.204593-1-imbrenda@linux.ibm.com?part=10
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v1 04/10] KVM: s390: Automatically enable storage keys for UCONTROL VMs
2026-10-22 15:55 ` [PATCH v1 04/10] KVM: s390: Automatically enable storage keys for UCONTROL VMs Claudio Imbrenda
@ 2026-10-22 16:13 ` sashiko-bot
0 siblings, 0 replies; 21+ messages in thread
From: sashiko-bot @ 2026-10-22 16:13 UTC (permalink / raw)
To: Claudio Imbrenda
Cc: linux-s390, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
Christian Borntraeger
> Automatically enable storage keys for UCONTROL VMs upon creation. There
> is no point in leaving it to the lazy initialization, as storage keys
> will almost surely be used very early in the boot process of a UCONTROL
> guest. Furthermore, if storage keys initialization fails, it will
> prevent the VM from starting, instead of failing at runtime.
>
> Opportunistically improve error handling if the fake memslot cannot be
> initialized properly.
>
> Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261022155531.204593-1-imbrenda@linux.ibm.com?part=4
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v1 08/10] KVM: s390: Improve unmapping of large ranges
2026-10-22 15:55 ` [PATCH v1 08/10] KVM: s390: Improve unmapping of large ranges Claudio Imbrenda
@ 2026-10-22 16:53 ` sashiko-bot
0 siblings, 0 replies; 21+ messages in thread
From: sashiko-bot @ 2026-10-22 16:53 UTC (permalink / raw)
To: Claudio Imbrenda, kvm
Cc: linux-s390, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
Christian Borntraeger
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] KVM: s390: hole PTE clobber and stale pgste.zero in _gmap_unmap_pmd_fast()
- [High] KVM: s390: missing compiler barriers around PTE loop in _gmap_unmap_pmd_fast()
--
commit 4a297a39db42bf18a8ae260ff7d2a55c3aa686ed
Author: Claudio Imbrenda <imbrenda@linux.ibm.com>
KVM: s390: Improve unmapping of large ranges
This commit adds a fast path for unmapping guest address ranges larger than
one segment by detaching the pmd via idte_crste(), clearing the pte entries
in the detached page table while saving storage keys, and reattaching the
pmd.
> diff --git a/arch/s390/kvm/gmap/gmap.c b/arch/s390/kvm/gmap/gmap.c
> index c10681f882c2d..6700a87a420a8 100644
> --- a/arch/s390/kvm/gmap/gmap.c
> +++ b/arch/s390/kvm/gmap/gmap.c
[ ... ]
> @@ -465,6 +456,103 @@ bool gmap_unmap_gfn_range(struct gmap *gmap, struct kvm_memory_slot *slot, gfn_t
> return false;
> }
>
> +static long _gmap_unmap_pud_fast(union crste *crstep, gfn_t gfn, gfn_t next, struct dat_walk *w)
> +{
[ ... ]
> +static long _gmap_unmap_pmd_fast(union crste *crstep, gfn_t gfn, gfn_t next, struct dat_walk *w)
> +{
> + union crste old = *crstep;
> + struct page_table *pt;
> + gfn_t first, last;
> + unsigned int i;
> +
> + if (old.h.fc) {
> + if (old.s.fc1.vsie_notif)
> + gmap_handle_vsie_unshadow_event(w->priv, gfn);
> +
> + idte_crste(crstep, gfn, w->datt.asce);
> + WRITE_ONCE(*crstep, _CRSTE_EMPTY(old.h.tt));
> +
> + return 0;
> + }
> +
> + if (old.h.i)
> + return 0;
> +
> + pt = dereference_pmd(old.pmd);
> + first = max(gfn, w->start) & (PTRS_PER_PTE - 1);
> + last = (min(next, w->end) - 1) & (PTRS_PER_PTE - 1);
> +
> + for (i = first; i <= last; i++)
> + if (pte_needs_unshadow(pt->ptes[i], _PTE_EMPTY, pt->pgstes[i]))
> + gmap_handle_vsie_unshadow_event(w->priv, gfn + i);
> +
> + /* Detach the page table */
> + idte_crste(crstep, gfn, w->datt.asce);
> +
> + for (i = first; i <= last; i++) {
> + if (!pt->ptes[i].h.i && w->datt.uses_skeys)
> + pt->pgstes[i] = dat_save_storage_key_into_pgste(pt->ptes[i], pt->pgstes[i]);
> + pt->ptes[i] = _PTE_EMPTY;
> + pt->pgstes[i].vsie_notif = 0;
> + pt->pgstes[i].vsie_gmem = 0;
> + }
[Severity: High]
Can this loop in _gmap_unmap_pmd_fast() clobber hole ptes and leave stale
pgste.zero bits set?
gmap_unmap_gfn_range_fast() passes DAT_WALK_IGN_HOLES to
_dat_walk_gfn_range(). In standard pte walks, dat_pte_walk_range() skips
entries where pte_hole() is true when DAT_WALK_IGN_HOLES is set without
DAT_WALK_ANY:
arch/s390/kvm/gmap/dat.c:dat_pte_walk_range() {
for ( ; gfn < end; idx++, gfn++) {
if (pte_hole(READ_ONCE(table->ptes[idx]))) {
if (!(w->flags & DAT_WALK_IGN_HOLES))
return -EFAULT;
if (!(w->flags & DAT_WALK_ANY))
continue;
}
}
When a non-segment-aligned memslot is deleted or moved via
kvm_set_memslot() -> kvm_invalidate_memslot() ->
kvm_prepare_memory_region() -> s390_kvm_mmu_prepare_memory_region(),
dat_delete_slot() -> _dat_slot_pte() marks the ptes in partially covered
boundary segments as _PTE_TOK(_DAT_TOKEN_PIC, PGM_ADDRESSING) and drops
kvm->mmu_lock while invalid_slot is still present in the active hva tree:
arch/s390/kvm/gmap/dat.c:_dat_slot_pte() {
new_pte = walk->priv ? _PTE_EMPTY : _PTE_TOK(_DAT_TOKEN_PIC, PGM_ADDRESSING);
}
If an mmu notifier invalidation larger than _PAGES_PER_SEGMENT runs on that
hva range before invalid_slot is removed, kvm_handle_hva_range() ->
kvm_unmap_gfn_range() -> gmap_unmap_gfn_range_fast() ->
_gmap_unmap_pmd_fast() iterates over [first, last] and unconditionally
writes _PTE_EMPTY, overwriting those PGM_ADDRESSING hole markers. Would
subsequent storage key or cmma operations (such as dat_get_storage_key(),
dat_set_storage_key(), dat_reset_reference_bit(), or dat_perform_essa()) on
those unmapped gfns then succeed instead of returning PGM_ADDRESSING?
Also, the slow path (_gmap_unmap_pte() -> gmap_ptep_xchg() ->
_gmap_ptep_xchg()) clears pgste.zero:
arch/s390/kvm/gmap/gmap.h:_gmap_ptep_xchg() {
pgste.zero = 0;
return __dat_ptep_xchg(ptep, pgste, newpte, gfn, gmap->datt);
}
Since _gmap_unmap_pmd_fast() only clears vsie_notif and vsie_gmem, could
stale pgste.zero bits remain set and be observed by ESSA_GET_STATE or
ESSA_SET_POT_VOLATILE?
> +
> + /* Reattach the modified page table */
> + WRITE_ONCE(*crstep, old);
[Severity: High]
What prevents the compiler from reordering the pt->ptes[i] and
pt->pgstes[i] accesses outside the detached pmd window in
_gmap_unmap_pmd_fast()?
In arch/s390/kvm/gmap/dat.h, idte_crste() only clobbers *crstep and "cc",
without a "memory" clobber:
arch/s390/kvm/gmap/dat.h:idte_crste() {
if (!machine_has_tlb_guest()) {
/* flush without guest asce */
asm volatile("idte %[table_origin],0,%[gaddr]"
: "+m" (*crstep)
: [table_origin] "a" (table_origin), [gaddr] "a" (gaddr)
: "cc");
} else {
/* flush with guest asce */
asm volatile("idte %[table_origin],%[asce],%[gaddr_opt]"
: "+m" (*crstep)
: [table_origin] "a" (table_origin),
[gaddr_opt] "a" (gaddr | IDTE_GUEST_ASCE),
[asce] "a" (asce.val)
: "cc");
}
}
Neither page_get_storage_key() nor WRITE_ONCE(*crstep, old) includes a
compiler memory barrier either (unlike pgste_set_unlock() in
arch/s390/kvm/gmap/dat.h, which places barrier() before WRITE_ONCE()).
Because pt->ptes[i] and pt->pgstes[i] are plain non-volatile accesses to
types distinct from *crstep (union crste), could the compiler reorder the
pte/pgste updates before idte_crste() or after WRITE_ONCE(*crstep, old),
allowing running vcpus to retain stale tlb entries to unmapped pages or
update hardware storage keys after dat_save_storage_key_into_pgste() has
read them?
> +
> + return 0;
> +}
> +
> +static bool gmap_unmap_gfn_range_fast(struct gmap *gmap, gfn_t start, gfn_t end)
> +{
> + const struct dat_walk_ops ops = {
> + .pud_entry = _gmap_unmap_pud_fast,
> + .pmd_entry = _gmap_unmap_pmd_fast,
> + };
> + bool failed;
> +
> + lockdep_assert_held_write(&gmap->kvm->mmu_lock);
> +
> + gmap_unmap_prefix(gmap, ALIGN_DOWN(start, _PAGE_ENTRIES), ALIGN(end, _PAGE_ENTRIES));
> +
> + failed = _dat_walk_gfn_range(start, end, gmap->datt, &ops, DAT_WALK_IGN_HOLES, gmap);
> + KVM_BUG_ON(failed, gmap->kvm);
> + return false;
> +}
[ ... ]
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261022155531.204593-1-imbrenda@linux.ibm.com?part=8
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v1 07/10] KVM: s390: Fix UCONTROL inconsistencies
2026-10-22 15:55 ` [PATCH v1 07/10] KVM: s390: Fix UCONTROL inconsistencies Claudio Imbrenda
@ 2026-10-22 17:00 ` sashiko-bot
0 siblings, 0 replies; 21+ messages in thread
From: sashiko-bot @ 2026-10-22 17:00 UTC (permalink / raw)
To: Claudio Imbrenda, kvm
Cc: linux-s390, Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
Christian Borntraeger
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] KVM: s390: _crste_ucas() collides with token encoding in _crste_hole()
--
commit 709d72ff37a22ce1d8ae00a0f230ee566c74c804
Author: Claudio Imbrenda <imbrenda@linux.ibm.com>
KVM: s390: Fix UCONTROL inconsistencies
This commit changes the flags used in union crste to represent mapped but
unallocated UCONTROL segment entries so that they do not collide with the
present bit. It introduces _crste_ucas() and updates crste_is_ucas() and
the UCONTROL gmap helpers accordingly.
> diff --git a/arch/s390/kvm/gmap/dat.h b/arch/s390/kvm/gmap/dat.h
> index 5dceb397fc1b8..5b03a21c180da 100644
> --- a/arch/s390/kvm/gmap/dat.h
> +++ b/arch/s390/kvm/gmap/dat.h
> @@ -105,6 +105,8 @@ union pte {
> #define _REGION3_FR_MASK (_REGION3_MASK >> PAGE_SHIFT)
> #define _PAGES_PER_SEGMENT _PAGE_ENTRIES
> #define _PAGES_PER_REGION3 (_PAGES_PER_SEGMENT * _CRST_ENTRIES)
> +#define _UCAS_ENTRY_LENGTH 0
> +#define _UCAS_ENTRY_OFFSET 3
>
> #define CRDTE_NODAT IPTE_NODAT
> #define CRDTE_GUEST_ASCE IPTE_GUEST_ASCE
> @@ -429,6 +431,17 @@ static inline union crste _crste_fc0(kvm_pfn_t pfn, int tt)
> return res;
> }
>
> +static inline union crste _crste_ucas(kvm_pfn_t pfn)
> +{
> + union crste res = { .val = PFN_PHYS(pfn) };
> +
> + res.h.i = 1;
> + res.h.tt = TABLE_TYPE_SEGMENT;
> + res.h.fc0.tl = _UCAS_ENTRY_LENGTH;
> + res.h.fc0.tf = _UCAS_ENTRY_OFFSET;
> + return res;
> +}
[Severity: Medium]
Does _crste_ucas() collide with the token encoding checked by _crste_hole()
and _crste_none()?
In _crste_ucas(), res.h.i is 1 and res.h.fc0.tl is 0, which means bit 63
(res.tok.pr and res.s.fc1.pr) is 0 while bits 0..43 of res.val hold the
segment-aligned host address PFN_PHYS(pfn).
In arch/s390/kvm/gmap/dat.h, _crste_hole() and _crste_none() do not check
crste_is_ucas() or crste.h.fc0.tf:
static inline bool _crste_hole(union crste crste)
{
return crste.h.i && !crste.tok.pr && crste.tok.type != _DAT_TOKEN_NONE;
}
static inline bool _crste_none(union crste crste)
{
return crste.h.i && !crste.tok.pr && crste.tok.type == _DAT_TOKEN_NONE;
}
When a UCONTROL segment is mapped via KVM_S390_UCAS_MAP with a host
userspace address at or above 0x0001000000000000UL (so bits 0..15,
crste.tok.type, are non-zero) before the parent segment is faulted in,
_crste_hole() returns true for the _crste_ucas() entry.
While this patch passes DAT_WALK_IGN_HOLES in gmap_ucas_map_one(),
gmap_ucas_translate_simple(), and gmap_ucas_unmap_one(), other walkers of
vcpu->arch.gmap->datt do not. For example:
kvm_arch_vcpu_ioctl()
-> kvm_s390_vcpu_memsida_op()
-> kvm_s390_vcpu_mem_op()
-> check_gva_range()
-> guest_range_to_gpas()
-> vcpu_check_access_key_gpa()
-> dat_get_storage_key()
In arch/s390/kvm/gmap/dat.c:dat_get_storage_key():
rc = dat_entry_walk(NULL, gfn, datt, DAT_WALK_ANY, TABLE_TYPE_PAGE_TABLE, &crstep, &ptep);
if (rc)
return rc;
And in arch/s390/kvm/gmap/dat.c:dat_entry_walk():
if (crste_hole(entry) && !ign_holes)
return entry.tok.type == _DAT_TOKEN_PIC ? entry.tok.par : -EFAULT;
Instead of returning storage key 0 for the unallocated segment,
dat_get_storage_key() will fail with -EFAULT (or return bits 16..31 of the
host address in entry.tok.par if entry.tok.type happens to equal
_DAT_TOKEN_PIC). Should _crste_hole() and _crste_none() exclude
crste_is_ucas(crste)?
[ ... ]
> @@ -1027,7 +1040,8 @@ static inline int dat_create_slot(struct kvm_s390_mmu_cache *mc, struct dat_tabl
>
> static inline bool crste_is_ucas(union crste crste)
> {
> - return is_pmd(crste) && crste.h.i && crste.h.fc0.tl == 1 && crste.h.fc == 0;
> + return is_pmd(crste) && crste.h.i && !crste.h.fc &&
> + crste.h.fc0.tl == _UCAS_ENTRY_LENGTH && crste.h.fc0.tf == _UCAS_ENTRY_OFFSET;
> }
>
> static inline void dat_set_storage_key_from_pgste(union pte pte, union pgste pgste)
[ ... ]
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261022155531.204593-1-imbrenda@linux.ibm.com?part=7
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2026-10-22 17:00 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-10-22 15:55 [PATCH v1 00/10] KVM: s390: Misc bugfixes and improvements for 7.4 Claudio Imbrenda
2026-10-22 15:55 ` [PATCH v1 01/10] KVM: s390: Move some skey functions to headers Claudio Imbrenda
2026-10-22 15:58 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 02/10] KVM: s390: Refactor and improve DAT tables handling Claudio Imbrenda
2026-10-22 16:05 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 03/10] KVM: s390: Move prefix marker from PGSTEs to PTEs Claudio Imbrenda
2026-10-22 16:06 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 04/10] KVM: s390: Automatically enable storage keys for UCONTROL VMs Claudio Imbrenda
2026-10-22 16:13 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 05/10] KVM: s390: Replace ASCE usage with a struct with more flags Claudio Imbrenda
2026-10-22 16:09 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 06/10] KVM: s390: Rework some helper functions Claudio Imbrenda
2026-10-22 16:00 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 07/10] KVM: s390: Fix UCONTROL inconsistencies Claudio Imbrenda
2026-10-22 17:00 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 08/10] KVM: s390: Improve unmapping of large ranges Claudio Imbrenda
2026-10-22 16:53 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 09/10] KVM: s390: vsie: Fix pinning using the wrong flag Claudio Imbrenda
2026-10-22 16:02 ` sashiko-bot
2026-10-22 15:55 ` [PATCH v1 10/10] KVM: s390: Enable 1M pages by default Claudio Imbrenda
2026-10-22 16:11 ` sashiko-bot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox