[XEN][PATCH 0/8] xen/arm64: make aarch32 support optional

[XEN][PATCH 0/8] xen/arm64: make aarch32 support optional

[Grygorii Strashko]

From: Grygorii Strashko <grygorii_strashko@epam.com>

Hi,

Now Arm64 AArch32 guest support is always enabled and built-in while not
all Arm64 platforms supports AArch32 or this support might not be needed.

Hence, this series introduces basic support for disabling Arm64 AArch32
guest support by introducing Kconfig option CONFIG_ARM64_AARCH32 to allow enable/disable
Arm64 AArch32 guest support (default y). 

Patches 1-4 Prerequisite patches
Patch 5 - Introduces Kconfig option CONFIG_ARM64_AARCH32 and prevents creating domains
          running Arm64 AArch32 guests if CONFIG_ARM64_AARCH32=n
Patches 6-8 - enables build-time optimization of AArch32 specific code by redefining some
              macro (like is_32/64bit_domain()) as constants

Grygorii Strashko (8):
  xen/arm: split set_domain_type() between arm64/arm32
  xen/arm: move vcpu_switch_to_aarch64_mode() in arch_vcpu_create()
  xen/arm: move vcpu_switch_to_aarch64_mode() in arm64
  xen/arm: split is_32bit/64bit_domain() between arm64/arm32
  xen/arm64: make aarch32 support optional
  xen/arm64: constify is_32/64bit_domain() macro for CONFIG_ARM64_AARCH32=n
  xen/arm: regs.h split subarch definitions between arm64/arm32
  xen/arm64: constify regs_mode_is_32bit macro for CONFIG_ARM64_AARCH32=n

 xen/arch/arm/Kconfig                       |  7 +++
 xen/arch/arm/arm32/Makefile                |  1 +
 xen/arch/arm/arm32/domain-build.c          | 22 +++++++
 xen/arch/arm/arm64/Makefile                |  1 +
 xen/arch/arm/arm64/domain-build.c          | 65 ++++++++++++++++++++
 xen/arch/arm/arm64/domain.c                |  5 ++
 xen/arch/arm/arm64/domctl.c                |  6 ++
 xen/arch/arm/dom0less-build.c              | 14 -----
 xen/arch/arm/domain.c                      |  9 ++-
 xen/arch/arm/domain_build.c                | 33 +++-------
 xen/arch/arm/include/asm/arm32/domain.h    | 28 +++++++++
 xen/arch/arm/include/asm/arm32/processor.h |  5 ++
 xen/arch/arm/include/asm/arm64/domain.h    | 71 ++++++++++++++++++++++
 xen/arch/arm/include/asm/arm64/processor.h | 19 ++++++
 xen/arch/arm/include/asm/domain.h          |  9 ++-
 xen/arch/arm/include/asm/regs.h            | 24 --------
 xen/arch/arm/setup.c                       |  2 +-
 xen/common/device-tree/dom0less-build.c    |  6 +-
 xen/include/asm-generic/dom0less-build.h   |  9 +++
 19 files changed, 258 insertions(+), 78 deletions(-)
 create mode 100644 xen/arch/arm/arm32/domain-build.c
 create mode 100644 xen/arch/arm/arm64/domain-build.c
 create mode 100644 xen/arch/arm/include/asm/arm32/domain.h
 create mode 100644 xen/arch/arm/include/asm/arm64/domain.h

-- 
2.34.1

[XEN][PATCH 2/8] xen/arm: move vcpu_switch_to_aarch64_mode() in arch_vcpu_create()

[Grygorii Strashko]

From: Grygorii Strashko <grygorii_strashko@epam.com>

Move vcpu_switch_to_aarch64_mode() in arch_vcpu_create() callback instead
of calling it manually from few different places after vcpu_create().

Before doing above ensure vcpu0 is created after kernel_probe() is done and
domain's guest execution mode (32-bit/64-bit) is set for dom0 and dom0less
domains.

Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
---
 xen/arch/arm/domain.c                    |  3 +++
 xen/arch/arm/domain_build.c              | 13 +++++--------
 xen/common/device-tree/dom0less-build.c  |  6 +++---
 xen/include/asm-generic/dom0less-build.h |  2 +-
 4 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 79a144e61be9..bbd4a764c696 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -586,6 +586,9 @@ int arch_vcpu_create(struct vcpu *v)
     if ( get_ssbd_state() == ARM_SSBD_RUNTIME )
         v->arch.cpu_info->flags |= CPUINFO_WORKAROUND_2_FLAG;
 
+    if ( is_64bit_domain(v->domain) )
+        vcpu_switch_to_aarch64_mode(v);
+
     return rc;
 
 fail:
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index d91a71acfd3b..af7e9d830ae1 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1885,10 +1885,6 @@ int __init construct_domain(struct domain *d, struct kernel_info *kinfo)
         printk("SVE is not available for 32-bit domain\n");
         return -EINVAL;
     }
-
-    if ( is_64bit_domain(d) )
-        vcpu_switch_to_aarch64_mode(v);
-
 #endif
 
     /*
@@ -1941,9 +1937,6 @@ int __init construct_domain(struct domain *d, struct kernel_info *kinfo)
             printk("Failed to allocate d%dv%d\n", d->domain_id, i);
             break;
         }
-
-        if ( is_64bit_domain(d) )
-            vcpu_switch_to_aarch64_mode(d->vcpu[i]);
     }
 
     domain_update_node_affinity(d);
@@ -1995,9 +1988,13 @@ int __init construct_hwdom(struct kernel_info *kinfo,
     iommu_hwdom_init(d);
 
 #ifdef CONFIG_ARM_64
-    /* type must be set before allocate_memory */
+    /* type must be set before allocate_memory or create cpu */
     d->arch.type = kinfo->arch.type;
 #endif
+
+    if ( vcpu_create(d, 0) == NULL )
+        panic("Error creating domain 0 vcpu0\n");
+
     find_gnttab_region(d, kinfo);
     if ( is_domain_direct_mapped(d) )
         allocate_memory_11(d, kinfo);
diff --git a/xen/common/device-tree/dom0less-build.c b/xen/common/device-tree/dom0less-build.c
index efa846da2a55..f02ce6c776de 100644
--- a/xen/common/device-tree/dom0less-build.c
+++ b/xen/common/device-tree/dom0less-build.c
@@ -771,9 +771,6 @@ static int __init construct_domU(struct domain *d,
     else if ( rc == 0 && !strcmp(dom0less_enhanced, "no-xenstore") )
         kinfo.dom0less_feature = DOM0LESS_ENHANCED_NO_XS;
 
-    if ( vcpu_create(d, 0) == NULL )
-        return -ENOMEM;
-
     d->max_pages = ((paddr_t)mem * SZ_1K) >> PAGE_SHIFT;
 
     kinfo.bd.d = d;
@@ -792,6 +789,9 @@ static int __init construct_domU(struct domain *d,
     }
     else
     {
+        if ( vcpu_create(d, 0) == NULL )
+            return -ENOMEM;
+
         if ( !dt_find_property(node, "xen,static-mem", NULL) )
             allocate_memory(d, &kinfo);
         else if ( !is_domain_direct_mapped(d) )
diff --git a/xen/include/asm-generic/dom0less-build.h b/xen/include/asm-generic/dom0less-build.h
index 6b80ffbd8679..13616975b3ca 100644
--- a/xen/include/asm-generic/dom0less-build.h
+++ b/xen/include/asm-generic/dom0less-build.h
@@ -59,7 +59,7 @@ int make_arch_nodes(struct kernel_info *kinfo);
 /*
  * Set domain type from struct kernel_info which defines guest Execution
  * State 32-bit/64-bit (for Arm AArch32/AArch64).
- * The domain type must be set before allocate_memory.
+ * The domain type must be set before allocate_memory or create vcpus.
  *
  * @d: pointer to the domain structure.
  * @kinfo: pointer to the kinfo structure.
-- 
2.34.1

[XEN][PATCH 1/8] xen/arm: split set_domain_type() between arm64/arm32

[Grygorii Strashko]

From: Grygorii Strashko <grygorii_strashko@epam.com>

Split set_domain_type() between Arm64/Arm32 sub-arches as
set_domain_type() implementation is going to be extended for Arm64.

Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
---
 xen/arch/arm/arm32/Makefile              |  1 +
 xen/arch/arm/arm32/domain-build.c        | 22 ++++++++++++++++++++++
 xen/arch/arm/arm64/Makefile              |  1 +
 xen/arch/arm/arm64/domain-build.c        | 24 ++++++++++++++++++++++++
 xen/arch/arm/dom0less-build.c            | 14 --------------
 xen/include/asm-generic/dom0less-build.h |  9 +++++++++
 6 files changed, 57 insertions(+), 14 deletions(-)
 create mode 100644 xen/arch/arm/arm32/domain-build.c
 create mode 100644 xen/arch/arm/arm64/domain-build.c

diff --git a/xen/arch/arm/arm32/Makefile b/xen/arch/arm/arm32/Makefile
index 531168f58a0a..0fd3f5272361 100644
--- a/xen/arch/arm/arm32/Makefile
+++ b/xen/arch/arm/arm32/Makefile
@@ -6,6 +6,7 @@ obj-y += cache.o
 obj-$(CONFIG_EARLY_PRINTK) += debug.o
 obj-y += domctl.o
 obj-y += domain.o
+obj-y += domain-build.o
 obj-y += entry.o
 obj-y += head.o
 obj-y += insn.o
diff --git a/xen/arch/arm/arm32/domain-build.c b/xen/arch/arm/arm32/domain-build.c
new file mode 100644
index 000000000000..e34261e4a2ad
--- /dev/null
+++ b/xen/arch/arm/arm32/domain-build.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#include <xen/fdt-kernel.h>
+#include <xen/sched.h>
+
+#include <asm/domain.h>
+
+#ifdef CONFIG_DOM0LESS_BOOT
+void __init set_domain_type(struct domain *d, struct kernel_info *kinfo)
+{
+    /* Nothing to do */
+}
+#endif
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/arm64/Makefile b/xen/arch/arm/arm64/Makefile
index 6491c5350b2e..3272fe7e4ca2 100644
--- a/xen/arch/arm/arm64/Makefile
+++ b/xen/arch/arm/arm64/Makefile
@@ -8,6 +8,7 @@ obj-$(CONFIG_HARDEN_BRANCH_PREDICTOR) += bpi.o
 obj-$(CONFIG_EARLY_PRINTK) += debug.o
 obj-y += domctl.o
 obj-y += domain.o
+obj-y += domain-build.o
 obj-y += entry.o
 obj-y += head.o
 obj-y += insn.o
diff --git a/xen/arch/arm/arm64/domain-build.c b/xen/arch/arm/arm64/domain-build.c
new file mode 100644
index 000000000000..3a89ee46b8c6
--- /dev/null
+++ b/xen/arch/arm/arm64/domain-build.c
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#include <xen/fdt-kernel.h>
+#include <xen/sched.h>
+
+#include <asm/domain.h>
+
+#ifdef CONFIG_DOM0LESS_BOOT
+/* TODO: make arch.type generic ? */
+void __init set_domain_type(struct domain *d, struct kernel_info *kinfo)
+{
+    /* type must be set before allocate memory */
+    d->arch.type = kinfo->arch.type;
+}
+#endif
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 69b9ea22ce32..c4b1c2915719 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -236,20 +236,6 @@ int __init make_arch_nodes(struct kernel_info *kinfo)
     return 0;
 }
 
-/* TODO: make arch.type generic ? */
-#ifdef CONFIG_ARM_64
-void __init set_domain_type(struct domain *d, struct kernel_info *kinfo)
-{
-    /* type must be set before allocate memory */
-    d->arch.type = kinfo->arch.type;
-}
-#else
-void __init set_domain_type(struct domain *d, struct kernel_info *kinfo)
-{
-    /* Nothing to do */
-}
-#endif
-
 int __init init_vuart(struct domain *d, struct kernel_info *kinfo,
                       const struct dt_device_node *node)
 {
diff --git a/xen/include/asm-generic/dom0less-build.h b/xen/include/asm-generic/dom0less-build.h
index e0ad0429ec74..6b80ffbd8679 100644
--- a/xen/include/asm-generic/dom0less-build.h
+++ b/xen/include/asm-generic/dom0less-build.h
@@ -56,6 +56,15 @@ int init_vuart(struct domain *d, struct kernel_info *kinfo,
 int make_intc_domU_node(struct kernel_info *kinfo);
 int make_arch_nodes(struct kernel_info *kinfo);
 
+/*
+ * Set domain type from struct kernel_info which defines guest Execution
+ * State 32-bit/64-bit (for Arm AArch32/AArch64).
+ * The domain type must be set before allocate_memory.
+ *
+ * @d: pointer to the domain structure.
+ * @kinfo: pointer to the kinfo structure.
+ */
+
 void set_domain_type(struct domain *d, struct kernel_info *kinfo);
 
 int init_intc_phandle(struct kernel_info *kinfo, const char *name,
-- 
2.34.1

[XEN][PATCH 3/8] xen/arm: move vcpu_switch_to_aarch64_mode() in arm64

[Grygorii Strashko]

From: Grygorii Strashko <grygorii_strashko@epam.com>

The vcpu_switch_to_aarch64_mode() is Arm64 specific, so move it in Arm64.
As part of this change:
- introduce arm32/arm64 domain.h headers and include them in asm/domain.h
basing on CONFIG_ARM_xx;
- declare vcpu_switch_to_aarch64_mode() for arm64;
- add vcpu_switch_to_aarch64_mode() as empty macro for arm32.

Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
---
 xen/arch/arm/arm64/domain.c             |  5 +++++
 xen/arch/arm/domain.c                   |  5 -----
 xen/arch/arm/include/asm/arm32/domain.h | 17 +++++++++++++++++
 xen/arch/arm/include/asm/arm64/domain.h | 22 ++++++++++++++++++++++
 xen/arch/arm/include/asm/domain.h       |  3 ++-
 5 files changed, 46 insertions(+), 6 deletions(-)
 create mode 100644 xen/arch/arm/include/asm/arm32/domain.h
 create mode 100644 xen/arch/arm/include/asm/arm64/domain.h

diff --git a/xen/arch/arm/arm64/domain.c b/xen/arch/arm/arm64/domain.c
index dd1909892995..1e78986b5a7b 100644
--- a/xen/arch/arm/arm64/domain.c
+++ b/xen/arch/arm/arm64/domain.c
@@ -55,6 +55,11 @@ void vcpu_regs_user_to_hyp(struct vcpu *vcpu,
 #undef C
 }
 
+void vcpu_switch_to_aarch64_mode(struct vcpu *v)
+{
+    v->arch.hcr_el2 |= HCR_RW;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index bbd4a764c696..e785278cdbd7 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -605,11 +605,6 @@ void arch_vcpu_destroy(struct vcpu *v)
     free_xenheap_pages(v->arch.stack, STACK_ORDER);
 }
 
-void vcpu_switch_to_aarch64_mode(struct vcpu *v)
-{
-    v->arch.hcr_el2 |= HCR_RW;
-}
-
 int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
 {
     unsigned int max_vcpus;
diff --git a/xen/arch/arm/include/asm/arm32/domain.h b/xen/arch/arm/include/asm/arm32/domain.h
new file mode 100644
index 000000000000..4d1251e9c128
--- /dev/null
+++ b/xen/arch/arm/include/asm/arm32/domain.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#ifndef ARM_ARM32_DOMAIN_H
+#define ARM_ARM32_DOMAIN_H
+
+#define vcpu_switch_to_aarch64_mode(v)
+
+#endif /* ARM_ARM32_DOMAIN_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/include/asm/arm64/domain.h b/xen/arch/arm/include/asm/arm64/domain.h
new file mode 100644
index 000000000000..b5f1177d2508
--- /dev/null
+++ b/xen/arch/arm/include/asm/arm64/domain.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#ifndef ARM_ARM64_DOMAIN_H
+#define ARM_ARM64_DOMAIN_H
+
+/*
+ * Set guest execution state to AArch64 (EL1) for selected vcpu
+ *
+ * @vcpu: pointer to the vcpu structure
+ */
+void vcpu_switch_to_aarch64_mode(struct vcpu *v);
+
+#endif /* ARM_ARM64_DOMAIN_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/include/asm/domain.h b/xen/arch/arm/include/asm/domain.h
index a3487ca71303..fa258eb8d359 100644
--- a/xen/arch/arm/include/asm/domain.h
+++ b/xen/arch/arm/include/asm/domain.h
@@ -24,9 +24,11 @@ enum domain_type {
 };
 #define is_32bit_domain(d) ((d)->arch.type == DOMAIN_32BIT)
 #define is_64bit_domain(d) ((d)->arch.type == DOMAIN_64BIT)
+#include <asm/arm64/domain.h>
 #else
 #define is_32bit_domain(d) (1)
 #define is_64bit_domain(d) (0)
+#include <asm/arm64/domain.h>
 #endif
 
 /*
@@ -246,7 +248,6 @@ struct arch_vcpu
 }  __cacheline_aligned;
 
 void vcpu_show_registers(struct vcpu *v);
-void vcpu_switch_to_aarch64_mode(struct vcpu *v);
 
 /*
  * Due to the restriction of GICv3, the number of vCPUs in AFF0 is
-- 
2.34.1

[XEN][PATCH 4/8] xen/arm: split is_32bit/64bit_domain() between arm64/arm32

[Grygorii Strashko]

From: Grygorii Strashko <grygorii_strashko@epam.com>

Split is_32bit/64bit_domain() macro implementations between arm64/arm32.

Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
---
 xen/arch/arm/include/asm/arm32/domain.h |  5 +++++
 xen/arch/arm/include/asm/arm64/domain.h | 14 ++++++++++++++
 xen/arch/arm/include/asm/domain.h       |  8 +++-----
 3 files changed, 22 insertions(+), 5 deletions(-)

diff --git a/xen/arch/arm/include/asm/arm32/domain.h b/xen/arch/arm/include/asm/arm32/domain.h
index 4d1251e9c128..c0a7fc35f38b 100644
--- a/xen/arch/arm/include/asm/arm32/domain.h
+++ b/xen/arch/arm/include/asm/arm32/domain.h
@@ -3,6 +3,11 @@
 #ifndef ARM_ARM32_DOMAIN_H
 #define ARM_ARM32_DOMAIN_H
 
+/* Arm32 always runs guests in AArch32 mode */
+
+#define is_32bit_domain(d) (1)
+#define is_64bit_domain(d) (0)
+
 #define vcpu_switch_to_aarch64_mode(v)
 
 #endif /* ARM_ARM32_DOMAIN_H */
diff --git a/xen/arch/arm/include/asm/arm64/domain.h b/xen/arch/arm/include/asm/arm64/domain.h
index b5f1177d2508..7a51ceedf25d 100644
--- a/xen/arch/arm/include/asm/arm64/domain.h
+++ b/xen/arch/arm/include/asm/arm64/domain.h
@@ -3,6 +3,20 @@
 #ifndef ARM_ARM64_DOMAIN_H
 #define ARM_ARM64_DOMAIN_H
 
+/*
+ * Returns true if guest execution state is AArch32
+ *
+ * @d: pointer to the domain structure
+ */
+#define is_32bit_domain(d) ((d)->arch.type == DOMAIN_32BIT)
+
+/*
+ * Returns true if guest execution state is AArch64
+ *
+ * @d: pointer to the domain structure
+ */
+#define is_64bit_domain(d) ((d)->arch.type == DOMAIN_64BIT)
+
 /*
  * Set guest execution state to AArch64 (EL1) for selected vcpu
  *
diff --git a/xen/arch/arm/include/asm/domain.h b/xen/arch/arm/include/asm/domain.h
index fa258eb8d359..249e2d3be5d3 100644
--- a/xen/arch/arm/include/asm/domain.h
+++ b/xen/arch/arm/include/asm/domain.h
@@ -22,13 +22,11 @@ enum domain_type {
     DOMAIN_32BIT,
     DOMAIN_64BIT,
 };
-#define is_32bit_domain(d) ((d)->arch.type == DOMAIN_32BIT)
-#define is_64bit_domain(d) ((d)->arch.type == DOMAIN_64BIT)
+
 #include <asm/arm64/domain.h>
 #else
-#define is_32bit_domain(d) (1)
-#define is_64bit_domain(d) (0)
-#include <asm/arm64/domain.h>
+
+#include <asm/arm32/domain.h>
 #endif
 
 /*
-- 
2.34.1

[XEN][PATCH 6/8] xen/arm64: constify is_32/64bit_domain() macro for CONFIG_ARM64_AARCH32=n

[Grygorii Strashko]

From: Grygorii Strashko <grygorii_strashko@epam.com>

Constify is_32/64bit_domain() macro for the case CONFIG_ARM64_AARCH32=n and
so allow compiler to opt out Aarch32 specific code.

(CONFIG_ARM64_AARCH32=y)
Before:
   text	   data	    bss	    dec	    hex	filename
 855232	 322404	 270880	1448516	 161a44	xen-syms

(CONFIG_ARM64_AARCH32=n, CONFIG_EXPERT=y)
After:
   text	   data	    bss	    dec	    hex	filename
 851568	 322404	 270880	1444852	 160bf4	xen-syms
diff: −3664 (dec)

Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
---
 xen/arch/arm/include/asm/arm64/domain.h | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/xen/arch/arm/include/asm/arm64/domain.h b/xen/arch/arm/include/asm/arm64/domain.h
index 18402ae3ca0d..a014ab9967ac 100644
--- a/xen/arch/arm/include/asm/arm64/domain.h
+++ b/xen/arch/arm/include/asm/arm64/domain.h
@@ -12,14 +12,22 @@ struct kernel_info;
  *
  * @d: pointer to the domain structure
  */
+#if defined(CONFIG_ARM64_AARCH32)
 #define is_32bit_domain(d) ((d)->arch.type == DOMAIN_32BIT)
+#else
+#define is_32bit_domain(d) (false)
+#endif /* CONFIG_ARM64_AARCH32 */
 
 /*
  * Returns true if guest execution state is AArch64
  *
  * @d: pointer to the domain structure
  */
+#if defined(CONFIG_ARM64_AARCH32)
 #define is_64bit_domain(d) ((d)->arch.type == DOMAIN_64BIT)
+#else
+#define is_64bit_domain(d) (true)
+#endif /* CONFIG_ARM64_AARCH32 */
 
 /*
  * Set guest execution state to AArch64 (EL1) for selected vcpu
-- 
2.34.1

[XEN][PATCH 5/8] xen/arm64: make aarch32 support optional

[Grygorii Strashko]

From: Grygorii Strashko <grygorii_strashko@epam.com>

Now Arm64 AArch32 guest support is always enabled and built-in while not
all Arm64 platforms supports AArch32 or this support might not be needed.

Hence, this patch introduces basic support for disabling Arm64 AArch32
guest support. The following changes are introduced:

- Introduce Kconfig option CONFIG_ARM64_AARCH32 to allow enable/disable
  Arm64 AArch32 guest support (default y)

- Introduce is_aarch32_enabled() helper which accounts Arm64 HW capability
  and CONFIG_ARM64_AARCH32 setting

- Set Arm64 domain type to DOMAIN_64BIT by default

- Introduce arm64_set_domain_type() to configure Arm64 domain type in
  unified way instead of open coding (d)->arch.type, and account
  CONFIG_ARM64_AARCH32 configuration.

- toolstack: do not advertise "xen-3.0-armv7l " capability if
  CONFIG_ARM64_AARCH32=n

- toolstack: reduce XEN_DOMCTL_set_address_size hypercall handler to a
  simple sanity check if CONFIG_ARM64_AARCH32=n

With CONFIG_ARM64_AARCH32=n the Xen will reject AArch32 guests (kernels) if
configured by user in the following way:
- Xen boot will fail with panic during dom0 or dom0less domains creation
- toolstack domain creation will be rejected due to xc_dom_compat_check()
  failure.

Making Arm64 AArch32 guest support open further possibilities for build
optimizations of Arm64 AArch32 guest support code.

Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
---
 xen/arch/arm/Kconfig                    |  7 ++++
 xen/arch/arm/arm64/domain-build.c       | 45 +++++++++++++++++++++++--
 xen/arch/arm/arm64/domctl.c             |  6 ++++
 xen/arch/arm/domain.c                   |  1 +
 xen/arch/arm/domain_build.c             | 24 +++----------
 xen/arch/arm/include/asm/arm32/domain.h |  6 ++++
 xen/arch/arm/include/asm/arm64/domain.h | 27 +++++++++++++++
 xen/arch/arm/setup.c                    |  2 +-
 8 files changed, 95 insertions(+), 23 deletions(-)

diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 17df147b2555..d41076143d81 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -267,6 +267,13 @@ config PCI_PASSTHROUGH
 	help
 	  This option enables PCI device passthrough
 
+config ARM64_AARCH32
+	bool "AArch32 Guests support on on ARM64 (UNSUPPORTED)" if UNSUPPORTED
+	depends on ARM_64
+	default y
+	help
+	  This option enables AArch32 Guests on ARM64.
+
 endmenu
 
 menu "ARM errata workaround via the alternative framework"
diff --git a/xen/arch/arm/arm64/domain-build.c b/xen/arch/arm/arm64/domain-build.c
index 3a89ee46b8c6..aa8c616dd92e 100644
--- a/xen/arch/arm/arm64/domain-build.c
+++ b/xen/arch/arm/arm64/domain-build.c
@@ -4,13 +4,54 @@
 #include <xen/sched.h>
 
 #include <asm/domain.h>
+#include <asm/arm64/sve.h>
+
+int __init arm64_set_domain_type(struct domain *d, struct kernel_info *kinfo)
+{
+    enum domain_type type;
+
+    ASSERT(d);
+    ASSERT(kinfo);
+
+    type = kinfo->arch.type;
+
+    if ( !is_aarch32_enabled() )
+    {
+        ASSERT(d->arch.type == DOMAIN_64BIT);
+
+        if ( type == DOMAIN_32BIT )
+        {
+            const char *str = "not available";
+
+            if ( !IS_ENABLED(CONFIG_ARM64_AARCH32) )
+                str = "disabled";
+            printk("aarch32 guests support is %s\n", str);
+            return -EINVAL;
+        }
+
+        return 0;
+    }
+
+    if ( is_sve_domain(d) && type == DOMAIN_32BIT )
+    {
+        printk("SVE is not available for 32-bit domain\n");
+        return -EINVAL;
+    }
+
+    d->arch.type = type;
+
+    return 0;
+}
 
 #ifdef CONFIG_DOM0LESS_BOOT
 /* TODO: make arch.type generic ? */
 void __init set_domain_type(struct domain *d, struct kernel_info *kinfo)
 {
-    /* type must be set before allocate memory */
-    d->arch.type = kinfo->arch.type;
+    int rc;
+
+    rc = arm64_set_domain_type(d, kinfo);
+    if ( rc < 0 )
+        panic("Unsupported guest type\n");
 }
 #endif
 
diff --git a/xen/arch/arm/arm64/domctl.c b/xen/arch/arm/arm64/domctl.c
index 8720d126c97d..8a243d3fbd15 100644
--- a/xen/arch/arm/arm64/domctl.c
+++ b/xen/arch/arm/arm64/domctl.c
@@ -13,6 +13,7 @@
 #include <asm/arm64/sve.h>
 #include <asm/cpufeature.h>
 
+#if defined(CONFIG_ARM64_AARCH32)
 static long switch_mode(struct domain *d, enum domain_type type)
 {
     struct vcpu *v;
@@ -50,6 +51,7 @@ static long set_address_size(struct domain *d, uint32_t address_size)
         return -EINVAL;
     }
 }
+#endif /* CONFIG_ARM64_AARCH32 */
 
 long subarch_do_domctl(struct xen_domctl *domctl, struct domain *d,
                        XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
@@ -57,7 +59,11 @@ long subarch_do_domctl(struct xen_domctl *domctl, struct domain *d,
     switch ( domctl->cmd )
     {
     case XEN_DOMCTL_set_address_size:
+#if defined(CONFIG_ARM64_AARCH32)
         return set_address_size(d, domctl->u.address_size.size);
+#else
+        return domctl->u.address_size.size == 32 ? -EINVAL : 0;
+#endif /* CONFIG_ARM64_AARCH32 */
 
     default:
         return -ENOSYS;
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index e785278cdbd7..d08ca458a15c 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -788,6 +788,7 @@ int arch_domain_create(struct domain *d,
     /* Copy the encoded vector length sve_vl from the domain configuration */
     d->arch.sve_vl = config->arch.sve_vl;
 #endif
+    domain_set_type_default(d);
 
     return 0;
 
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index af7e9d830ae1..06dfd5bedd14 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1872,21 +1872,6 @@ int __init construct_domain(struct domain *d, struct kernel_info *kinfo)
     BUG_ON(d->vcpu[0] == NULL);
     BUG_ON(v->is_initialised);
 
-#ifdef CONFIG_ARM_64
-    /* if aarch32 mode is not supported at EL1 do not allow 32-bit domain */
-    if ( !(cpu_has_el1_32) && kinfo->arch.type == DOMAIN_32BIT )
-    {
-        printk("Platform does not support 32-bit domain\n");
-        return -EINVAL;
-    }
-
-    if ( is_sve_domain(d) && (kinfo->arch.type == DOMAIN_32BIT) )
-    {
-        printk("SVE is not available for 32-bit domain\n");
-        return -EINVAL;
-    }
-#endif
-
     /*
      * kernel_load will determine the placement of the kernel as well
      * as the initrd & fdt in RAM, so call it first.
@@ -1976,6 +1961,10 @@ static int __init construct_dom0(struct domain *d)
     if ( rc < 0 )
         return rc;
 
+    rc = arm64_set_domain_type(d, &kinfo);
+    if ( rc < 0 )
+        return rc;
+
     return construct_hwdom(&kinfo, NULL);
 }
 
@@ -1987,11 +1976,6 @@ int __init construct_hwdom(struct kernel_info *kinfo,
 
     iommu_hwdom_init(d);
 
-#ifdef CONFIG_ARM_64
-    /* type must be set before allocate_memory or create cpu */
-    d->arch.type = kinfo->arch.type;
-#endif
-
     if ( vcpu_create(d, 0) == NULL )
         panic("Error creating domain 0 vcpu0\n");
 
diff --git a/xen/arch/arm/include/asm/arm32/domain.h b/xen/arch/arm/include/asm/arm32/domain.h
index c0a7fc35f38b..bdbb9e3cad47 100644
--- a/xen/arch/arm/include/asm/arm32/domain.h
+++ b/xen/arch/arm/include/asm/arm32/domain.h
@@ -10,6 +10,12 @@
 
 #define vcpu_switch_to_aarch64_mode(v)
 
+#define is_aarch32_enabled() (true)
+
+#define domain_set_type_default(d)
+
+#define arm64_set_domain_type(d, kinfo) (0)
+
 #endif /* ARM_ARM32_DOMAIN_H */
 
 /*
diff --git a/xen/arch/arm/include/asm/arm64/domain.h b/xen/arch/arm/include/asm/arm64/domain.h
index 7a51ceedf25d..18402ae3ca0d 100644
--- a/xen/arch/arm/include/asm/arm64/domain.h
+++ b/xen/arch/arm/include/asm/arm64/domain.h
@@ -3,6 +3,10 @@
 #ifndef ARM_ARM64_DOMAIN_H
 #define ARM_ARM64_DOMAIN_H
 
+#include <asm/cpufeature.h>
+
+struct kernel_info;
+
 /*
  * Returns true if guest execution state is AArch32
  *
@@ -24,6 +28,29 @@
  */
 void vcpu_switch_to_aarch64_mode(struct vcpu *v);
 
+/*
+ * Arm64 declares AArch32 (32bit) Execution State support in the
+ * Processor Feature Registers (PFR0), but also can be disabled manually.
+ */
+#define is_aarch32_enabled()                                                   \
+        (IS_ENABLED(CONFIG_ARM64_AARCH32) && cpu_has_aarch32)
+
+/*
+ * Set default Execution State to AArch64 (64bit) during domain creation.
+ */
+#define domain_set_type_default(d) ((d)->arch.type = DOMAIN_64BIT)
+
+/*
+ * Set domain type from struct kernel_info which defines guest Execution
+ * State AArch32/AArch64 during regular dom0 or predefined (dom0less)
+ * domains creation .
+ * Type must be set before allocate_memory or create vcpus.
+ *
+ * @d: pointer to the domain structure.
+ * @kinfo: pointer to the kinfo structure.
+ */
+int arm64_set_domain_type(struct domain *d, struct kernel_info *kinfo);
+
 #endif /* ARM_ARM64_DOMAIN_H */
 
 /*
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 8abc1d641df0..368350226973 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -530,7 +530,7 @@ static int __init init_xen_cap_info(void)
 #ifdef CONFIG_ARM_64
     safe_strcat(xen_cap_info, "xen-3.0-aarch64 ");
 #endif
-    if ( cpu_has_aarch32 )
+    if ( is_aarch32_enabled() )
         safe_strcat(xen_cap_info, "xen-3.0-armv7l ");
 
     return 0;
-- 
2.34.1

[XEN][PATCH 7/8] xen/arm: regs.h split subarch definitions between arm64/arm32

[Grygorii Strashko]

From: Grygorii Strashko <grygorii_strashko@epam.com>

Split subarch definitions between arm64/arm32:
hyp_mode()
regs_mode_is_user()
regs_mode_is_32bit()

Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
---
 xen/arch/arm/include/asm/arm32/processor.h |  5 +++++
 xen/arch/arm/include/asm/arm64/processor.h | 15 ++++++++++++++
 xen/arch/arm/include/asm/regs.h            | 24 ----------------------
 3 files changed, 20 insertions(+), 24 deletions(-)

diff --git a/xen/arch/arm/include/asm/arm32/processor.h b/xen/arch/arm/include/asm/arm32/processor.h
index 4e679f3273ab..10d5ff5c192e 100644
--- a/xen/arch/arm/include/asm/arm32/processor.h
+++ b/xen/arch/arm/include/asm/arm32/processor.h
@@ -56,6 +56,11 @@ struct cpu_user_regs
     uint32_t pad1; /* Doubleword-align the user half of the frame */
 };
 
+#define hyp_mode(r)          psr_mode((r)->cpsr, PSR_MODE_HYP)
+#define regs_mode_is_user(r) usr_mode(r)
+
+#define regs_mode_is_32bit(regs) (true)
+
 #endif
 
 #endif /* __ASM_ARM_ARM32_PROCESSOR_H */
diff --git a/xen/arch/arm/include/asm/arm64/processor.h b/xen/arch/arm/include/asm/arm64/processor.h
index c749f80ad91b..daf890708d87 100644
--- a/xen/arch/arm/include/asm/arm64/processor.h
+++ b/xen/arch/arm/include/asm/arm64/processor.h
@@ -86,6 +86,21 @@ struct cpu_user_regs
 
 #undef __DECL_REG
 
+#define hyp_mode(r)                                                            \
+        (psr_mode((r)->cpsr, PSR_MODE_EL2h) ||                                 \
+         psr_mode((r)->cpsr, PSR_MODE_EL2t))
+
+/*
+ * Trap may have been taken from EL0, which might be in AArch32 usr
+ * mode, or in AArch64 mode (PSR_MODE_EL0t).
+ */
+#define regs_mode_is_user(r) (psr_mode((r)->cpsr, PSR_MODE_EL0t) || usr_mode(r))
+
+static inline bool regs_mode_is_32bit(const struct cpu_user_regs *regs)
+{
+    return !!(regs->cpsr & PSR_MODE_BIT);
+}
+
 #endif /* __ASSEMBLY__ */
 
 #endif /* __ASM_ARM_ARM64_PROCESSOR_H */
diff --git a/xen/arch/arm/include/asm/regs.h b/xen/arch/arm/include/asm/regs.h
index 0d9f239a7782..22d3a1688876 100644
--- a/xen/arch/arm/include/asm/regs.h
+++ b/xen/arch/arm/include/asm/regs.h
@@ -13,15 +13,6 @@
 
 #define psr_mode(psr,m) (((psr) & PSR_MODE_MASK) == (m))
 
-static inline bool regs_mode_is_32bit(const struct cpu_user_regs *regs)
-{
-#ifdef CONFIG_ARM_32
-    return true;
-#else
-    return !!(regs->cpsr & PSR_MODE_BIT);
-#endif
-}
-
 #define usr_mode(r)     psr_mode((r)->cpsr,PSR_MODE_USR)
 #define fiq_mode(r)     psr_mode((r)->cpsr,PSR_MODE_FIQ)
 #define irq_mode(r)     psr_mode((r)->cpsr,PSR_MODE_IRQ)
@@ -31,21 +22,6 @@ static inline bool regs_mode_is_32bit(const struct cpu_user_regs *regs)
 #define und_mode(r)     psr_mode((r)->cpsr,PSR_MODE_UND)
 #define sys_mode(r)     psr_mode((r)->cpsr,PSR_MODE_SYS)
 
-#ifdef CONFIG_ARM_32
-#define hyp_mode(r)     psr_mode((r)->cpsr,PSR_MODE_HYP)
-#define regs_mode_is_user(r) usr_mode(r)
-#else
-#define hyp_mode(r)     (psr_mode((r)->cpsr,PSR_MODE_EL2h) || \
-                         psr_mode((r)->cpsr,PSR_MODE_EL2t))
-
-/*
- * Trap may have been taken from EL0, which might be in AArch32 usr
- * mode, or in AArch64 mode (PSR_MODE_EL0t).
- */
-#define regs_mode_is_user(r) \
-    (psr_mode((r)->cpsr,PSR_MODE_EL0t) || usr_mode(r))
-#endif
-
 static inline bool guest_mode(const struct cpu_user_regs *r)
 {
     unsigned long diff = (uintptr_t)guest_cpu_user_regs() - (uintptr_t)(r);
-- 
2.34.1

[XEN][PATCH 8/8] xen/arm64: constify regs_mode_is_32bit macro for CONFIG_ARM64_AARCH32=n

[Grygorii Strashko]

From: Grygorii Strashko <grygorii_strashko@epam.com>

Constify regs_mode_is_32bit() macro for the case CONFIG_ARM64_AARCH32=n  and
so allow compiler to opt out Aarch32 specific code.

(CONFIG_ARM64_AARCH32=y)
Before:
 855232	 322404	 270880	1448516	 161a44	xen-syms

(CONFIG_ARM64_AARCH32=n, CONFIG_EXPERT=y)
After:
 849548	 322404	 270880	1442832	 160410	xen-syms
diff: -5684 (dec)

Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
---
 xen/arch/arm/include/asm/arm64/processor.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/xen/arch/arm/include/asm/arm64/processor.h b/xen/arch/arm/include/asm/arm64/processor.h
index daf890708d87..a3d83869f02d 100644
--- a/xen/arch/arm/include/asm/arm64/processor.h
+++ b/xen/arch/arm/include/asm/arm64/processor.h
@@ -96,10 +96,14 @@ struct cpu_user_regs
  */
 #define regs_mode_is_user(r) (psr_mode((r)->cpsr, PSR_MODE_EL0t) || usr_mode(r))
 
+#if defined(CONFIG_ARM64_AARCH32)
 static inline bool regs_mode_is_32bit(const struct cpu_user_regs *regs)
 {
     return !!(regs->cpsr & PSR_MODE_BIT);
 }
+#else
+#define regs_mode_is_32bit(regs) (false)
+#endif /* CONFIG_ARM64_AARCH32 */
 
 #endif /* __ASSEMBLY__ */
 
-- 
2.34.1

Re: [XEN][PATCH 0/8] xen/arm64: make aarch32 support optional

[Julien Grall]

On 23/07/2025 08:58, Grygorii Strashko wrote:
> From: Grygorii Strashko <grygorii_strashko@epam.com>
> 
> Hi,

Hi Grygorii,

> Now Arm64 AArch32 guest support is always enabled and built-in while not
> all Arm64 platforms supports AArch32 or this support might not be needed.

I am not entirely sure I like the proliferation of using CONFIG_* for 
every single feature. This makes the testing a bit more complicated.

Can you clarify what the goal with this patch?

Cheers,

-- 
Julien Grall

Re: [XEN][PATCH 4/8] xen/arm: split is_32bit/64bit_domain() between arm64/arm32

[Andrew Cooper]

On 23/07/2025 8:58 am, Grygorii Strashko wrote:
> diff --git a/xen/arch/arm/include/asm/arm32/domain.h b/xen/arch/arm/include/asm/arm32/domain.h
> index 4d1251e9c128..c0a7fc35f38b 100644
> --- a/xen/arch/arm/include/asm/arm32/domain.h
> +++ b/xen/arch/arm/include/asm/arm32/domain.h
> @@ -3,6 +3,11 @@
>  #ifndef ARM_ARM32_DOMAIN_H
>  #define ARM_ARM32_DOMAIN_H
>  
> +/* Arm32 always runs guests in AArch32 mode */
> +
> +#define is_32bit_domain(d) (1)
> +#define is_64bit_domain(d) (0)

I know you're just moving code, but this was buggy before.

These need to be ((void)(d), 1/0) so d gets evaluated consistently.

~Andrew

Re: [XEN][PATCH 6/8] xen/arm64: constify is_32/64bit_domain() macro for CONFIG_ARM64_AARCH32=n

[Andrew Cooper]

On 23/07/2025 8:58 am, Grygorii Strashko wrote:
> diff --git a/xen/arch/arm/include/asm/arm64/domain.h b/xen/arch/arm/include/asm/arm64/domain.h
> index 18402ae3ca0d..a014ab9967ac 100644
> --- a/xen/arch/arm/include/asm/arm64/domain.h
> +++ b/xen/arch/arm/include/asm/arm64/domain.h
> @@ -12,14 +12,22 @@ struct kernel_info;
>   *
>   * @d: pointer to the domain structure
>   */
> +#if defined(CONFIG_ARM64_AARCH32)
>  #define is_32bit_domain(d) ((d)->arch.type == DOMAIN_32BIT)
> +#else
> +#define is_32bit_domain(d) (false)
> +#endif /* CONFIG_ARM64_AARCH32 */

There's no need to make two separate definitions.  Use IS_ENABLED(). 
(This also fixes the evaluation of d problem you've introduced.)

IS_ENABLED(CONFIG_ARM64_AARCH32) && (d)->arch.type == DOMAIN_32BIT

>  
>  /*
>   * Returns true if guest execution state is AArch64
>   *
>   * @d: pointer to the domain structure
>   */
> +#if defined(CONFIG_ARM64_AARCH32)
>  #define is_64bit_domain(d) ((d)->arch.type == DOMAIN_64BIT)
> +#else
> +#define is_64bit_domain(d) (true)
> +#endif /* CONFIG_ARM64_AARCH32 */

!IS_ENABLED(CONFIG_ARM64_AARCH32) || (d)->arch.type == DOMAIN_64BIT

~Andrew

Re: [XEN][PATCH 2/8] xen/arm: move vcpu_switch_to_aarch64_mode() in arch_vcpu_create()

[Julien Grall]

Hi,

On 23/07/2025 08:58, Grygorii Strashko wrote:
> From: Grygorii Strashko <grygorii_strashko@epam.com>
> 
> Move vcpu_switch_to_aarch64_mode() in arch_vcpu_create() callback instead
> of calling it manually from few different places after vcpu_create().
> 
> Before doing above ensure vcpu0 is created after kernel_probe() is done and
> domain's guest execution mode (32-bit/64-bit) is set for dom0 and dom0less
> domains.

The commit message doesn't mention anything about domains created by the 
toolstack. In this case, from my understanding, the switch to 64-bit 
domain happens *after* the vCPUs are created.

At the moment, I think this is probably ok to call...

> 
> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
> ---
>   xen/arch/arm/domain.c                    |  3 +++
>   xen/arch/arm/domain_build.c              | 13 +++++--------
>   xen/common/device-tree/dom0less-build.c  |  6 +++---
>   xen/include/asm-generic/dom0less-build.h |  2 +-
>   4 files changed, 12 insertions(+), 12 deletions(-)
> 
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 79a144e61be9..bbd4a764c696 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -586,6 +586,9 @@ int arch_vcpu_create(struct vcpu *v)
>       if ( get_ssbd_state() == ARM_SSBD_RUNTIME )
>           v->arch.cpu_info->flags |= CPUINFO_WORKAROUND_2_FLAG;
>   
> +    if ( is_64bit_domain(v->domain) )
> +        vcpu_switch_to_aarch64_mode(v);

... this function here because I *think* it would be NOP. But this feels 
really fragile.

If the desire is to make 32-bit domain optional on Arm64. Then I think 
it would be better to pass the domain type when the domain is created 
(IOW add an extra flags to XEN_DOMCTL_createdomain). This will require 
more work, but it will be a lot more robust.

Cheers,

-- 
Julien Grall

Re: [XEN][PATCH 3/8] xen/arm: move vcpu_switch_to_aarch64_mode() in arm64

[Julien Grall]

Hi,

On 23/07/2025 08:58, Grygorii Strashko wrote:
> From: Grygorii Strashko <grygorii_strashko@epam.com>
> 
> The vcpu_switch_to_aarch64_mode() is Arm64 specific, so move it in Arm64.
> As part of this change:
> - introduce arm32/arm64 domain.h headers and include them in asm/domain.h
> basing on CONFIG_ARM_xx;
> - declare vcpu_switch_to_aarch64_mode() for arm64;
> - add vcpu_switch_to_aarch64_mode() as empty macro for arm32.
> 
> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
> ---
>   xen/arch/arm/arm64/domain.c             |  5 +++++
>   xen/arch/arm/domain.c                   |  5 -----
>   xen/arch/arm/include/asm/arm32/domain.h | 17 +++++++++++++++++
>   xen/arch/arm/include/asm/arm64/domain.h | 22 ++++++++++++++++++++++
>   xen/arch/arm/include/asm/domain.h       |  3 ++-
>   5 files changed, 46 insertions(+), 6 deletions(-)
>   create mode 100644 xen/arch/arm/include/asm/arm32/domain.h
>   create mode 100644 xen/arch/arm/include/asm/arm64/domain.h
> 
> diff --git a/xen/arch/arm/arm64/domain.c b/xen/arch/arm/arm64/domain.c
> index dd1909892995..1e78986b5a7b 100644
> --- a/xen/arch/arm/arm64/domain.c
> +++ b/xen/arch/arm/arm64/domain.c
> @@ -55,6 +55,11 @@ void vcpu_regs_user_to_hyp(struct vcpu *vcpu,
>   #undef C
>   }
>   
> +void vcpu_switch_to_aarch64_mode(struct vcpu *v)
> +{
> +    v->arch.hcr_el2 |= HCR_RW;
> +}

Strictly speaking arm/domain.c is GPLv2-or-later. But arm64/domain.c 
doesn't have a license. So it would default to GPLv2-only. There have 
been argument in the past on whether we would re-license code from 
GPLv2-or-later to GPLv2-only. But this was never concluded. So I am not 
entirely sure what to do with this change...

Bertrand, Michal,  Stefano?

> +
>   /*
>    * Local variables:
>    * mode: C
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index bbd4a764c696..e785278cdbd7 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -605,11 +605,6 @@ void arch_vcpu_destroy(struct vcpu *v)
>       free_xenheap_pages(v->arch.stack, STACK_ORDER);
>   }
>   
> -void vcpu_switch_to_aarch64_mode(struct vcpu *v)
> -{
> -    v->arch.hcr_el2 |= HCR_RW;
> -}
> -
>   int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>   {
>       unsigned int max_vcpus;
> diff --git a/xen/arch/arm/include/asm/arm32/domain.h b/xen/arch/arm/include/asm/arm32/domain.h
> new file mode 100644
> index 000000000000..4d1251e9c128
> --- /dev/null
> +++ b/xen/arch/arm/include/asm/arm32/domain.h
> @@ -0,0 +1,17 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +
> +#ifndef ARM_ARM32_DOMAIN_H
> +#define ARM_ARM32_DOMAIN_H
> +
> +#define vcpu_switch_to_aarch64_mode(v)

I think you want to "consume" v. IOW (void)(v).  That said, we tend to 
prefer using a static inline whenever it is possible. Have you tried it?

> +
> +#endif /* ARM_ARM32_DOMAIN_H */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/include/asm/arm64/domain.h b/xen/arch/arm/include/asm/arm64/domain.h
> new file mode 100644
> index 000000000000..b5f1177d2508
> --- /dev/null
> +++ b/xen/arch/arm/include/asm/arm64/domain.h
> @@ -0,0 +1,22 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +
> +#ifndef ARM_ARM64_DOMAIN_H
> +#define ARM_ARM64_DOMAIN_H
> +
> +/*
> + * Set guest execution state to AArch64 (EL1) for selected vcpu
> + *
> + * @vcpu: pointer to the vcpu structure
> + */
> +void vcpu_switch_to_aarch64_mode(struct vcpu *v);
> +
> +#endif /* ARM_ARM64_DOMAIN_H */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/include/asm/domain.h b/xen/arch/arm/include/asm/domain.h
> index a3487ca71303..fa258eb8d359 100644
> --- a/xen/arch/arm/include/asm/domain.h
> +++ b/xen/arch/arm/include/asm/domain.h
> @@ -24,9 +24,11 @@ enum domain_type {
>   };
>   #define is_32bit_domain(d) ((d)->arch.type == DOMAIN_32BIT)
>   #define is_64bit_domain(d) ((d)->arch.type == DOMAIN_64BIT)
> +#include <asm/arm64/domain.h>
>   #else
>   #define is_32bit_domain(d) (1)
>   #define is_64bit_domain(d) (0)
> +#include <asm/arm64/domain.h>
>   #endif
>   
>   /*
> @@ -246,7 +248,6 @@ struct arch_vcpu
>   }  __cacheline_aligned;
>   
>   void vcpu_show_registers(struct vcpu *v);
> -void vcpu_switch_to_aarch64_mode(struct vcpu *v);
>   
>   /*
>    * Due to the restriction of GICv3, the number of vCPUs in AFF0 is

Cheers,

-- 
Julien Grall

Re: [XEN][PATCH 2/8] xen/arm: move vcpu_switch_to_aarch64_mode() in arch_vcpu_create()

[Grygorii Strashko]

On 23.07.25 12:16, Julien Grall wrote:
> Hi,
> 
> On 23/07/2025 08:58, Grygorii Strashko wrote:
>> From: Grygorii Strashko <grygorii_strashko@epam.com>
>>
>> Move vcpu_switch_to_aarch64_mode() in arch_vcpu_create() callback instead
>> of calling it manually from few different places after vcpu_create().
>>
>> Before doing above ensure vcpu0 is created after kernel_probe() is done and
>> domain's guest execution mode (32-bit/64-bit) is set for dom0 and dom0less
>> domains.
> 
> The commit message doesn't mention anything about domains created by the toolstack. In this case, from my understanding, the switch to 64-bit domain happens *after* the vCPUs are created.
> 
> At the moment, I think this is probably ok to call...
> 
>>
>> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
>> ---
>>   xen/arch/arm/domain.c                    |  3 +++
>>   xen/arch/arm/domain_build.c              | 13 +++++--------
>>   xen/common/device-tree/dom0less-build.c  |  6 +++---
>>   xen/include/asm-generic/dom0less-build.h |  2 +-
>>   4 files changed, 12 insertions(+), 12 deletions(-)
>>
>> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
>> index 79a144e61be9..bbd4a764c696 100644
>> --- a/xen/arch/arm/domain.c
>> +++ b/xen/arch/arm/domain.c
>> @@ -586,6 +586,9 @@ int arch_vcpu_create(struct vcpu *v)
>>       if ( get_ssbd_state() == ARM_SSBD_RUNTIME )
>>           v->arch.cpu_info->flags |= CPUINFO_WORKAROUND_2_FLAG;
>> +    if ( is_64bit_domain(v->domain) )
>> +        vcpu_switch_to_aarch64_mode(v);
> 
> ... this function here because I *think* it would be NOP. But this feels really fragile.

The toolstack configures domain and vcpus through XEN_DOMCTL_set_address_size on Arm64:
- toolstack creates domain and parses kernel binary: domain created with DOMAIN_32BIT mode by default
- toolstack creates vcpus (still 32 bit mode): libxl__build_pre()->xc_domain_max_vcpus()
- toolstack switches domain mode depending on kernel binary type: libxl__build_dom()->xc_dom_boot_mem_init(),
   which triggers XEN_DOMCTL_set_address_size hypercall.
   Xen: arm64: switches domain mode and re-configures vcpus: subarch_do_domctl()->set_address_size()

So, this patch does not affect toolstack path, only optimizes Xen boots a bit.

Also, during Xen boot or by toolstack - the domain is always created before it's type is even known, which, in turn,
is based on guest binary which is parsed later during domain configuration stage.

I can add note in commit message "This patch doesn't affect on the toolstack Arm64 domain creation path as toolstack always
re-configures domain mode and vcpus through XEN_DOMCTL_set_address_size hypercall during domain configuration stage"

> 
> If the desire is to make 32-bit domain optional on Arm64. Then I think it would be better to pass the domain type when the domain
> is created (IOW add an extra flags to XEN_DOMCTL_createdomain). This will require more work, but it will be a lot more robust.



-- 
Best regards,
-grygorii

Re: [XEN][PATCH 3/8] xen/arm: move vcpu_switch_to_aarch64_mode() in arm64

[Grygorii Strashko]

On 23.07.25 12:22, Julien Grall wrote:
> Hi,
> 
> On 23/07/2025 08:58, Grygorii Strashko wrote:
>> From: Grygorii Strashko <grygorii_strashko@epam.com>
>>
>> The vcpu_switch_to_aarch64_mode() is Arm64 specific, so move it in Arm64.
>> As part of this change:
>> - introduce arm32/arm64 domain.h headers and include them in asm/domain.h
>> basing on CONFIG_ARM_xx;
>> - declare vcpu_switch_to_aarch64_mode() for arm64;
>> - add vcpu_switch_to_aarch64_mode() as empty macro for arm32.
>>
>> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
>> ---
>>   xen/arch/arm/arm64/domain.c             |  5 +++++
>>   xen/arch/arm/domain.c                   |  5 -----
>>   xen/arch/arm/include/asm/arm32/domain.h | 17 +++++++++++++++++
>>   xen/arch/arm/include/asm/arm64/domain.h | 22 ++++++++++++++++++++++
>>   xen/arch/arm/include/asm/domain.h       |  3 ++-
>>   5 files changed, 46 insertions(+), 6 deletions(-)
>>   create mode 100644 xen/arch/arm/include/asm/arm32/domain.h
>>   create mode 100644 xen/arch/arm/include/asm/arm64/domain.h
>>
>> diff --git a/xen/arch/arm/arm64/domain.c b/xen/arch/arm/arm64/domain.c
>> index dd1909892995..1e78986b5a7b 100644
>> --- a/xen/arch/arm/arm64/domain.c
>> +++ b/xen/arch/arm/arm64/domain.c
>> @@ -55,6 +55,11 @@ void vcpu_regs_user_to_hyp(struct vcpu *vcpu,
>>   #undef C
>>   }
>> +void vcpu_switch_to_aarch64_mode(struct vcpu *v)
>> +{
>> +    v->arch.hcr_el2 |= HCR_RW;
>> +}
> 
> Strictly speaking arm/domain.c is GPLv2-or-later. But arm64/domain.c doesn't have a license.
  So it would default to GPLv2-only. There have been argument in the past on whether we would
  re-license code from GPLv2-or-later to GPLv2-only. But this was never concluded. So I am not entirely sure what to do with this change...
  
> Bertrand, Michal,  Stefano?
> 
>> +
>>   /*
>>    * Local variables:
>>    * mode: C
>> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
>> index bbd4a764c696..e785278cdbd7 100644
>> --- a/xen/arch/arm/domain.c
>> +++ b/xen/arch/arm/domain.c
>> @@ -605,11 +605,6 @@ void arch_vcpu_destroy(struct vcpu *v)
>>       free_xenheap_pages(v->arch.stack, STACK_ORDER);
>>   }
>> -void vcpu_switch_to_aarch64_mode(struct vcpu *v)
>> -{
>> -    v->arch.hcr_el2 |= HCR_RW;
>> -}
>> -
>>   int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>>   {
>>       unsigned int max_vcpus;
>> diff --git a/xen/arch/arm/include/asm/arm32/domain.h b/xen/arch/arm/include/asm/arm32/domain.h
>> new file mode 100644
>> index 000000000000..4d1251e9c128
>> --- /dev/null
>> +++ b/xen/arch/arm/include/asm/arm32/domain.h
>> @@ -0,0 +1,17 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +
>> +#ifndef ARM_ARM32_DOMAIN_H
>> +#define ARM_ARM32_DOMAIN_H
>> +
>> +#define vcpu_switch_to_aarch64_mode(v)
> 
> I think you want to "consume" v. IOW (void)(v).  That said, we tend to prefer using a static inline whenever it is possible. Have you tried it?

will do static inline.

Thank you.

-- 
Best regards,
-grygorii

Re: [XEN][PATCH 0/8] xen/arm64: make aarch32 support optional

[Orzel, Michal]

On 23/07/2025 10:06, Julien Grall wrote:
> 
> 
> On 23/07/2025 08:58, Grygorii Strashko wrote:
>> From: Grygorii Strashko <grygorii_strashko@epam.com>
>>
>> Hi,
> 
> Hi Grygorii,
> 
>> Now Arm64 AArch32 guest support is always enabled and built-in while not
>> all Arm64 platforms supports AArch32 or this support might not be needed.
> 
> I am not entirely sure I like the proliferation of using CONFIG_* for 
> every single feature. This makes the testing a bit more complicated.
> 
> Can you clarify what the goal with this patch?
AArch32 is used quite rarely in embedded systems. Also, in Armv9A it might only
be implemented at EL0 if at all. When focusing on safety certification, AArch32
related code in Xen leaves a gap in terms of coverage that cannot really be
justified in words. This leaves us with two options: either support it (lots of
additional testing, requirements and documents would be needed) or compile it out.

~Michal

Re: [XEN][PATCH 2/8] xen/arm: move vcpu_switch_to_aarch64_mode() in arch_vcpu_create()

[Julien Grall]

Hi,

On 23/07/2025 11:19, Grygorii Strashko wrote:
> On 23.07.25 12:16, Julien Grall wrote:
>> On 23/07/2025 08:58, Grygorii Strashko wrote:
>>> From: Grygorii Strashko <grygorii_strashko@epam.com>
>>>
>>> Move vcpu_switch_to_aarch64_mode() in arch_vcpu_create() callback 
>>> instead
>>> of calling it manually from few different places after vcpu_create().
>>>
>>> Before doing above ensure vcpu0 is created after kernel_probe() is 
>>> done and
>>> domain's guest execution mode (32-bit/64-bit) is set for dom0 and 
>>> dom0less
>>> domains.
>>
>> The commit message doesn't mention anything about domains created by 
>> the toolstack. In this case, from my understanding, the switch to 64- 
>> bit domain happens *after* the vCPUs are created.
>>
>> At the moment, I think this is probably ok to call...
>>
>>>
>>> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
>>> ---
>>>   xen/arch/arm/domain.c                    |  3 +++
>>>   xen/arch/arm/domain_build.c              | 13 +++++--------
>>>   xen/common/device-tree/dom0less-build.c  |  6 +++---
>>>   xen/include/asm-generic/dom0less-build.h |  2 +-
>>>   4 files changed, 12 insertions(+), 12 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
>>> index 79a144e61be9..bbd4a764c696 100644
>>> --- a/xen/arch/arm/domain.c
>>> +++ b/xen/arch/arm/domain.c
>>> @@ -586,6 +586,9 @@ int arch_vcpu_create(struct vcpu *v)
>>>       if ( get_ssbd_state() == ARM_SSBD_RUNTIME )
>>>           v->arch.cpu_info->flags |= CPUINFO_WORKAROUND_2_FLAG;
>>> +    if ( is_64bit_domain(v->domain) )
>>> +        vcpu_switch_to_aarch64_mode(v);
>>
>> ... this function here because I *think* it would be NOP. But this 
>> feels really fragile.
> 
> The toolstack configures domain and vcpus through 
> XEN_DOMCTL_set_address_size on Arm64:
> - toolstack creates domain and parses kernel binary: domain created with 
> DOMAIN_32BIT mode by default
> - toolstack creates vcpus (still 32 bit mode): libxl__build_pre()- 
>  >xc_domain_max_vcpus()
> - toolstack switches domain mode depending on kernel binary type: 
> libxl__build_dom()->xc_dom_boot_mem_init(),
>    which triggers XEN_DOMCTL_set_address_size hypercall.
>    Xen: arm64: switches domain mode and re-configures vcpus: 
> subarch_do_domctl()->set_address_size()
> 
> So, this patch does not affect toolstack path, only optimizes Xen boots 
> a bit. 

Thanks for providing more detaisl. I am not sure what you mean by 
optimize. It reduces the number of places where 
vcpu_switch_to_aarch64_mode() is called, but there should be no 
difference in term of boot time.

> 
> Also, during Xen boot or by toolstack - the domain is always created 
> before it's type is even known, which, in turn,
> is based on guest binary which is parsed later during domain 
> configuration stage.

What you are describing is the current situation. But this doesn't tell 
me *why* we can't provide the type when the domain is created.

> 
> I can add note in commit message "This patch doesn't affect on the 
> toolstack Arm64 domain creation path as toolstack always
> re-configures domain mode and vcpus through XEN_DOMCTL_set_address_size 
> hypercall during domain configuration stage"

Well, as I wrote before, I find this code extremely fragile. And you so 
far, you don't seem to have address this concern in your reply. In fact...

> 
>>
>> If the desire is to make 32-bit domain optional on Arm64. Then I think 
>> it would be better to pass the domain type when the domain
>> is created (IOW add an extra flags to XEN_DOMCTL_createdomain). This 
>> will require more work, but it will be a lot more robust.

... I proposed what I think is a better alternative. Did you consider it?

Cheers,

-- 
Julien Grall

Re: [XEN][PATCH 2/8] xen/arm: move vcpu_switch_to_aarch64_mode() in arch_vcpu_create()

[Andrew Cooper]

On 23/07/2025 11:19 am, Grygorii Strashko wrote:
>
>
> On 23.07.25 12:16, Julien Grall wrote:
>> Hi,
>>
>> On 23/07/2025 08:58, Grygorii Strashko wrote:
>>> From: Grygorii Strashko <grygorii_strashko@epam.com>
>>>
>>> Move vcpu_switch_to_aarch64_mode() in arch_vcpu_create() callback
>>> instead
>>> of calling it manually from few different places after vcpu_create().
>>>
>>> Before doing above ensure vcpu0 is created after kernel_probe() is
>>> done and
>>> domain's guest execution mode (32-bit/64-bit) is set for dom0 and
>>> dom0less
>>> domains.
>>
>> The commit message doesn't mention anything about domains created by
>> the toolstack. In this case, from my understanding, the switch to
>> 64-bit domain happens *after* the vCPUs are created.
>>
>> At the moment, I think this is probably ok to call...
>>
>>>
>>> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
>>> ---
>>>   xen/arch/arm/domain.c                    |  3 +++
>>>   xen/arch/arm/domain_build.c              | 13 +++++--------
>>>   xen/common/device-tree/dom0less-build.c  |  6 +++---
>>>   xen/include/asm-generic/dom0less-build.h |  2 +-
>>>   4 files changed, 12 insertions(+), 12 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
>>> index 79a144e61be9..bbd4a764c696 100644
>>> --- a/xen/arch/arm/domain.c
>>> +++ b/xen/arch/arm/domain.c
>>> @@ -586,6 +586,9 @@ int arch_vcpu_create(struct vcpu *v)
>>>       if ( get_ssbd_state() == ARM_SSBD_RUNTIME )
>>>           v->arch.cpu_info->flags |= CPUINFO_WORKAROUND_2_FLAG;
>>> +    if ( is_64bit_domain(v->domain) )
>>> +        vcpu_switch_to_aarch64_mode(v);
>>
>> ... this function here because I *think* it would be NOP. But this
>> feels really fragile.
>
> The toolstack configures domain and vcpus through
> XEN_DOMCTL_set_address_size on Arm64:
> - toolstack creates domain and parses kernel binary: domain created
> with DOMAIN_32BIT mode by default
> - toolstack creates vcpus (still 32 bit mode):
> libxl__build_pre()->xc_domain_max_vcpus()
> - toolstack switches domain mode depending on kernel binary type:
> libxl__build_dom()->xc_dom_boot_mem_init(),
>   which triggers XEN_DOMCTL_set_address_size hypercall.
>   Xen: arm64: switches domain mode and re-configures vcpus:
> subarch_do_domctl()->set_address_size()
>
> So, this patch does not affect toolstack path, only optimizes Xen
> boots a bit.
>
> Also, during Xen boot or by toolstack - the domain is always created
> before it's type is even known, which, in turn,
> is based on guest binary which is parsed later during domain
> configuration stage.

This is an error which has existed in Xen since the outset.  ARM
inherited it from x86 PV (albeit the opposite way around).

It is literally backwards to create a VM in one mode, do some setup,
then decide "no actually I want it in the other mode".

For both x86 PV, and ARM it seems, parsing the kernel first and choosing
the right mode(s) at create time would be a substantial improvement.

As a note, x86 HVM has no concept of 64bit existing without 32bit.

~Andrew

Re: [XEN][PATCH 0/8] xen/arm64: make aarch32 support optional

[Grygorii Strashko]

On 23.07.25 13:54, Orzel, Michal wrote:
> 
> 
> On 23/07/2025 10:06, Julien Grall wrote:
>>
>>
>> On 23/07/2025 08:58, Grygorii Strashko wrote:
>>> From: Grygorii Strashko <grygorii_strashko@epam.com>
>>>
>>> Hi,
>>
>> Hi Grygorii,
>>
>>> Now Arm64 AArch32 guest support is always enabled and built-in while not
>>> all Arm64 platforms supports AArch32 or this support might not be needed.
>>
>> I am not entirely sure I like the proliferation of using CONFIG_* for
>> every single feature. This makes the testing a bit more complicated.
>>
>> Can you clarify what the goal with this patch?
> AArch32 is used quite rarely in embedded systems. Also, in Armv9A it might only
> be implemented at EL0 if at all. When focusing on safety certification, AArch32
> related code in Xen leaves a gap in terms of coverage that cannot really be
> justified in words. This leaves us with two options: either support it (lots of
> additional testing, requirements and documents would be needed) or compile it out.

FYI. bloat-o-meter report for this series with CONFIG_ARM64_AARCH32=n, CONFIG_EXPERT=y

add/remove: 0/6 grow/shrink: 2/32 up/down: 276/-5672 (-5396)
Function                                     old     new   delta
do_trap_guest_sync                          1588    1856    +268
start_xen                                   2456    2464      +8
make_cpus_node                               832     820     -12
arch_vcpu_create                             328     312     -16
vfp_save_state                               132     112     -20
vfp_restore_state                            132     112     -20
is_guest_pv32_psr                             20       -     -20
construct_hwdom                             1208    1188     -20
construct_domU                              2544    2524     -20
make_hypervisor_node                        2048    2016     -32
inject_undef64_exception                     112      80     -32
inject_abt64_exception                       164     132     -32
show_registers                               260     224     -36
arm64_set_domain_type                        128      88     -40
do_deprecated_hypercall                      124      80     -44
construct_domain                             316     272     -44
init_xen_cap_info                             96      44     -52
continue_new_vcpu                            188     136     -52
check_conditional_instr                      192     140     -52
vsmccc_handle_call                          1328    1272     -56
make_timer_node                              492     436     -56
vcpu_regs_user_to_hyp                        512     448     -64
vcpu_regs_hyp_to_user                        512     448     -64
do_common_cpu_on                             440     372     -68
cpsr_switch_mode                              80       -     -80
advance_pc                                   140      32    -108
do_trap_stage2_abort_guest                   836     724    -112
arch_set_info_guest                          336     184    -152
inject_undef_exception                       180      20    -160
arch_do_multicall_call                       632     468    -164
schedule_tail                                752     576    -176
decode_thumb2                                200       -    -200
subarch_do_domctl                            276      40    -236
inject_abt32_exception                       236       -    -236
do_debug_trap                                256       -    -256
hypercall_create_continuation               1228     848    -380
_show_registers                              832     380    -452
decode_instruction                          1032     476    -556
do_trap_hypercall                            640       -    -640
guest_walk_tables                           1896     984    -912
Total: Before=670920, After=665524, chg -0.80%
add/remove: 0/0 grow/shrink: 0/0 up/down: 0/0 (0)
Data                                         old     new   delta
Total: Before=448625, After=448625, chg +0.00%
add/remove: 0/0 grow/shrink: 1/0 up/down: 191/0 (191)
RO Data                                      old     new   delta
xen_config_data                             1345    1536    +191
Total: Before=18498, After=18689, chg +1.03%

-- 
Best regards,
-grygorii

Re: [XEN][PATCH 0/8] xen/arm64: make aarch32 support optional

[Julien Grall]

Hi,

On 23/07/2025 12:48, Grygorii Strashko wrote:
> 
> 
> On 23.07.25 13:54, Orzel, Michal wrote:
>>
>>
>> On 23/07/2025 10:06, Julien Grall wrote:
>>>
>>>
>>> On 23/07/2025 08:58, Grygorii Strashko wrote:
>>>> From: Grygorii Strashko <grygorii_strashko@epam.com>
>>>>
>>>> Hi,
>>>
>>> Hi Grygorii,
>>>
>>>> Now Arm64 AArch32 guest support is always enabled and built-in while 
>>>> not
>>>> all Arm64 platforms supports AArch32 or this support might not be 
>>>> needed.
>>>
>>> I am not entirely sure I like the proliferation of using CONFIG_* for
>>> every single feature. This makes the testing a bit more complicated.
>>>
>>> Can you clarify what the goal with this patch?
>> AArch32 is used quite rarely in embedded systems. Also, in Armv9A it 
>> might only
>> be implemented at EL0 if at all. When focusing on safety 
>> certification, AArch32
>> related code in Xen leaves a gap in terms of coverage that cannot 
>> really be
>> justified in words. This leaves us with two options: either support it 
>> (lots of
>> additional testing, requirements and documents would be needed) or 
>> compile it out.
> 
> FYI. bloat-o-meter report for this series with CONFIG_ARM64_AARCH32=n, 
> CONFIG_EXPERT=y

Thanks for sharing the bloat-o-meter. But I don't think the result below 
warrant a new config. The reason provided by Michal is a better reason 
as the impact on safety certification is more significant.

Cheers,

-- 
Julien Grall

Re: [XEN][PATCH 0/8] xen/arm64: make aarch32 support optional

[Grygorii Strashko]

On 23.07.25 15:02, Julien Grall wrote:
> Hi,
> 
> On 23/07/2025 12:48, Grygorii Strashko wrote:
>>
>>
>> On 23.07.25 13:54, Orzel, Michal wrote:
>>>
>>>
>>> On 23/07/2025 10:06, Julien Grall wrote:
>>>>
>>>>
>>>> On 23/07/2025 08:58, Grygorii Strashko wrote:
>>>>> From: Grygorii Strashko <grygorii_strashko@epam.com>
>>>>>
>>>>> Hi,
>>>>
>>>> Hi Grygorii,
>>>>
>>>>> Now Arm64 AArch32 guest support is always enabled and built-in while not
>>>>> all Arm64 platforms supports AArch32 or this support might not be needed.
>>>>
>>>> I am not entirely sure I like the proliferation of using CONFIG_* for
>>>> every single feature. This makes the testing a bit more complicated.
>>>>
>>>> Can you clarify what the goal with this patch?
>>> AArch32 is used quite rarely in embedded systems. Also, in Armv9A it might only
>>> be implemented at EL0 if at all. When focusing on safety certification, AArch32
>>> related code in Xen leaves a gap in terms of coverage that cannot really be
>>> justified in words. This leaves us with two options: either support it (lots of
>>> additional testing, requirements and documents would be needed) or compile it out.
>>
>> FYI. bloat-o-meter report for this series with CONFIG_ARM64_AARCH32=n, CONFIG_EXPERT=y
> 
> Thanks for sharing the bloat-o-meter. But I don't think the result below warrant a new config.
  The reason provided by Michal is a better reason as the impact on safety certification is more significant.

It just an additional info to illustrate achieved build-time optimization results
which reduces coverage gaps.

-- 
Best regards,
-grygorii

Re: [XEN][PATCH 2/8] xen/arm: move vcpu_switch_to_aarch64_mode() in arch_vcpu_create()

[Grygorii Strashko]

Hi Julien, Andrew Cooper <andrew.cooper3@citrix.com>

Thanks for your comment.

On 23.07.25 14:09, Julien Grall wrote:
> Hi,
> 
> On 23/07/2025 11:19, Grygorii Strashko wrote:
>> On 23.07.25 12:16, Julien Grall wrote:
>>> On 23/07/2025 08:58, Grygorii Strashko wrote:
>>>> From: Grygorii Strashko <grygorii_strashko@epam.com>
>>>>
>>>> Move vcpu_switch_to_aarch64_mode() in arch_vcpu_create() callback instead
>>>> of calling it manually from few different places after vcpu_create().
>>>>
>>>> Before doing above ensure vcpu0 is created after kernel_probe() is done and
>>>> domain's guest execution mode (32-bit/64-bit) is set for dom0 and dom0less
>>>> domains.
>>>
>>> The commit message doesn't mention anything about domains created by the toolstack. In this case, from my understanding, the switch to 64- bit domain happens *after* the vCPUs are created.
>>>
>>> At the moment, I think this is probably ok to call...
>>>
>>>>
>>>> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
>>>> ---
>>>>   xen/arch/arm/domain.c                    |  3 +++
>>>>   xen/arch/arm/domain_build.c              | 13 +++++--------
>>>>   xen/common/device-tree/dom0less-build.c  |  6 +++---
>>>>   xen/include/asm-generic/dom0less-build.h |  2 +-
>>>>   4 files changed, 12 insertions(+), 12 deletions(-)
>>>>
>>>> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
>>>> index 79a144e61be9..bbd4a764c696 100644
>>>> --- a/xen/arch/arm/domain.c
>>>> +++ b/xen/arch/arm/domain.c
>>>> @@ -586,6 +586,9 @@ int arch_vcpu_create(struct vcpu *v)
>>>>       if ( get_ssbd_state() == ARM_SSBD_RUNTIME )
>>>>           v->arch.cpu_info->flags |= CPUINFO_WORKAROUND_2_FLAG;
>>>> +    if ( is_64bit_domain(v->domain) )
>>>> +        vcpu_switch_to_aarch64_mode(v);
>>>
>>> ... this function here because I *think* it would be NOP. But this feels really fragile.
>>
>> The toolstack configures domain and vcpus through XEN_DOMCTL_set_address_size on Arm64:
>> - toolstack creates domain and parses kernel binary: domain created with DOMAIN_32BIT mode by default
>> - toolstack creates vcpus (still 32 bit mode): libxl__build_pre()-  >xc_domain_max_vcpus()
>> - toolstack switches domain mode depending on kernel binary type: libxl__build_dom()->xc_dom_boot_mem_init(),
>>    which triggers XEN_DOMCTL_set_address_size hypercall.
>>    Xen: arm64: switches domain mode and re-configures vcpus: subarch_do_domctl()->set_address_size()
>>
>> So, this patch does not affect toolstack path, only optimizes Xen boots a bit. 
> 
> Thanks for providing more detaisl. I am not sure what you mean by optimize. It reduces the number of places where vcpu_switch_to_aarch64_mode() is called, but there should be no difference in term of boot time.
> 
>>
>> Also, during Xen boot or by toolstack - the domain is always created before it's type is even known, which, in turn,
>> is based on guest binary which is parsed later during domain configuration stage.
> 
> What you are describing is the current situation. But this doesn't tell me *why* we can't provide the type when the domain is created.
> 
>>
>> I can add note in commit message "This patch doesn't affect on the toolstack Arm64 domain creation path as toolstack always
>> re-configures domain mode and vcpus through XEN_DOMCTL_set_address_size hypercall during domain configuration stage"
> 
> Well, as I wrote before, I find this code extremely fragile. And you so far, you don't seem to have address this concern in your reply. In fact...
> 
>>
>>>
>>> If the desire is to make 32-bit domain optional on Arm64. Then I think it would be better to pass the domain type when the domain
>>> is created (IOW add an extra flags to XEN_DOMCTL_createdomain). This will require more work, but it will be a lot more robust.
> 
> ... I proposed what I think is a better alternative. Did you consider it?


Yes, sorry for delay. I've been considering it and tried to study and estimate it, at least preliminary.
  ...It's definitely more work.

I'd be appreciated if you could clarify some points, please:

- Am understood correctly that proposal is to add "XEN_DOMCTL_CDF_is_32bit" flag which will be passed in struct xen_domctl_createdomain->flags?
   (or "XEN_DOMCTL_CDF_is_32bit_mode")

   [Arm] The Arm64 specific enum domain_type and (d)->arch.type can be dropped (use XEN_DOMCTL_CDF_is_32bit instead)
   [x86] the d->arch.pv.is_32bit can be dropped (use XEN_DOMCTL_CDF_is_32bit instead)

- is corresponding XEN_DOMINF_is_32bit needed?

- Assumption XEN_DOMCTL_set_address_size will become obsolete finally. Right?

After studying the topic, I have below thought regarding the requested change.
(I might be missing/misunderstanding smth, so will be appreciated for any advice or guidance to the right direction.
  Also I'm not very familiar with x86, sorry.)

The goal:
- domain mode (32bit/64bit) should be determined by probing guest binary before creating domain;
- domain mode (32bit/64bit) should be set during domain creation (XEN_DOMCTL_createdomain) using new flag XEN_DOMCTL_CDF_is_32bit.

Possible steps:
1) Introduce XEN_DOMCTL_CDF_is_32bit flag;
2) Arm: re-work regular dom0 creation code;
    challenges: cross references kernel_info vs domain.
3) Arm: re-work dom0les boot mode;
4) x86_pv: re-work regular PV dom0 creation code;
5) x86_hvm: ???;
6) toolstack: re-work domain creation, so guest binary probed and domain mode determined before creating domain;
    challenges: running "bootloader" to get guest binaries seems the most difficult part.
7) de-scope XEN_DOMCTL_set_address_size;

The toolstack behavior is left unchanged until step 6.

Hence the amount of work is significant It's preferred to be done in independent phases (series) to
easy review and testing process.

I'd try to come up with patches for items 1-3 as the first phase.

Thank you.

-- 
Best regards,
-grygorii

Re: [XEN][PATCH 0/8] xen/arm64: make aarch32 support optional

[Grygorii Strashko]

Hi All,

On 23.07.25 15:12, Grygorii Strashko wrote:
> 
> 
> On 23.07.25 15:02, Julien Grall wrote:
>> Hi,
>>
>> On 23/07/2025 12:48, Grygorii Strashko wrote:
>>>
>>>
>>> On 23.07.25 13:54, Orzel, Michal wrote:
>>>>
>>>>
>>>> On 23/07/2025 10:06, Julien Grall wrote:
>>>>>
>>>>>
>>>>> On 23/07/2025 08:58, Grygorii Strashko wrote:
>>>>>> From: Grygorii Strashko <grygorii_strashko@epam.com>
>>>>>>
>>>>>> Hi,
>>>>>
>>>>> Hi Grygorii,
>>>>>
>>>>>> Now Arm64 AArch32 guest support is always enabled and built-in while not
>>>>>> all Arm64 platforms supports AArch32 or this support might not be needed.
>>>>>
>>>>> I am not entirely sure I like the proliferation of using CONFIG_* for
>>>>> every single feature. This makes the testing a bit more complicated.
>>>>>
>>>>> Can you clarify what the goal with this patch?
>>>> AArch32 is used quite rarely in embedded systems. Also, in Armv9A it might only
>>>> be implemented at EL0 if at all. When focusing on safety certification, AArch32
>>>> related code in Xen leaves a gap in terms of coverage that cannot really be
>>>> justified in words. This leaves us with two options: either support it (lots of
>>>> additional testing, requirements and documents would be needed) or compile it out.
>>>
>>> FYI. bloat-o-meter report for this series with CONFIG_ARM64_AARCH32=n, CONFIG_EXPERT=y
>>
>> Thanks for sharing the bloat-o-meter. But I don't think the result below warrant a new config.
>   The reason provided by Michal is a better reason as the impact on safety certification is more significant.
> 
> It just an additional info to illustrate achieved build-time optimization results
> which reduces coverage gaps.
> 

Would it be reasonable to send patches which split arm64/arm32 code separately (Patches 1,3,4,7)?
(with comments applied).

-- 
Best regards,
-grygorii