[XEN][PATCH 0/7] xen/arm: rework to probe kernel before creating domains during xen boot

[XEN][PATCH 0/7] xen/arm: rework to probe kernel before creating domains during xen boot

[Grygorii Strashko]

From: Grygorii Strashko <grygorii_strashko@epam.com>

This series follows discussion [1][2] which pointed to "historical" issue present in Xen and
related to creating domains on 64bit Arches which allows running both 64/32bit guests (like Arm AArch64):

Now, during Xen boot or by toolstack, the domain is always created before knowing the guest type (32/64bit).
For example, on ARM64 during Xen boot:
- dom0 is created with default type 32bit
- vcpu[0] is created
- kernel binary probed and guest type is determined (for example 64bit)
- dom0 type changed according to guest type causing vcpu[0] reconfiguration
  (with restriction applied that domain type have to be properly set before allocating domain'a memory)

The same domain creation sequence executed for dom0less boot and for creating domains by toolstack
(The toolstack uses XEN_DOMCTL_set_address_size hypercall to reconfigure domain type).

As indicated by Julien Grall and Andrew Cooper, above domain creation sequence
is not robust and fragile, so it was proposed to introduce extra flags to XEN_DOMCTL_createdomain
to allow configuring domain type properly at domain creation time and perform further rework of
domain creation sequence to probe guest type before creating domain.

Hence, this series:
- introduces extra "XEN_DOMCTL_CDF_is_32bits" flag which is intended to be used by
  64bit Arches for proper configuration of domain type when domain is created.

  Now it adds initial support for this flag form Arm64 arch only. The default Arm64 domain type is
  changed to 64bit:
    - the Arm Xen boot code is handling this case properly already;
    - for toolstack case the XEN_DOMCTL_set_address_size hypercall handling
    updated to forcibly configure domain type regardless of current domain type
    configuration. Hence toolstack configures vcpus and memory after
    configuring domain type it allows to start with domain default type AArch64
    and then switch to requested domain type and ensures all required domain
    settings applied.

  For Arm32 this flag is ignored.

- dom0 setup sequence reworked to probe kernel before creating domain

- dom0less domains setup sequence reworked to probe kernel before creating domains

Xen toolstack is not changed as part of this series and it expected to be reworked
further to follow the same approach to probe guest binary before creating domain.

Tested (Arm64):
- dom0 a32/a64 boot
- dom0less a32/a64 domU boot
- toolstack create a32/a64

[1] https://lists.xen.org/archives/html/xen-devel/2025-07/msg01647.html
[2] https://lists.xen.org/archives/html/xen-devel/2025-07/msg01648.html

Alejandro Vallejo (1):
  dom0less: Parse memory properties in the common bindings

Grygorii Strashko (6):
  xen/arm64: domctl: set_address_size add check for vcpus not
    initialized
  xen/domctl: introduce XEN_DOMCTL_CDF_is_32bits
  xen/fdt: kernel:  add generic is_32bit_type flag to struct kernel_info
  xen/arm: probe kernel before creating dom0
  dom0less: probe kernel before creating domains
  xen/dt: kernel: add assert(!domain) in kernel_probe

 xen/arch/arm/arm64/domctl.c                | 28 ++++++++++++---
 xen/arch/arm/dom0less-build.c              |  2 +-
 xen/arch/arm/domain.c                      | 10 +++++-
 xen/arch/arm/domain_build.c                | 40 ++++++++++++----------
 xen/arch/arm/include/asm/kernel.h          |  7 +---
 xen/arch/arm/kernel.c                      | 10 +++---
 xen/common/device-tree/dom0less-bindings.c |  3 ++
 xen/common/device-tree/dom0less-build.c    | 28 +++++++--------
 xen/common/device-tree/kernel.c            | 12 ++++---
 xen/common/domain.c                        |  3 +-
 xen/include/public/domctl.h                |  7 +++-
 xen/include/xen/bootfdt.h                  |  2 ++
 xen/include/xen/fdt-kernel.h               |  3 ++
 13 files changed, 97 insertions(+), 58 deletions(-)

-- 
2.34.1

[XEN][PATCH 1/7] xen/arm64: domctl: set_address_size add check for vcpus not initialized

[Grygorii Strashko]

From: Grygorii Strashko <grygorii_strashko@epam.com>

The vcpu ctx initialization (arch_set_info_guest()) is depends on proper
domain type (32/64bit) configuration, so check that vcpus are not
initialized when toolstack issues XEN_DOMCTL_set_address_size hypercall.

Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
---
 xen/arch/arm/arm64/domctl.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/xen/arch/arm/arm64/domctl.c b/xen/arch/arm/arm64/domctl.c
index 8720d126c97d..82eff26fb0d1 100644
--- a/xen/arch/arm/arm64/domctl.c
+++ b/xen/arch/arm/arm64/domctl.c
@@ -13,6 +13,19 @@
 #include <asm/arm64/sve.h>
 #include <asm/cpufeature.h>
 
+static bool vcpus_check_initialised(struct domain *d)
+{
+    struct vcpu *v;
+
+    for_each_vcpu(d, v)
+    {
+        if ( v->is_initialised )
+            return true;
+    }
+
+    return false;
+}
+
 static long switch_mode(struct domain *d, enum domain_type type)
 {
     struct vcpu *v;
@@ -21,6 +34,8 @@ static long switch_mode(struct domain *d, enum domain_type type)
         return -EINVAL;
     if ( domain_tot_pages(d) != 0 )
         return -EBUSY;
+    if ( vcpus_check_initialised(d) )
+        return -EBUSY;
     if ( d->arch.type == type )
         return 0;
 
-- 
2.34.1

[XEN][PATCH 2/7] xen/domctl: introduce XEN_DOMCTL_CDF_is_32bits

[Grygorii Strashko]

From: Grygorii Strashko <grygorii_strashko@epam.com>

This patch follows discussion [1][2] which is pointed to "historical" issue
present in Xen and related to creating domains sequence on 64bit Arches
which allows running both 64/32bit guests (like AArch64):

Now, during Xen boot or by toolstack, the domain is always created before
knowing the guest type (32/64bit). For example, on ARM64 during Xen boot:
- dom0 is created with default type 32bit
- vcpu[0] is created
- kernel binary probed and guest type is determined (for example 64bit)
- dom0 type changed according to guest type causing vcpu[0] reconfiguration
  (with restriction applied that domain type have to be properly set before
  allocating domain'a memory)

The same domain creation sequence is executed for dom0less boot and for
creating domains by toolstack (The toolstack uses
XEN_DOMCTL_set_address_size hypercall to reconfigure domain type).

As indicated by Julien Grall and Andrew Cooper, above domain creation
sequence is not robust and fragile, so it was proposed to introduce extra
flags to XEN_DOMCTL_createdomain to allow configuring domain type properly
at domain creation time and perform further rework of domain creation
sequence to probe guest type before creating domain.

Hence, this patch introduces extra "XEN_DOMCTL_CDF_is_32bits" flag which is
intended to be used by 64bit Arches for proper configuration of domain type
when domain is created.

Now it adds initial support for this flag for Arm64 arch only. The default
Arm64 domain type is changed to 64bit:
- the Arm Xen boot code is handling this case properly already;
- for toolstack case the XEN_DOMCTL_set_address_size hypercall handling
updated to forcibly configure domain type regardless of current domain type
configuration. Hence toolstack configures vcpus and memory after
configuring domain type it allows to start with domain default AArch64 type
and then switch to requested domain type and ensures all required domain
settings applied.

For Arm32 this flag is ignored.

Note. For Arm64, Once toolstack is updated to probe guest binary before
creating domain the XEN_DOMCTL_set_address_size will become obsolete.

[1] https://lists.xen.org/archives/html/xen-devel/2025-07/msg01647.html
[2] https://lists.xen.org/archives/html/xen-devel/2025-07/msg01648.html

Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
---
 xen/arch/arm/arm64/domctl.c | 13 +++++++++----
 xen/arch/arm/domain.c       | 10 +++++++++-
 xen/common/domain.c         |  3 ++-
 xen/include/public/domctl.h |  7 ++++++-
 4 files changed, 26 insertions(+), 7 deletions(-)

diff --git a/xen/arch/arm/arm64/domctl.c b/xen/arch/arm/arm64/domctl.c
index 82eff26fb0d1..5346a533d888 100644
--- a/xen/arch/arm/arm64/domctl.c
+++ b/xen/arch/arm/arm64/domctl.c
@@ -26,6 +26,11 @@ static bool vcpus_check_initialised(struct domain *d)
     return false;
 }
 
+static void vcpu_switch_to_aarch32_mode(struct vcpu *v)
+{
+    v->arch.hcr_el2 &= ~HCR_RW;
+}
+
 static long switch_mode(struct domain *d, enum domain_type type)
 {
     struct vcpu *v;
@@ -36,14 +41,14 @@ static long switch_mode(struct domain *d, enum domain_type type)
         return -EBUSY;
     if ( vcpus_check_initialised(d) )
         return -EBUSY;
-    if ( d->arch.type == type )
-        return 0;
 
     d->arch.type = type;
 
-    if ( is_64bit_domain(d) )
-        for_each_vcpu(d, v)
+    for_each_vcpu(d, v)
+        if ( is_64bit_domain(d) )
             vcpu_switch_to_aarch64_mode(v);
+        else
+            vcpu_switch_to_aarch32_mode(v);
 
     return 0;
 }
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 79a144e61be9..078002f964ba 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -613,7 +613,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
     unsigned int flags_required = (XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap);
     unsigned int flags_optional = (XEN_DOMCTL_CDF_iommu | XEN_DOMCTL_CDF_vpmu |
                                    XEN_DOMCTL_CDF_xs_domain |
-                                   XEN_DOMCTL_CDF_trap_unmapped_accesses );
+                                   XEN_DOMCTL_CDF_trap_unmapped_accesses |
+                                   XEN_DOMCTL_CDF_is_32bits );
     unsigned int sve_vl_bits = sve_decode_vl(config->arch.sve_vl);
 
     if ( (config->flags & ~flags_optional) != flags_required )
@@ -711,6 +712,13 @@ int arch_domain_create(struct domain *d,
 
     BUILD_BUG_ON(GUEST_MAX_VCPUS < MAX_VIRT_CPUS);
 
+#ifdef CONFIG_ARM_64
+    if ( d->options & XEN_DOMCTL_CDF_is_32bits )
+        d->arch.type = DOMAIN_32BIT;
+    else
+        d->arch.type = DOMAIN_64BIT;
+#endif
+
 #ifdef CONFIG_IOREQ_SERVER
     ioreq_domain_init(d);
 #endif
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 303c338ef293..3193deb9c6bd 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -722,7 +722,8 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config)
            XEN_DOMCTL_CDF_s3_integrity | XEN_DOMCTL_CDF_oos_off |
            XEN_DOMCTL_CDF_xs_domain | XEN_DOMCTL_CDF_iommu |
            XEN_DOMCTL_CDF_nested_virt | XEN_DOMCTL_CDF_vpmu |
-           XEN_DOMCTL_CDF_trap_unmapped_accesses) )
+           XEN_DOMCTL_CDF_trap_unmapped_accesses |
+           XEN_DOMCTL_CDF_is_32bits) )
     {
         dprintk(XENLOG_INFO, "Unknown CDF flags %#x\n", config->flags);
         return -EINVAL;
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index a69dd960840a..ca59995f6c4d 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -68,9 +68,14 @@ struct xen_domctl_createdomain {
 #define XEN_DOMCTL_CDF_vpmu           (1U << 7)
 /* Should we trap guest accesses to unmapped addresses? */
 #define XEN_DOMCTL_CDF_trap_unmapped_accesses  (1U << 8)
+/*
+ * Is this domain running 32bit guest?
+ * Used for 64bits arches.
+ */
+#define XEN_DOMCTL_CDF_is_32bits (1U << 9)
 
 /* Max XEN_DOMCTL_CDF_* constant.  Used for ABI checking. */
-#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_trap_unmapped_accesses
+#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_is_32bits
 
     uint32_t flags;
 
-- 
2.34.1

[XEN][PATCH 5/7] dom0less: Parse memory properties in the common bindings

[Grygorii Strashko]

From: Alejandro Vallejo <alejandro.garciavallejo@amd.com>

Move the "memory" binding from dom0less-build to dom0less-bindings.

Signed-off-by: Alejandro Vallejo <alejandro.garciavallejo@amd.com>
Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
---
 xen/common/device-tree/dom0less-bindings.c | 3 +++
 xen/common/device-tree/dom0less-build.c    | 8 +-------
 xen/include/xen/bootfdt.h                  | 2 ++
 3 files changed, 6 insertions(+), 7 deletions(-)

diff --git a/xen/common/device-tree/dom0less-bindings.c b/xen/common/device-tree/dom0less-bindings.c
index 41d72d0d580a..4513f03e367b 100644
--- a/xen/common/device-tree/dom0less-bindings.c
+++ b/xen/common/device-tree/dom0less-bindings.c
@@ -141,5 +141,8 @@ int __init parse_dom0less_node(struct dt_device_node *node,
         panic("'llc-colors' found, but LLC coloring is disabled\n");
 #endif
 
+    if ( !dt_property_read_u64(node, "memory", &bd->memory) )
+        panic("missing memory binding for %s.\n", dt_node_name(node));
+
     return arch_parse_dom0less_node(node, bd);
 }
diff --git a/xen/common/device-tree/dom0less-build.c b/xen/common/device-tree/dom0less-build.c
index 6bb038111de9..8906e35e6978 100644
--- a/xen/common/device-tree/dom0less-build.c
+++ b/xen/common/device-tree/dom0less-build.c
@@ -738,14 +738,8 @@ static int __init construct_domU(struct kernel_info *kinfo,
     struct domain *d = kinfo->bd.d;
     const char *dom0less_enhanced;
     int rc;
-    u64 mem;
+    uint64_t mem = kinfo->bd.memory;
 
-    rc = dt_property_read_u64(node, "memory", &mem);
-    if ( !rc )
-    {
-        printk("Error building DomU: cannot read \"memory\" property\n");
-        return -EINVAL;
-    }
     kinfo->unassigned_mem = (paddr_t)mem * SZ_1K;
 
     rc = domain_p2m_set_allocation(d, mem, node);
diff --git a/xen/include/xen/bootfdt.h b/xen/include/xen/bootfdt.h
index 0e82ccea2f84..0b73ed835000 100644
--- a/xen/include/xen/bootfdt.h
+++ b/xen/include/xen/bootfdt.h
@@ -121,6 +121,8 @@ struct boot_domain {
     struct boot_module *initrd;
 
     const char *cmdline;
+    /* Memory the domain is expected to have attached after creation */
+    uint64_t memory;
 
     /* Input arguments to create_domain() */
     struct xen_domctl_createdomain create_cfg;
-- 
2.34.1

[XEN][PATCH 3/7] xen/fdt: kernel: add generic is_32bit_type flag to struct kernel_info

[Grygorii Strashko]

From: Grygorii Strashko <grygorii_strashko@epam.com>

It is possible to have 32/64bit guests not only on Arm64 arch, so
add generic is_32bit_type flag to the struct kernel_info instead of Arm64
specific enum domain_type type.

Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
---
 xen/arch/arm/dom0less-build.c     |  2 +-
 xen/arch/arm/domain_build.c       |  6 +++---
 xen/arch/arm/include/asm/kernel.h |  7 +------
 xen/arch/arm/kernel.c             | 10 +++++-----
 xen/include/xen/fdt-kernel.h      |  3 +++
 5 files changed, 13 insertions(+), 15 deletions(-)

diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index c8d07213e247..3ac2da0b3576 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -241,7 +241,7 @@ int __init make_arch_nodes(struct kernel_info *kinfo)
 void __init set_domain_type(struct domain *d, struct kernel_info *kinfo)
 {
     /* type must be set before allocate memory */
-    d->arch.type = kinfo->arch.type;
+    d->arch.type = kinfo->is_32bit_type ? DOMAIN_32BIT : DOMAIN_64BIT;
 }
 #else
 void __init set_domain_type(struct domain *d, struct kernel_info *kinfo)
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 463ae4474d30..fc9bcc6fbbd5 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1874,13 +1874,13 @@ int __init construct_domain(struct domain *d, struct kernel_info *kinfo)
 
 #ifdef CONFIG_ARM_64
     /* if aarch32 mode is not supported at EL1 do not allow 32-bit domain */
-    if ( !(cpu_has_el1_32) && kinfo->arch.type == DOMAIN_32BIT )
+    if ( !(cpu_has_el1_32) && kinfo->is_32bit_type )
     {
         printk("Platform does not support 32-bit domain\n");
         return -EINVAL;
     }
 
-    if ( is_sve_domain(d) && (kinfo->arch.type == DOMAIN_32BIT) )
+    if ( is_sve_domain(d) && kinfo->is_32bit_type )
     {
         printk("SVE is not available for 32-bit domain\n");
         return -EINVAL;
@@ -1996,7 +1996,7 @@ int __init construct_hwdom(struct kernel_info *kinfo,
 
 #ifdef CONFIG_ARM_64
     /* type must be set before allocate_memory */
-    d->arch.type = kinfo->arch.type;
+    d->arch.type = kinfo->is_32bit_type ? DOMAIN_32BIT : DOMAIN_64BIT;
 #endif
     find_gnttab_region(d, kinfo);
     if ( is_domain_direct_mapped(d) )
diff --git a/xen/arch/arm/include/asm/kernel.h b/xen/arch/arm/include/asm/kernel.h
index 7c3b7fde5b64..150cd9e30944 100644
--- a/xen/arch/arm/include/asm/kernel.h
+++ b/xen/arch/arm/include/asm/kernel.h
@@ -8,12 +8,7 @@
 
 #include <asm/domain.h>
 
-struct arch_kernel_info
-{
-#ifdef CONFIG_ARM_64
-    enum domain_type type;
-#endif
-
+struct arch_kernel_info {
     /* Enable pl011 emulation */
     bool vpl011;
 };
diff --git a/xen/arch/arm/kernel.c b/xen/arch/arm/kernel.c
index 48f4b56d0ccc..607758bedb22 100644
--- a/xen/arch/arm/kernel.c
+++ b/xen/arch/arm/kernel.c
@@ -103,7 +103,7 @@ static paddr_t __init kernel_zimage_place(struct kernel_info *info)
     paddr_t load_addr;
 
 #ifdef CONFIG_ARM_64
-    if ( (info->arch.type == DOMAIN_64BIT) && (info->zimage.start == 0) )
+    if ( (!info->is_32bit_type) && (info->zimage.start == 0) )
         return mem->bank[0].start + info->zimage.text_offset;
 #endif
 
@@ -274,10 +274,10 @@ int __init kernel_uimage_probe(struct kernel_info *info,
     switch ( uimage.arch )
     {
     case IH_ARCH_ARM:
-        info->arch.type = DOMAIN_32BIT;
+        info->is_32bit_type = true;
         break;
     case IH_ARCH_ARM64:
-        info->arch.type = DOMAIN_64BIT;
+        info->is_32bit_type = false;
         break;
     default:
         printk(XENLOG_ERR "Unsupported uImage arch type %d\n", uimage.arch);
@@ -347,7 +347,7 @@ static int __init kernel_zimage64_probe(struct kernel_info *info,
 
     info->load = kernel_zimage_load;
 
-    info->arch.type = DOMAIN_64BIT;
+    info->is_32bit_type = false;
 
     return 0;
 }
@@ -399,7 +399,7 @@ static int __init kernel_zimage32_probe(struct kernel_info *info,
     info->load = kernel_zimage_load;
 
 #ifdef CONFIG_ARM_64
-    info->arch.type = DOMAIN_32BIT;
+    info->is_32bit_type = true;
 #endif
 
     return 0;
diff --git a/xen/include/xen/fdt-kernel.h b/xen/include/xen/fdt-kernel.h
index 33a60597bb4d..c862ff6c17ca 100644
--- a/xen/include/xen/fdt-kernel.h
+++ b/xen/include/xen/fdt-kernel.h
@@ -41,6 +41,9 @@ struct kernel_info {
     /* Enable/Disable PV drivers interfaces */
     uint16_t dom0less_feature;
 
+    /* Is 32bits guest detected? */
+    bool is_32bit_type;
+
     /* Interrupt controller phandle */
     uint32_t phandle_intc;
 
-- 
2.34.1

[XEN][PATCH 4/7] xen/arm: probe kernel before creating dom0

[Grygorii Strashko]

From: Grygorii Strashko <grygorii_strashko@epam.com>

As was indicated in [1][2], performing guest kernel probe after creating
domain is not robust.

Hence, rework Arm dom0 creation sequence to probe guest kernel first,
before creating domain, so guest type (32/64bit) can be properly identified
and dom0 type configured correctly from very beginning.

To move kernel_probe() before dom0 domain_create() the domain references
need to be removed from kernel_probe():
- remove ASSERT(is_hardware_domain(info->bd.d))
- remove printing domain id ("%pd") from "Loading kernel from boot module.."
  log msg.

[1] https://lists.xen.org/archives/html/xen-devel/2025-07/msg01647.html
[2] https://lists.xen.org/archives/html/xen-devel/2025-07/msg01648.html

Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
---
 xen/arch/arm/domain_build.c     | 32 +++++++++++++++++++-------------
 xen/common/device-tree/kernel.c |  6 ++----
 2 files changed, 21 insertions(+), 17 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index fc9bcc6fbbd5..59966f1bcd49 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1954,16 +1954,13 @@ int __init construct_domain(struct domain *d, struct kernel_info *kinfo)
     return 0;
 }
 
-static int __init construct_dom0(struct domain *d)
+static int __init construct_dom0(struct kernel_info *kinfo)
 {
-    struct kernel_info kinfo = KERNEL_INFO_INIT;
-    int rc;
+    struct domain *d = kinfo->bd.d;
 
     /* Sanity! */
     BUG_ON(d->domain_id != 0);
 
-    printk("*** LOADING DOMAIN 0 ***\n");
-
     /* The ordering of operands is to work around a clang5 issue. */
     if ( CONFIG_DOM0_MEM[0] && !dom0_mem_set )
         parse_dom0_mem(CONFIG_DOM0_MEM);
@@ -1976,14 +1973,9 @@ static int __init construct_dom0(struct domain *d)
 
     d->max_pages = dom0_mem >> PAGE_SHIFT;
 
-    kinfo.unassigned_mem = dom0_mem;
-    kinfo.bd.d = d;
+    kinfo->unassigned_mem = dom0_mem;
 
-    rc = kernel_probe(&kinfo, NULL);
-    if ( rc < 0 )
-        return rc;
-
-    return construct_hwdom(&kinfo, NULL);
+    return construct_hwdom(kinfo, NULL);
 }
 
 int __init construct_hwdom(struct kernel_info *kinfo,
@@ -2040,6 +2032,7 @@ int __init construct_hwdom(struct kernel_info *kinfo,
 
 void __init create_dom0(void)
 {
+    struct kernel_info kinfo = KERNEL_INFO_INIT;
     struct domain *dom0;
     struct xen_domctl_createdomain dom0_cfg = {
         .flags = XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap |
@@ -2052,6 +2045,17 @@ void __init create_dom0(void)
     unsigned int flags = CDF_privileged | CDF_hardware;
     int rc;
 
+    printk("*** LOADING DOMAIN 0 ***\n");
+
+    rc = kernel_probe(&kinfo, NULL);
+    if ( rc < 0 )
+        panic("Error probing  domain 0 guest kernel (rc = %d)\n", rc);
+
+#ifdef CONFIG_ARM_64
+    if ( kinfo.is_32bit_type )
+        dom0_cfg.flags |= XEN_DOMCTL_CDF_is_32bits;
+#endif
+
     /* The vGIC for DOM0 is exactly emulating the hardware GIC */
     dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
     dom0_cfg.arch.nr_spis = VGIC_DEF_NR_SPIS;
@@ -2078,13 +2082,15 @@ void __init create_dom0(void)
     if ( IS_ERR(dom0) )
         panic("Error creating domain 0 (rc = %ld)\n", PTR_ERR(dom0));
 
+    kinfo.bd.d = dom0;
+
     if ( llc_coloring_enabled && (rc = dom0_set_llc_colors(dom0)) )
         panic("Error initializing LLC coloring for domain 0 (rc = %d)\n", rc);
 
     if ( vcpu_create(dom0, 0) == NULL )
         panic("Error creating domain 0 vcpu0\n");
 
-    rc = construct_dom0(dom0);
+    rc = construct_dom0(&kinfo);
     if ( rc )
         panic("Could not set up DOM0 guest OS (rc = %d)\n", rc);
 
diff --git a/xen/common/device-tree/kernel.c b/xen/common/device-tree/kernel.c
index 28096121a52d..f67229f66d30 100644
--- a/xen/common/device-tree/kernel.c
+++ b/xen/common/device-tree/kernel.c
@@ -140,8 +140,6 @@ int __init kernel_probe(struct kernel_info *info,
     /* domain is NULL only for the hardware domain */
     if ( domain == NULL )
     {
-        ASSERT(is_hardware_domain(info->bd.d));
-
         mod = boot_module_find_by_kind(BOOTMOD_KERNEL);
 
         info->bd.kernel = mod;
@@ -204,8 +202,8 @@ int __init kernel_probe(struct kernel_info *info,
         return -ENOENT;
     }
 
-    printk("Loading %pd kernel from boot module @ %"PRIpaddr"\n",
-           info->bd.d, info->bd.kernel->start);
+    printk("Loading kernel from boot module @ %"PRIpaddr"\n",
+           info->bd.kernel->start);
     if ( info->bd.initrd )
         printk("Loading ramdisk from boot module @ %"PRIpaddr"\n",
                info->bd.initrd->start);
-- 
2.34.1

[XEN][PATCH 7/7] xen/dt: kernel: add assert(!domain) in kernel_probe

[Grygorii Strashko]

From: Grygorii Strashko <grygorii_strashko@epam.com>

Existing users of kernel_probe() reworked to probe guest kernel before
creating domain and any new users must follow the same approach, so add
assert to check that domain was not created before calling kernel_probe().

Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
---
 xen/common/device-tree/kernel.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/xen/common/device-tree/kernel.c b/xen/common/device-tree/kernel.c
index f67229f66d30..445a521607e0 100644
--- a/xen/common/device-tree/kernel.c
+++ b/xen/common/device-tree/kernel.c
@@ -128,6 +128,12 @@ int __init kernel_probe(struct kernel_info *info,
     u64 kernel_addr, initrd_addr, dtb_addr, size;
     int rc;
 
+    /*
+     * Kernel probe should be done and guest type (32/64bit) detected before
+     * creating domain and any new users must follow this approach.
+     */
+    ASSERT(!info->bd.d);
+
     /*
      * We need to initialize start to 0. This field may be populated during
      * kernel_xxx_probe() if the image has a fixed entry point (for e.g.
-- 
2.34.1

[XEN][PATCH 6/7] dom0less: probe kernel before creating domains

[Grygorii Strashko]

From: Grygorii Strashko <grygorii_strashko@epam.com>

As was indicated in [1][2], performing guest kernel probe after creating
domain is not robust.

Hence, rework dom0less domain creation sequence to probe guest kernels
first, before creating domains, so guest type (32/64bit) can be properly
identified and domain type configured correctly from very beginning.

[1] https://lists.xen.org/archives/html/xen-devel/2025-07/msg01647.html
[2] https://lists.xen.org/archives/html/xen-devel/2025-07/msg01648.html

Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
---
 xen/arch/arm/domain_build.c             |  4 ----
 xen/common/device-tree/dom0less-build.c | 20 +++++++++++---------
 2 files changed, 11 insertions(+), 13 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 59966f1bcd49..bcd642477d1d 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1986,10 +1986,6 @@ int __init construct_hwdom(struct kernel_info *kinfo,
 
     iommu_hwdom_init(d);
 
-#ifdef CONFIG_ARM_64
-    /* type must be set before allocate_memory */
-    d->arch.type = kinfo->is_32bit_type ? DOMAIN_32BIT : DOMAIN_64BIT;
-#endif
     find_gnttab_region(d, kinfo);
     if ( is_domain_direct_mapped(d) )
         allocate_memory_11(d, kinfo);
diff --git a/xen/common/device-tree/dom0less-build.c b/xen/common/device-tree/dom0less-build.c
index 8906e35e6978..14829ec465d1 100644
--- a/xen/common/device-tree/dom0less-build.c
+++ b/xen/common/device-tree/dom0less-build.c
@@ -746,9 +746,6 @@ static int __init construct_domU(struct kernel_info *kinfo,
     if ( rc != 0 )
         return rc;
 
-    printk("*** LOADING DOMU cpus=%u memory=%#"PRIx64"KB ***\n",
-           d->max_vcpus, mem);
-
     rc = dt_property_read_string(node, "xen,enhanced", &dom0less_enhanced);
     if ( rc == -EILSEQ ||
          rc == -ENODATA ||
@@ -770,12 +767,6 @@ static int __init construct_domU(struct kernel_info *kinfo,
 
     d->max_pages = ((paddr_t)mem * SZ_1K) >> PAGE_SHIFT;
 
-    rc = kernel_probe(kinfo, node);
-    if ( rc < 0 )
-        return rc;
-
-    set_domain_type(d, kinfo);
-
     if ( is_hardware_domain(d) )
     {
         rc = construct_hwdom(kinfo, node);
@@ -836,6 +827,17 @@ void __init create_domUs(void)
         if ( (max_init_domid + 1) >= DOMID_FIRST_RESERVED )
             panic("No more domain IDs available\n");
 
+        printk("*** LOADING DOMU cpus=%u memory=%#" PRIx64 "KB ***\n",
+               ki.bd.create_cfg.max_vcpus, ki.bd.memory);
+
+        rc = kernel_probe(&ki, node);
+        if ( rc < 0 )
+            panic("Error probing domain %s guest kernel (rc = %d)\n",
+                  dt_node_name(node), rc);
+
+        if ( ki.is_32bit_type )
+            ki.bd.create_cfg.flags |= XEN_DOMCTL_CDF_is_32bits;
+
         /*
          * The variable max_init_domid is initialized with zero, so here it's
          * very important to use the pre-increment operator to call
-- 
2.34.1

Re: [XEN][PATCH 2/7] xen/domctl: introduce XEN_DOMCTL_CDF_is_32bits

[Jan Beulich]

On 31.07.2025 11:42, Grygorii Strashko wrote:
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -613,7 +613,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>      unsigned int flags_required = (XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap);
>      unsigned int flags_optional = (XEN_DOMCTL_CDF_iommu | XEN_DOMCTL_CDF_vpmu |
>                                     XEN_DOMCTL_CDF_xs_domain |
> -                                   XEN_DOMCTL_CDF_trap_unmapped_accesses );
> +                                   XEN_DOMCTL_CDF_trap_unmapped_accesses |
> +                                   XEN_DOMCTL_CDF_is_32bits );
>      unsigned int sve_vl_bits = sve_decode_vl(config->arch.sve_vl);
>  
>      if ( (config->flags & ~flags_optional) != flags_required )
> @@ -711,6 +712,13 @@ int arch_domain_create(struct domain *d,
>  
>      BUILD_BUG_ON(GUEST_MAX_VCPUS < MAX_VIRT_CPUS);
>  
> +#ifdef CONFIG_ARM_64
> +    if ( d->options & XEN_DOMCTL_CDF_is_32bits )
> +        d->arch.type = DOMAIN_32BIT;
> +    else
> +        d->arch.type = DOMAIN_64BIT;
> +#endif
> +
>  #ifdef CONFIG_IOREQ_SERVER
>      ioreq_domain_init(d);
>  #endif

No adjustment to x86'es arch_sanitise_domain_config()? Until the new bit
is actually properly handled, I think its use will need rejecting.

> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -68,9 +68,14 @@ struct xen_domctl_createdomain {
>  #define XEN_DOMCTL_CDF_vpmu           (1U << 7)
>  /* Should we trap guest accesses to unmapped addresses? */
>  #define XEN_DOMCTL_CDF_trap_unmapped_accesses  (1U << 8)
> +/*
> + * Is this domain running 32bit guest?
> + * Used for 64bits arches.
> + */
> +#define XEN_DOMCTL_CDF_is_32bits (1U << 9)

Please pad suitably, as is the case for XEN_DOMCTL_CDF_vpmu visible in
context. XEN_DOMCTL_CDF_trap_unmapped_accesses simply is too long to fit
that padding scheme.

Jan

Re: [XEN][PATCH 2/7] xen/domctl: introduce XEN_DOMCTL_CDF_is_32bits

[Alejandro Vallejo]

On Thu Jul 31, 2025 at 11:42 AM CEST, Grygorii Strashko wrote:
> From: Grygorii Strashko <grygorii_strashko@epam.com>
>
> This patch follows discussion [1][2] which is pointed to "historical" issue
> present in Xen and related to creating domains sequence on 64bit Arches
> which allows running both 64/32bit guests (like AArch64):
>
> Now, during Xen boot or by toolstack, the domain is always created before
> knowing the guest type (32/64bit). For example, on ARM64 during Xen boot:
> - dom0 is created with default type 32bit
> - vcpu[0] is created
> - kernel binary probed and guest type is determined (for example 64bit)
> - dom0 type changed according to guest type causing vcpu[0] reconfiguration
>   (with restriction applied that domain type have to be properly set before
>   allocating domain'a memory)
>
> The same domain creation sequence is executed for dom0less boot and for
> creating domains by toolstack (The toolstack uses
> XEN_DOMCTL_set_address_size hypercall to reconfigure domain type).
>
> As indicated by Julien Grall and Andrew Cooper, above domain creation
> sequence is not robust and fragile, so it was proposed to introduce extra
> flags to XEN_DOMCTL_createdomain to allow configuring domain type properly
> at domain creation time and perform further rework of domain creation
> sequence to probe guest type before creating domain.
>
> Hence, this patch introduces extra "XEN_DOMCTL_CDF_is_32bits" flag which is
> intended to be used by 64bit Arches for proper configuration of domain type
> when domain is created.
>
> Now it adds initial support for this flag for Arm64 arch only. The default
> Arm64 domain type is changed to 64bit:
> - the Arm Xen boot code is handling this case properly already;
> - for toolstack case the XEN_DOMCTL_set_address_size hypercall handling
> updated to forcibly configure domain type regardless of current domain type
> configuration. Hence toolstack configures vcpus and memory after
> configuring domain type it allows to start with domain default AArch64 type
> and then switch to requested domain type and ensures all required domain
> settings applied.
>
> For Arm32 this flag is ignored.
>
> Note. For Arm64, Once toolstack is updated to probe guest binary before
> creating domain the XEN_DOMCTL_set_address_size will become obsolete.
>
> [1] https://lists.xen.org/archives/html/xen-devel/2025-07/msg01647.html
> [2] https://lists.xen.org/archives/html/xen-devel/2025-07/msg01648.html
>
> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>

Neat idea. I like it.

> ---
>  xen/arch/arm/arm64/domctl.c | 13 +++++++++----
>  xen/arch/arm/domain.c       | 10 +++++++++-
>  xen/common/domain.c         |  3 ++-
>  xen/include/public/domctl.h |  7 ++++++-
>  4 files changed, 26 insertions(+), 7 deletions(-)
>
> diff --git a/xen/arch/arm/arm64/domctl.c b/xen/arch/arm/arm64/domctl.c
> index 82eff26fb0d1..5346a533d888 100644
> --- a/xen/arch/arm/arm64/domctl.c
> +++ b/xen/arch/arm/arm64/domctl.c
> @@ -26,6 +26,11 @@ static bool vcpus_check_initialised(struct domain *d)
>      return false;
>  }
>  
> +static void vcpu_switch_to_aarch32_mode(struct vcpu *v)
> +{
> +    v->arch.hcr_el2 &= ~HCR_RW;
> +}
> +
>  static long switch_mode(struct domain *d, enum domain_type type)
>  {
>      struct vcpu *v;
> @@ -36,14 +41,14 @@ static long switch_mode(struct domain *d, enum domain_type type)
>          return -EBUSY;
>      if ( vcpus_check_initialised(d) )
>          return -EBUSY;
> -    if ( d->arch.type == type )
> -        return 0;
>  
>      d->arch.type = type;
>  
> -    if ( is_64bit_domain(d) )
> -        for_each_vcpu(d, v)
> +    for_each_vcpu(d, v)
> +        if ( is_64bit_domain(d) )
>              vcpu_switch_to_aarch64_mode(v);
> +        else
> +            vcpu_switch_to_aarch32_mode(v);
>  
>      return 0;
>  }
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 79a144e61be9..078002f964ba 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -613,7 +613,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>      unsigned int flags_required = (XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap);
>      unsigned int flags_optional = (XEN_DOMCTL_CDF_iommu | XEN_DOMCTL_CDF_vpmu |
>                                     XEN_DOMCTL_CDF_xs_domain |
> -                                   XEN_DOMCTL_CDF_trap_unmapped_accesses );
> +                                   XEN_DOMCTL_CDF_trap_unmapped_accesses |
> +                                   XEN_DOMCTL_CDF_is_32bits );
>      unsigned int sve_vl_bits = sve_decode_vl(config->arch.sve_vl);
>  
>      if ( (config->flags & ~flags_optional) != flags_required )
> @@ -711,6 +712,13 @@ int arch_domain_create(struct domain *d,
>  
>      BUILD_BUG_ON(GUEST_MAX_VCPUS < MAX_VIRT_CPUS);
>  
> +#ifdef CONFIG_ARM_64
> +    if ( d->options & XEN_DOMCTL_CDF_is_32bits )
> +        d->arch.type = DOMAIN_32BIT;
> +    else
> +        d->arch.type = DOMAIN_64BIT;
> +#endif
> +
>  #ifdef CONFIG_IOREQ_SERVER
>      ioreq_domain_init(d);
>  #endif
> diff --git a/xen/common/domain.c b/xen/common/domain.c
> index 303c338ef293..3193deb9c6bd 100644
> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -722,7 +722,8 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config)
>             XEN_DOMCTL_CDF_s3_integrity | XEN_DOMCTL_CDF_oos_off |
>             XEN_DOMCTL_CDF_xs_domain | XEN_DOMCTL_CDF_iommu |
>             XEN_DOMCTL_CDF_nested_virt | XEN_DOMCTL_CDF_vpmu |
> -           XEN_DOMCTL_CDF_trap_unmapped_accesses) )
> +           XEN_DOMCTL_CDF_trap_unmapped_accesses |
> +           XEN_DOMCTL_CDF_is_32bits) )
>      {
>          dprintk(XENLOG_INFO, "Unknown CDF flags %#x\n", config->flags);
>          return -EINVAL;
> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> index a69dd960840a..ca59995f6c4d 100644
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -68,9 +68,14 @@ struct xen_domctl_createdomain {
>  #define XEN_DOMCTL_CDF_vpmu           (1U << 7)
>  /* Should we trap guest accesses to unmapped addresses? */
>  #define XEN_DOMCTL_CDF_trap_unmapped_accesses  (1U << 8)
> +/*
> + * Is this domain running 32bit guest?
> + * Used for 64bits arches.
> + */

On arm what this means seems clear because aarch64 (IIRC) start in 64bits, but
x86 doesn't. This needs expanding. For x86 HVM guest's start in 16 bits and work
their way up to 64 bits, and PVH starts in 32 and works towards 64bits.

Do we mean to hide long mode (or the ability to transition onto it) altogether
with this?

> +#define XEN_DOMCTL_CDF_is_32bits (1U << 9)

I don't like having a flag suggesting "set this for 32bit guests, unless your
hypervisor is 64bits". If anything because a 32bit dom0 doesn't have to care
whether the hypervisor is 32 or 64 bits.

Unless we go with "domains are by default created with as much *bitness* as
possible". If so, I'd suggest XEN_DOMCTL_CDF_32bits_max, which would work
with the "hide long mode" semantics. For 32bit hypervisors you could set it
or ignore it.

>  
>  /* Max XEN_DOMCTL_CDF_* constant.  Used for ABI checking. */
> -#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_trap_unmapped_accesses
> +#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_is_32bits
>  
>      uint32_t flags;
>  

x86, riscv and ppc need accounting for. If anything to fail the domctl if they
see the flag set (until they themselves can handle it).

Cheers,
Alejandro

Re: [XEN][PATCH 5/7] dom0less: Parse memory properties in the common bindings

[Alejandro Vallejo]

On Thu Jul 31, 2025 at 11:42 AM CEST, Grygorii Strashko wrote:
> From: Alejandro Vallejo <alejandro.garciavallejo@amd.com>
>
> Move the "memory" binding from dom0less-build to dom0less-bindings.
>
> Signed-off-by: Alejandro Vallejo <alejandro.garciavallejo@amd.com>
> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
> ---
>  xen/common/device-tree/dom0less-bindings.c | 3 +++
>  xen/common/device-tree/dom0less-build.c    | 8 +-------
>  xen/include/xen/bootfdt.h                  | 2 ++
>  3 files changed, 6 insertions(+), 7 deletions(-)
>
> diff --git a/xen/common/device-tree/dom0less-bindings.c b/xen/common/device-tree/dom0less-bindings.c
> index 41d72d0d580a..4513f03e367b 100644
> --- a/xen/common/device-tree/dom0less-bindings.c
> +++ b/xen/common/device-tree/dom0less-bindings.c
> @@ -141,5 +141,8 @@ int __init parse_dom0less_node(struct dt_device_node *node,
>          panic("'llc-colors' found, but LLC coloring is disabled\n");
>  #endif
>  
> +    if ( !dt_property_read_u64(node, "memory", &bd->memory) )
> +        panic("missing memory binding for %s.\n", dt_node_name(node));
> +
>      return arch_parse_dom0less_node(node, bd);
>  }
> diff --git a/xen/common/device-tree/dom0less-build.c b/xen/common/device-tree/dom0less-build.c
> index 6bb038111de9..8906e35e6978 100644
> --- a/xen/common/device-tree/dom0less-build.c
> +++ b/xen/common/device-tree/dom0less-build.c
> @@ -738,14 +738,8 @@ static int __init construct_domU(struct kernel_info *kinfo,
>      struct domain *d = kinfo->bd.d;
>      const char *dom0less_enhanced;
>      int rc;
> -    u64 mem;
> +    uint64_t mem = kinfo->bd.memory;
>  
> -    rc = dt_property_read_u64(node, "memory", &mem);
> -    if ( !rc )
> -    {
> -        printk("Error building DomU: cannot read \"memory\" property\n");
> -        return -EINVAL;
> -    }
>      kinfo->unassigned_mem = (paddr_t)mem * SZ_1K;
>  
>      rc = domain_p2m_set_allocation(d, mem, node);
> diff --git a/xen/include/xen/bootfdt.h b/xen/include/xen/bootfdt.h
> index 0e82ccea2f84..0b73ed835000 100644
> --- a/xen/include/xen/bootfdt.h
> +++ b/xen/include/xen/bootfdt.h
> @@ -121,6 +121,8 @@ struct boot_domain {
>      struct boot_module *initrd;
>  
>      const char *cmdline;
> +    /* Memory the domain is expected to have attached after creation */
> +    uint64_t memory;

Either state here that the units are 1K, or make the units bytes by multiplying
the result from the binding by 1K. If the latter, you can probably replace all
instances of "unassigned_mem" for "memory" in boot_domain and remove that field
in kernel_info, which goes in line with the intention of shrinking kernel_info
in favour of boot_domain.

Cheers,
Alejandro

Re: [XEN][PATCH 1/7] xen/arm64: domctl: set_address_size add check for vcpus not initialized

[Alejandro Vallejo]

On Thu Jul 31, 2025 at 11:42 AM CEST, Grygorii Strashko wrote:
> From: Grygorii Strashko <grygorii_strashko@epam.com>
>
> The vcpu ctx initialization (arch_set_info_guest()) is depends on proper
                                                       ^
                                                   stray "is"

> domain type (32/64bit) configuration, so check that vcpus are not
> initialized when toolstack issues XEN_DOMCTL_set_address_size hypercall.
>
> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
> ---
>  xen/arch/arm/arm64/domctl.c | 15 +++++++++++++++
>  1 file changed, 15 insertions(+)
>
> diff --git a/xen/arch/arm/arm64/domctl.c b/xen/arch/arm/arm64/domctl.c
> index 8720d126c97d..82eff26fb0d1 100644
> --- a/xen/arch/arm/arm64/domctl.c
> +++ b/xen/arch/arm/arm64/domctl.c
> @@ -13,6 +13,19 @@
>  #include <asm/arm64/sve.h>
>  #include <asm/cpufeature.h>
>  
> +static bool vcpus_check_initialised(struct domain *d)

nit: I'd call it vcpus_any_initialised(), that way it's far easier to
     see what it does without jumping into the implementation.

> +{
> +    struct vcpu *v;
> +
> +    for_each_vcpu(d, v)
> +    {
> +        if ( v->is_initialised )
> +            return true;
> +    }
> +
> +    return false;
> +}
> +
>  static long switch_mode(struct domain *d, enum domain_type type)
>  {
>      struct vcpu *v;
> @@ -21,6 +34,8 @@ static long switch_mode(struct domain *d, enum domain_type type)
>          return -EINVAL;
>      if ( domain_tot_pages(d) != 0 )
>          return -EBUSY;
> +    if ( vcpus_check_initialised(d) )
> +        return -EBUSY;
>      if ( d->arch.type == type )
>          return 0;
>  

Re: [XEN][PATCH 2/7] xen/domctl: introduce XEN_DOMCTL_CDF_is_32bits

[Teddy Astie]

Hello,

Le 31/07/2025 à 11:44, Grygorii Strashko a écrit :
> Hence, this patch introduces extra "XEN_DOMCTL_CDF_is_32bits" flag which is
> intended to be used by 64bit Arches for proper configuration of domain type
> when domain is created.
>
> Now it adds initial support for this flag for Arm64 arch only. The default
> Arm64 domain type is changed to 64bit:
> - the Arm Xen boot code is handling this case properly already;
> - for toolstack case the XEN_DOMCTL_set_address_size hypercall handling
> updated to forcibly configure domain type regardless of current domain type
> configuration. Hence toolstack configures vcpus and memory after
> configuring domain type it allows to start with domain default AArch64 type
> and then switch to requested domain type and ensures all required domain
> settings applied.
>
> For Arm32 this flag is ignored.
>
> Note. For Arm64, Once toolstack is updated to probe guest binary before
> creating domain the XEN_DOMCTL_set_address_size will become obsolete.
>
> [1] https://lists.xen.org/archives/html/xen-devel/2025-07/msg01647.html
> [2] https://lists.xen.org/archives/html/xen-devel/2025-07/msg01648.html
>
> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
> ---

I would prefer it be better presented as a platform-specific enum.
Something like

```
enum xen_arch_subarch {
   XEN_ARM_SUBARCH_NATIVE = 0,
   XEN_ARM_SUBARCH_ARMV7 = 1,
   XEN_ARM_SUBARCH_AARCH64 = 2,
   // ...
}
```

As there could be cases of limited backward compatibility, or multiple
diverging subarchs variants.

Teddy


Teddy Astie | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech

Re: [XEN][PATCH 2/7] xen/domctl: introduce XEN_DOMCTL_CDF_is_32bits

[Julien Grall]

Hi Teddy,

On 31/07/2025 15:02, Teddy Astie wrote:
> Hello,
> 
> Le 31/07/2025 à 11:44, Grygorii Strashko a écrit :
>> Hence, this patch introduces extra "XEN_DOMCTL_CDF_is_32bits" flag which is
>> intended to be used by 64bit Arches for proper configuration of domain type
>> when domain is created.
>>
>> Now it adds initial support for this flag for Arm64 arch only. The default
>> Arm64 domain type is changed to 64bit:
>> - the Arm Xen boot code is handling this case properly already;
>> - for toolstack case the XEN_DOMCTL_set_address_size hypercall handling
>> updated to forcibly configure domain type regardless of current domain type
>> configuration. Hence toolstack configures vcpus and memory after
>> configuring domain type it allows to start with domain default AArch64 type
>> and then switch to requested domain type and ensures all required domain
>> settings applied.
>>
>> For Arm32 this flag is ignored.
>>
>> Note. For Arm64, Once toolstack is updated to probe guest binary before
>> creating domain the XEN_DOMCTL_set_address_size will become obsolete.
>>
>> [1] https://lists.xen.org/archives/html/xen-devel/2025-07/msg01647.html
>> [2] https://lists.xen.org/archives/html/xen-devel/2025-07/msg01648.html
>>
>> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
>> ---
> 
> I would prefer it be better presented as a platform-specific enum.
> Something like
> 
> ```
> enum xen_arch_subarch {

We need to be cautious when using enum in the public interface because, 
if I am not mistaken, the size of the enum if not stable.

>     XEN_ARM_SUBARCH_NATIVE = 0,

"native" is a bit ambiguous when a platform is able to support 32-bit 
and 64-bit. Does this refer to the bitness of the hypervisor (EL2)? 
Firmware (EL3)?

That said, I am not entirely sure I see the value of native because EL1 
(the kernel exception level) has to be known at boot. So it is better to 
explicitly mention whether the kernel is 32-bit or 64-bit.

>     XEN_ARM_SUBARCH_ARMV7 = 1,

This name would not be correct. You can have a 32-bit Armv8 guest. A 
better name is SUBARCH_AARCH32.

>     XEN_ARM_SUBARCH_AARCH64 = 2,

Cheers,

-- 
Julien Grall

Re: [XEN][PATCH 2/7] xen/domctl: introduce XEN_DOMCTL_CDF_is_32bits

[Grygorii Strashko]

On 31.07.25 18:19, Julien Grall wrote:
> Hi Teddy,
> 
> On 31/07/2025 15:02, Teddy Astie wrote:
>> Hello,
>>
>> Le 31/07/2025 à 11:44, Grygorii Strashko a écrit :
>>> Hence, this patch introduces extra "XEN_DOMCTL_CDF_is_32bits" flag which is
>>> intended to be used by 64bit Arches for proper configuration of domain type
>>> when domain is created.
>>>
>>> Now it adds initial support for this flag for Arm64 arch only. The default
>>> Arm64 domain type is changed to 64bit:
>>> - the Arm Xen boot code is handling this case properly already;
>>> - for toolstack case the XEN_DOMCTL_set_address_size hypercall handling
>>> updated to forcibly configure domain type regardless of current domain type
>>> configuration. Hence toolstack configures vcpus and memory after
>>> configuring domain type it allows to start with domain default AArch64 type
>>> and then switch to requested domain type and ensures all required domain
>>> settings applied.
>>>
>>> For Arm32 this flag is ignored.
>>>
>>> Note. For Arm64, Once toolstack is updated to probe guest binary before
>>> creating domain the XEN_DOMCTL_set_address_size will become obsolete.
>>>
>>> [1] https://lists.xen.org/archives/html/xen-devel/2025-07/msg01647.html
>>> [2] https://lists.xen.org/archives/html/xen-devel/2025-07/msg01648.html
>>>
>>> Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
>>> ---
>>
>> I would prefer it be better presented as a platform-specific enum.
>> Something like
>>
>> ```
>> enum xen_arch_subarch {
> 
> We need to be cautious when using enum in the public interface because, if I am not mistaken, the size of the enum if not stable.
> 
>>     XEN_ARM_SUBARCH_NATIVE = 0,
> 
> "native" is a bit ambiguous when a platform is able to support 32-bit and 64-bit. Does this refer to the bitness of the hypervisor (EL2)? Firmware (EL3)?
> 
> That said, I am not entirely sure I see the value of native because EL1 (the kernel exception level) has to be known at boot. So it is better to explicitly mention whether the kernel is 32-bit or 64-bit.
> 
>>     XEN_ARM_SUBARCH_ARMV7 = 1,
> 
> This name would not be correct. You can have a 32-bit Armv8 guest. A better name is SUBARCH_AARCH32.
> 
>>     XEN_ARM_SUBARCH_AARCH64 = 2,

I've been thinking how about using ELF "e_machine" to identify guest?

It defines all arch ids already (no need to invent anything custom),
plus supports EM_NONE which can be treated as default value or indicate that
guest type is not set - arch specific handling.


config/domain structs:
    uint16_t e_machine;

Arch code can define and check number of supported ids, for example:
  arm: EM_NONE, EM_ARM (32bit), EM_AARCH64 (64bit)
  x86: EM_NONE [, EM_386 (32bit), EM_X86_64 (64bit) ]



-- 
Best regards,
-grygorii