[PATCH v3 00/16] Hyperlaunch device tree for dom0

[PATCH v3 00/16] Hyperlaunch device tree for dom0

[Alejandro Vallejo]

Hi,

This is an updated version of the hyperlaunch DTB series, with pending
feedback addressed. Some parts (notably the way modules are located) has
changed a little and is more generic with less duplication.

I've purposefully not added my S-by on anything I haven't touched
(besides rebasing) as most of the feedback had already been addressed by
Jason by the time I looked at it and it would be utterly nonsensical to
give myself authorship over it.

I'll be looking after the series moving forward.

v3 pipeline: https://gitlab.com/xen-project/people/agvallejo/xen/-/pipelines/1758167851

v2: https://lore.kernel.org/xen-devel/20241226165740.29812-1-dpsmith@apertussolutions.com/
v1: https://lore.kernel.org/xen-devel/20241123182044.30687-1-dpsmith@apertussolutions.com/

Cheers,
Alejandro

========= Original cover letter:

The Hyperlaunch device tree for dom0 series is the second split out for the
introduction of the Hyperlaunch domain builder logic. These changes focus on
introducing the ability to express a domain configuration that is then used to
populate the struct boot_domain structure for dom0. This ability to express a
domain configuration provides the next step towards a general domain builder.

The splitting of Hyperlaunch into a set of series are twofold, to reduce the
effort in reviewing a much larger series, and to reduce the effort in handling
the knock-on effects to the construction logic from requested review changes.

=========

Alejandro Vallejo (1):
  x86/hyperlaunch: Add helpers to locate multiboot modules

Daniel P. Smith (15):
  x86/boot: introduce boot domain
  x86/boot: introduce domid field to struct boot_domain
  x86/boot: add cmdline to struct boot_domain
  kconfig: introduce option to independently enable libfdt
  kconfig: introduce domain builder config option
  x86/hyperlaunch: introduce the domain builder
  x86/hyperlaunch: initial support for hyperlaunch device tree
  x86/hyperlaunch: locate dom0 kernel with hyperlaunch
  x86/hyperlaunch: obtain cmdline from device tree
  x86/hyperlaunch: locate dom0 initrd with hyperlaunch
  x86/hyperlaunch: add domain id parsing to domain config
  x86/hyperlaunch: specify dom0 mode with device tree
  x86/hyperlaunch: add memory parsing to domain config
  x86/hyperlaunch: add max vcpu parsing of hyperlaunch device tree
  x86/hyperlaunch: add capabilities to boot domain

 xen/arch/x86/Kconfig                      |   2 +
 xen/arch/x86/Makefile                     |   1 +
 xen/arch/x86/dom0_build.c                 |  19 +-
 xen/arch/x86/domain-builder/Kconfig       |  15 +
 xen/arch/x86/domain-builder/Makefile      |   2 +
 xen/arch/x86/domain-builder/core.c        | 112 ++++++
 xen/arch/x86/domain-builder/fdt.c         | 448 ++++++++++++++++++++++
 xen/arch/x86/domain-builder/fdt.h         |  53 +++
 xen/arch/x86/hvm/dom0_build.c             |  35 +-
 xen/arch/x86/include/asm/boot-domain.h    |  48 +++
 xen/arch/x86/include/asm/bootinfo.h       |  15 +-
 xen/arch/x86/include/asm/dom0_build.h     |   6 +-
 xen/arch/x86/include/asm/domain-builder.h |  12 +
 xen/arch/x86/include/asm/setup.h          |   4 +-
 xen/arch/x86/pv/dom0_build.c              |  28 +-
 xen/arch/x86/setup.c                      | 106 +++--
 xen/common/Kconfig                        |   4 +
 xen/common/Makefile                       |   2 +-
 xen/include/xen/libfdt/libfdt-xen.h       | 101 +++++
 19 files changed, 927 insertions(+), 86 deletions(-)
 create mode 100644 xen/arch/x86/domain-builder/Kconfig
 create mode 100644 xen/arch/x86/domain-builder/Makefile
 create mode 100644 xen/arch/x86/domain-builder/core.c
 create mode 100644 xen/arch/x86/domain-builder/fdt.c
 create mode 100644 xen/arch/x86/domain-builder/fdt.h
 create mode 100644 xen/arch/x86/include/asm/boot-domain.h
 create mode 100644 xen/arch/x86/include/asm/domain-builder.h

-- 
2.43.0

[PATCH v3 01/16] x86/boot: introduce boot domain

[Alejandro Vallejo]

From: "Daniel P. Smith" <dpsmith@apertussolutions.com>

To begin moving toward allowing the hypervisor to construct more than one
domain at boot, a container is needed for a domain's build information.
Introduce a new header, <xen/asm/bootdomain.h>, that contains the initial
struct boot_domain that encapsulate the build information for a domain.

Add a kernel and ramdisk boot module reference along with a struct domain
reference to the new struct boot_domain. This allows a struct boot_domain
reference to be the only parameter necessary to pass down through the domain
construction call chain.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
---
v3:
  * Rename bootdomain.h -> boot-domain.h
  * const-ify boot_domain parameter passed to construct_dom0()
  * Rename boot_domain.ramdisk to boot_domain.module
---
 xen/arch/x86/dom0_build.c              |  8 +++++---
 xen/arch/x86/hvm/dom0_build.c          | 23 ++++++++-------------
 xen/arch/x86/include/asm/boot-domain.h | 28 ++++++++++++++++++++++++++
 xen/arch/x86/include/asm/bootinfo.h    |  5 +++++
 xen/arch/x86/include/asm/dom0_build.h  |  6 +++---
 xen/arch/x86/include/asm/setup.h       |  4 ++--
 xen/arch/x86/pv/dom0_build.c           | 24 ++++++++--------------
 xen/arch/x86/setup.c                   | 24 +++++++++-------------
 8 files changed, 69 insertions(+), 53 deletions(-)
 create mode 100644 xen/arch/x86/include/asm/boot-domain.h

diff --git a/xen/arch/x86/dom0_build.c b/xen/arch/x86/dom0_build.c
index 8191d90a22..0b467fd4a4 100644
--- a/xen/arch/x86/dom0_build.c
+++ b/xen/arch/x86/dom0_build.c
@@ -13,6 +13,7 @@
 #include <xen/softirq.h>
 
 #include <asm/amd.h>
+#include <asm/bootinfo.h>
 #include <asm/dom0_build.h>
 #include <asm/guest.h>
 #include <asm/hpet.h>
@@ -614,9 +615,10 @@ int __init dom0_setup_permissions(struct domain *d)
     return rc;
 }
 
-int __init construct_dom0(struct boot_info *bi, struct domain *d)
+int __init construct_dom0(const struct boot_domain *bd)
 {
     int rc;
+    const struct domain *d = bd->d;
 
     /* Sanity! */
     BUG_ON(!pv_shim && d->domain_id != 0);
@@ -626,9 +628,9 @@ int __init construct_dom0(struct boot_info *bi, struct domain *d)
     process_pending_softirqs();
 
     if ( is_hvm_domain(d) )
-        rc = dom0_construct_pvh(bi, d);
+        rc = dom0_construct_pvh(bd);
     else if ( is_pv_domain(d) )
-        rc = dom0_construct_pv(bi, d);
+        rc = dom0_construct_pv(bd);
     else
         panic("Cannot construct Dom0. No guest interface available\n");
 
diff --git a/xen/arch/x86/hvm/dom0_build.c b/xen/arch/x86/hvm/dom0_build.c
index 9fd68df7b9..2a094b3145 100644
--- a/xen/arch/x86/hvm/dom0_build.c
+++ b/xen/arch/x86/hvm/dom0_build.c
@@ -644,9 +644,11 @@ static bool __init check_and_adjust_load_address(
 }
 
 static int __init pvh_load_kernel(
-    struct domain *d, struct boot_module *image, struct boot_module *initrd,
-    paddr_t *entry, paddr_t *start_info_addr)
+    const struct boot_domain *bd, paddr_t *entry, paddr_t *start_info_addr)
 {
+    struct domain *d = bd->d;
+    struct boot_module *image = bd->kernel;
+    struct boot_module *initrd = bd->module;
     void *image_base = bootstrap_map_bm(image);
     void *image_start = image_base + image->headroom;
     unsigned long image_len = image->size;
@@ -1328,26 +1330,17 @@ static void __hwdom_init pvh_setup_mmcfg(struct domain *d)
     }
 }
 
-int __init dom0_construct_pvh(struct boot_info *bi, struct domain *d)
+int __init dom0_construct_pvh(const struct boot_domain *bd)
 {
     paddr_t entry, start_info;
-    struct boot_module *image;
-    struct boot_module *initrd = NULL;
-    unsigned int idx;
+    struct domain *d = bd->d;
     int rc;
 
     printk(XENLOG_INFO "*** Building a PVH Dom%d ***\n", d->domain_id);
 
-    idx = first_boot_module_index(bi, BOOTMOD_KERNEL);
-    if ( idx >= bi->nr_modules )
+    if ( bd->kernel == NULL )
         panic("Missing kernel boot module for %pd construction\n", d);
 
-    image = &bi->mods[idx];
-
-    idx = first_boot_module_index(bi, BOOTMOD_RAMDISK);
-    if ( idx < bi->nr_modules )
-        initrd = &bi->mods[idx];
-
     if ( is_hardware_domain(d) )
     {
         /*
@@ -1385,7 +1378,7 @@ int __init dom0_construct_pvh(struct boot_info *bi, struct domain *d)
         return rc;
     }
 
-    rc = pvh_load_kernel(d, image, initrd, &entry, &start_info);
+    rc = pvh_load_kernel(bd, &entry, &start_info);
     if ( rc )
     {
         printk("Failed to load Dom0 kernel\n");
diff --git a/xen/arch/x86/include/asm/boot-domain.h b/xen/arch/x86/include/asm/boot-domain.h
new file mode 100644
index 0000000000..5e1e1a0b61
--- /dev/null
+++ b/xen/arch/x86/include/asm/boot-domain.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2024 Apertus Solutions, LLC
+ * Author: Daniel P. Smith <dpsmith@apertussolutions.com>
+ * Copyright (c) 2024 Christopher Clark <christopher.w.clark@gmail.com>
+ */
+
+#ifndef __XEN_X86_BOOTDOMAIN_H__
+#define __XEN_X86_BOOTDOMAIN_H__
+
+struct boot_domain {
+    struct boot_module *kernel;
+    struct boot_module *module;
+
+    struct domain *d;
+};
+
+#endif
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/x86/include/asm/bootinfo.h b/xen/arch/x86/include/asm/bootinfo.h
index f8b4229130..3afc214c17 100644
--- a/xen/arch/x86/include/asm/bootinfo.h
+++ b/xen/arch/x86/include/asm/bootinfo.h
@@ -11,10 +11,14 @@
 #include <xen/init.h>
 #include <xen/multiboot.h>
 #include <xen/types.h>
+#include <asm/boot-domain.h>
 
 /* Max number of boot modules a bootloader can provide in addition to Xen */
 #define MAX_NR_BOOTMODS 63
 
+/* Max number of boot domains that Xen can construct */
+#define MAX_NR_BOOTDOMS 1
+
 /* Boot module binary type / purpose */
 enum bootmod_type {
     BOOTMOD_UNKNOWN,
@@ -78,6 +82,7 @@ struct boot_info {
 
     unsigned int nr_modules;
     struct boot_module mods[MAX_NR_BOOTMODS + 1];
+    struct boot_domain domains[MAX_NR_BOOTDOMS];
 };
 
 /*
diff --git a/xen/arch/x86/include/asm/dom0_build.h b/xen/arch/x86/include/asm/dom0_build.h
index 2d67b17213..ff021c24af 100644
--- a/xen/arch/x86/include/asm/dom0_build.h
+++ b/xen/arch/x86/include/asm/dom0_build.h
@@ -13,9 +13,9 @@ unsigned long dom0_compute_nr_pages(struct domain *d,
                                     unsigned long initrd_len);
 int dom0_setup_permissions(struct domain *d);
 
-struct boot_info;
-int dom0_construct_pv(struct boot_info *bi, struct domain *d);
-int dom0_construct_pvh(struct boot_info *bi, struct domain *d);
+struct boot_domain;
+int dom0_construct_pv(const struct boot_domain *bd);
+int dom0_construct_pvh(const struct boot_domain *bd);
 
 unsigned long dom0_paging_pages(const struct domain *d,
                                 unsigned long nr_pages);
diff --git a/xen/arch/x86/include/asm/setup.h b/xen/arch/x86/include/asm/setup.h
index 5c2391a868..ac34c69855 100644
--- a/xen/arch/x86/include/asm/setup.h
+++ b/xen/arch/x86/include/asm/setup.h
@@ -26,8 +26,8 @@ void subarch_init_memory(void);
 
 void init_IRQ(void);
 
-struct boot_info;
-int construct_dom0(struct boot_info *bi, struct domain *d);
+struct boot_domain;
+int construct_dom0(const struct boot_domain *bd);
 
 void setup_io_bitmap(struct domain *d);
 
diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c
index 96e28c7b6a..b485eea05f 100644
--- a/xen/arch/x86/pv/dom0_build.c
+++ b/xen/arch/x86/pv/dom0_build.c
@@ -355,7 +355,7 @@ static struct page_info * __init alloc_chunk(struct domain *d,
     return page;
 }
 
-static int __init dom0_construct(struct boot_info *bi, struct domain *d)
+static int __init dom0_construct(const struct boot_domain *bd)
 {
     unsigned int i;
     int rc, order, machine;
@@ -371,14 +371,15 @@ static int __init dom0_construct(struct boot_info *bi, struct domain *d)
     struct page_info *page = NULL;
     unsigned int flush_flags = 0;
     start_info_t *si;
+    struct domain *d = bd->d;
     struct vcpu *v = d->vcpu[0];
 
-    struct boot_module *image;
-    struct boot_module *initrd = NULL;
+    struct boot_module *image = bd->kernel;
+    struct boot_module *initrd = bd->module;
     void *image_base;
     unsigned long image_len;
     void *image_start;
-    unsigned long initrd_len = 0;
+    unsigned long initrd_len = initrd ? initrd->size : 0;
 
     l4_pgentry_t *l4tab = NULL, *l4start = NULL;
     l3_pgentry_t *l3tab = NULL, *l3start = NULL;
@@ -416,22 +417,13 @@ static int __init dom0_construct(struct boot_info *bi, struct domain *d)
 
     printk(XENLOG_INFO "*** Building a PV Dom%d ***\n", d->domain_id);
 
-    i = first_boot_module_index(bi, BOOTMOD_KERNEL);
-    if ( i >= bi->nr_modules )
+    if ( !image )
         panic("Missing kernel boot module for %pd construction\n", d);
 
-    image = &bi->mods[i];
     image_base = bootstrap_map_bm(image);
     image_len = image->size;
     image_start = image_base + image->headroom;
 
-    i = first_boot_module_index(bi, BOOTMOD_RAMDISK);
-    if ( i < bi->nr_modules )
-    {
-        initrd = &bi->mods[i];
-        initrd_len = initrd->size;
-    }
-
     d->max_pages = ~0U;
 
     if ( (rc = bzimage_parse(image_base, &image_start, &image_len)) != 0 )
@@ -1078,7 +1070,7 @@ out:
     return rc;
 }
 
-int __init dom0_construct_pv(struct boot_info *bi, struct domain *d)
+int __init dom0_construct_pv(const struct boot_domain *bd)
 {
     unsigned long cr4 = read_cr4();
     int rc;
@@ -1096,7 +1088,7 @@ int __init dom0_construct_pv(struct boot_info *bi, struct domain *d)
         write_cr4(cr4 & ~X86_CR4_SMAP);
     }
 
-    rc = dom0_construct(bi, d);
+    rc = dom0_construct(bd);
 
     if ( cr4 & X86_CR4_SMAP )
     {
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index d70abb7e0c..ddb10ea03d 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -992,16 +992,9 @@ static struct domain *__init create_dom0(struct boot_info *bi)
             .misc_flags = opt_dom0_msr_relaxed ? XEN_X86_MSR_RELAXED : 0,
         },
     };
+    struct boot_domain *bd = &bi->domains[0];
     struct domain *d;
     domid_t domid;
-    struct boot_module *image;
-    unsigned int idx;
-
-    idx = first_boot_module_index(bi, BOOTMOD_KERNEL);
-    if ( idx >= bi->nr_modules )
-        panic("Missing kernel boot module for building domain\n");
-
-    image = &bi->mods[idx];
 
     if ( opt_dom0_pvh )
     {
@@ -1028,11 +1021,11 @@ static struct domain *__init create_dom0(struct boot_info *bi)
         panic("Error creating d%uv0\n", domid);
 
     /* Grab the DOM0 command line. */
-    if ( image->cmdline_pa || bi->kextra )
+    if ( bd->kernel->cmdline_pa || bi->kextra )
     {
-        if ( image->cmdline_pa )
-            safe_strcpy(
-                cmdline, cmdline_cook(__va(image->cmdline_pa), bi->loader));
+        if ( bd->kernel->cmdline_pa )
+            safe_strcpy(cmdline,
+                        cmdline_cook(__va(bd->kernel->cmdline_pa), bi->loader));
 
         if ( bi->kextra )
             /* kextra always includes exactly one leading space. */
@@ -1054,10 +1047,11 @@ static struct domain *__init create_dom0(struct boot_info *bi)
             safe_strcat(cmdline, acpi_param);
         }
 
-        image->cmdline_pa = __pa(cmdline);
+        bd->kernel->cmdline_pa = __pa(cmdline);
     }
 
-    if ( construct_dom0(bi, d) != 0 )
+    bd->d = d;
+    if ( construct_dom0(bd) != 0 )
         panic("Could not construct domain 0\n");
 
     return d;
@@ -1263,6 +1257,7 @@ void asmlinkage __init noreturn __start_xen(void)
 
     /* Dom0 kernel is always first */
     bi->mods[0].type = BOOTMOD_KERNEL;
+    bi->domains[0].kernel = &bi->mods[0];
 
     if ( pvh_boot )
     {
@@ -2129,6 +2124,7 @@ void asmlinkage __init noreturn __start_xen(void)
     if ( initrdidx < MAX_NR_BOOTMODS )
     {
         bi->mods[initrdidx].type = BOOTMOD_RAMDISK;
+        bi->domains[0].module = &bi->mods[initrdidx];
         if ( first_boot_module_index(bi, BOOTMOD_UNKNOWN) < MAX_NR_BOOTMODS )
             printk(XENLOG_WARNING
                    "Multiple initrd candidates, picking module #%u\n",
-- 
2.43.0

Re: [PATCH v3 01/16] x86/boot: introduce boot domain

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
> 
> To begin moving toward allowing the hypervisor to construct more than one
> domain at boot, a container is needed for a domain's build information.
> Introduce a new header, <xen/asm/bootdomain.h>, that contains the initial
> struct boot_domain that encapsulate the build information for a domain.
> 
> Add a kernel and ramdisk boot module reference along with a struct domain
> reference to the new struct boot_domain. This allows a struct boot_domain
> reference to be the only parameter necessary to pass down through the domain
> construction call chain.
> 
> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

> --- /dev/null
> +++ b/xen/arch/x86/include/asm/boot-domain.h
> @@ -0,0 +1,28 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +/*
> + * Copyright (c) 2024 Apertus Solutions, LLC
> + * Author: Daniel P. Smith <dpsmith@apertussolutions.com>
> + * Copyright (c) 2024 Christopher Clark <christopher.w.clark@gmail.com>
> + */

I wonder if the 2024-s here shouldn't have been amended by 2025 now. (I'm
also, more generally, unclear about the purpose of such, especially when
the file is really small and simple. It's only going to go stale, as we
can see from various other files having such copyright lines.)

Jan

Re: [PATCH v3 01/16] x86/boot: introduce boot domain

[Alejandro Vallejo]

On Wed Apr 9, 2025 at 7:24 AM BST, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>> 
>> To begin moving toward allowing the hypervisor to construct more than one
>> domain at boot, a container is needed for a domain's build information.
>> Introduce a new header, <xen/asm/bootdomain.h>, that contains the initial
>> struct boot_domain that encapsulate the build information for a domain.
>> 
>> Add a kernel and ramdisk boot module reference along with a struct domain
>> reference to the new struct boot_domain. This allows a struct boot_domain
>> reference to be the only parameter necessary to pass down through the domain
>> construction call chain.
>> 
>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
>
> Acked-by: Jan Beulich <jbeulich@suse.com>

Thanks

>
>> --- /dev/null
>> +++ b/xen/arch/x86/include/asm/boot-domain.h
>> @@ -0,0 +1,28 @@
>> +/* SPDX-License-Identifier: GPL-2.0-or-later */
>> +/*
>> + * Copyright (c) 2024 Apertus Solutions, LLC
>> + * Author: Daniel P. Smith <dpsmith@apertussolutions.com>
>> + * Copyright (c) 2024 Christopher Clark <christopher.w.clark@gmail.com>
>> + */
>
> I wonder if the 2024-s here shouldn't have been amended by 2025 now.

Maybe. I didn't think about it, tbh. One could argue that Daniel and
Christopher's original contribution was indeed in 2024 and the fact they
weren't committed until (hopefully!) 2025 doesn't remove the fact they
did exist in some form in 2024.

I don't particularly care either way, but tend to lean on the "it's fine
how it is".

Cheers,
Alejandro

Re: [PATCH v3 01/16] x86/boot: introduce boot domain

[Daniel P. Smith]

On 4/9/25 06:28, Alejandro Vallejo wrote:
> On Wed Apr 9, 2025 at 7:24 AM BST, Jan Beulich wrote:
>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>>
>>> To begin moving toward allowing the hypervisor to construct more than one
>>> domain at boot, a container is needed for a domain's build information.
>>> Introduce a new header, <xen/asm/bootdomain.h>, that contains the initial
>>> struct boot_domain that encapsulate the build information for a domain.
>>>
>>> Add a kernel and ramdisk boot module reference along with a struct domain
>>> reference to the new struct boot_domain. This allows a struct boot_domain
>>> reference to be the only parameter necessary to pass down through the domain
>>> construction call chain.
>>>
>>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>>> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
>>
>> Acked-by: Jan Beulich <jbeulich@suse.com>
> 
> Thanks
> 
>>
>>> --- /dev/null
>>> +++ b/xen/arch/x86/include/asm/boot-domain.h
>>> @@ -0,0 +1,28 @@
>>> +/* SPDX-License-Identifier: GPL-2.0-or-later */
>>> +/*
>>> + * Copyright (c) 2024 Apertus Solutions, LLC
>>> + * Author: Daniel P. Smith <dpsmith@apertussolutions.com>
>>> + * Copyright (c) 2024 Christopher Clark <christopher.w.clark@gmail.com>
>>> + */
>>
>> I wonder if the 2024-s here shouldn't have been amended by 2025 now.
> 
> Maybe. I didn't think about it, tbh. One could argue that Daniel and
> Christopher's original contribution was indeed in 2024 and the fact they
> weren't committed until (hopefully!) 2025 doesn't remove the fact they
> did exist in some form in 2024.
> 
> I don't particularly care either way, but tend to lean on the "it's fine
> how it is".

Jan is correct, now that we have moved into 2025, they should be 
updated. Maybe we can actually get it all committed this year.

v/r,
dps

Re: [PATCH v3 01/16] x86/boot: introduce boot domain

[Jason Andryuk]

On 2025-04-10 09:13, Daniel P. Smith wrote:
> On 4/9/25 06:28, Alejandro Vallejo wrote:
>> On Wed Apr 9, 2025 at 7:24 AM BST, Jan Beulich wrote:
>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>> --- /dev/null
>>>> +++ b/xen/arch/x86/include/asm/boot-domain.h
>>>> @@ -0,0 +1,28 @@
>>>> +/* SPDX-License-Identifier: GPL-2.0-or-later */
>>>> +/*
>>>> + * Copyright (c) 2024 Apertus Solutions, LLC
>>>> + * Author: Daniel P. Smith <dpsmith@apertussolutions.com>
>>>> + * Copyright (c) 2024 Christopher Clark 
>>>> <christopher.w.clark@gmail.com>
>>>> + */
>>>
>>> I wonder if the 2024-s here shouldn't have been amended by 2025 now.
>>
>> Maybe. I didn't think about it, tbh. One could argue that Daniel and
>> Christopher's original contribution was indeed in 2024 and the fact they
>> weren't committed until (hopefully!) 2025 doesn't remove the fact they
>> did exist in some form in 2024.
>>
>> I don't particularly care either way, but tend to lean on the "it's fine
>> how it is".
> 
> Jan is correct, now that we have moved into 2025, they should be 
> updated. Maybe we can actually get it all committed this year.

These were written in 2024.  Neither Christopher nor Dan touched them in 
2025, so why would the date change?

Regards,
Jason

Re: [PATCH v3 01/16] x86/boot: introduce boot domain

[Daniel P. Smith]

On 4/9/25 02:24, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>
>> To begin moving toward allowing the hypervisor to construct more than one
>> domain at boot, a container is needed for a domain's build information.
>> Introduce a new header, <xen/asm/bootdomain.h>, that contains the initial
>> struct boot_domain that encapsulate the build information for a domain.
>>
>> Add a kernel and ramdisk boot module reference along with a struct domain
>> reference to the new struct boot_domain. This allows a struct boot_domain
>> reference to be the only parameter necessary to pass down through the domain
>> construction call chain.
>>
>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
> 
> Acked-by: Jan Beulich <jbeulich@suse.com>

I have to object because the meaningless rename is going cause 
significant pain in the rebase of the follow-on series for no improved 
code clarity.

V/r,
DPS

Re: [PATCH v3 01/16] x86/boot: introduce boot domain

[Jan Beulich]

On 10.04.2025 15:09, Daniel P. Smith wrote:
> On 4/9/25 02:24, Jan Beulich wrote:
>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>>
>>> To begin moving toward allowing the hypervisor to construct more than one
>>> domain at boot, a container is needed for a domain's build information.
>>> Introduce a new header, <xen/asm/bootdomain.h>, that contains the initial
>>> struct boot_domain that encapsulate the build information for a domain.
>>>
>>> Add a kernel and ramdisk boot module reference along with a struct domain
>>> reference to the new struct boot_domain. This allows a struct boot_domain
>>> reference to be the only parameter necessary to pass down through the domain
>>> construction call chain.
>>>
>>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>>> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
>>
>> Acked-by: Jan Beulich <jbeulich@suse.com>
> 
> I have to object because the meaningless rename is going cause 
> significant pain in the rebase of the follow-on series for no improved 
> code clarity.

Sorry, then an incremental patch undoing the rename that happened (with
appropriate justification) will need proposing - the patch here has gone
in already.

Jan

Re: [PATCH v3 01/16] x86/boot: introduce boot domain

[Jason Andryuk]

On 2025-04-10 11:01, Jan Beulich wrote:
> On 10.04.2025 15:09, Daniel P. Smith wrote:
>> On 4/9/25 02:24, Jan Beulich wrote:
>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>>>
>>>> To begin moving toward allowing the hypervisor to construct more than one
>>>> domain at boot, a container is needed for a domain's build information.
>>>> Introduce a new header, <xen/asm/bootdomain.h>, that contains the initial
>>>> struct boot_domain that encapsulate the build information for a domain.
>>>>
>>>> Add a kernel and ramdisk boot module reference along with a struct domain
>>>> reference to the new struct boot_domain. This allows a struct boot_domain
>>>> reference to be the only parameter necessary to pass down through the domain
>>>> construction call chain.
>>>>
>>>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>>>> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
>>>
>>> Acked-by: Jan Beulich <jbeulich@suse.com>
>>
>> I have to object because the meaningless rename is going cause
>> significant pain in the rebase of the follow-on series for no improved
>> code clarity.
> 
> Sorry, then an incremental patch undoing the rename that happened (with
> appropriate justification) will need proposing - the patch here has gone
> in already.

Coming from a Linux background, ramdisk seemed more natural to me.  But 
looking at hvm_start_info, the fields are called module there.  And 
since we shouldn't tie this to the Linux naming, the more generic 
"module" name seemed fine to me.

Regards,
Jason

Re: [PATCH v3 01/16] x86/boot: introduce boot domain

[Daniel P. Smith]

On 4/10/25 16:56, Jason Andryuk wrote:
> On 2025-04-10 11:01, Jan Beulich wrote:
>> On 10.04.2025 15:09, Daniel P. Smith wrote:
>>> On 4/9/25 02:24, Jan Beulich wrote:
>>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>>>>
>>>>> To begin moving toward allowing the hypervisor to construct more 
>>>>> than one
>>>>> domain at boot, a container is needed for a domain's build 
>>>>> information.
>>>>> Introduce a new header, <xen/asm/bootdomain.h>, that contains the 
>>>>> initial
>>>>> struct boot_domain that encapsulate the build information for a 
>>>>> domain.
>>>>>
>>>>> Add a kernel and ramdisk boot module reference along with a struct 
>>>>> domain
>>>>> reference to the new struct boot_domain. This allows a struct 
>>>>> boot_domain
>>>>> reference to be the only parameter necessary to pass down through 
>>>>> the domain
>>>>> construction call chain.
>>>>>
>>>>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>>>>> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
>>>>
>>>> Acked-by: Jan Beulich <jbeulich@suse.com>
>>>
>>> I have to object because the meaningless rename is going cause
>>> significant pain in the rebase of the follow-on series for no improved
>>> code clarity.
>>
>> Sorry, then an incremental patch undoing the rename that happened (with
>> appropriate justification) will need proposing - the patch here has gone
>> in already.
> 
> Coming from a Linux background, ramdisk seemed more natural to me.  But 
> looking at hvm_start_info, the fields are called module there.  And 
> since we shouldn't tie this to the Linux naming, the more generic 
> "module" name seemed fine to me.

Again, as I have stated, ramdisk is not a Linux only concept. In fact, 
as Jan points out, initrd/initramfs are Linux specific implementations 
of a ramdisk for which Xen doesn't even fully support. I am inclined to 
ask the inverse of why hvm_start_info uses the name module. But that 
aside, let's consider the fact that the field is only populated by the 
device tree when a module type of BOOTMOD_RAMDISK is matched. And all 
the uses of the field are when its value is stored into a local variable 
called initrd.

Though the biggest irony is that generally obtuse abstraction are 
routinely blocked unless there is a tangible future case. Yet none was 
offered in the comment. Thus on that principle alone, a request for a 
tangible future use should have been requested and provided for the 
change to be considered.

V/r,
DPS

Re: [PATCH v3 01/16] x86/boot: introduce boot domain

[Jan Beulich]

On 16.04.2025 15:02, Daniel P. Smith wrote:
> On 4/10/25 16:56, Jason Andryuk wrote:
>> On 2025-04-10 11:01, Jan Beulich wrote:
>>> On 10.04.2025 15:09, Daniel P. Smith wrote:
>>>> On 4/9/25 02:24, Jan Beulich wrote:
>>>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>>>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>>>>>
>>>>>> To begin moving toward allowing the hypervisor to construct more 
>>>>>> than one
>>>>>> domain at boot, a container is needed for a domain's build 
>>>>>> information.
>>>>>> Introduce a new header, <xen/asm/bootdomain.h>, that contains the 
>>>>>> initial
>>>>>> struct boot_domain that encapsulate the build information for a 
>>>>>> domain.
>>>>>>
>>>>>> Add a kernel and ramdisk boot module reference along with a struct 
>>>>>> domain
>>>>>> reference to the new struct boot_domain. This allows a struct 
>>>>>> boot_domain
>>>>>> reference to be the only parameter necessary to pass down through 
>>>>>> the domain
>>>>>> construction call chain.
>>>>>>
>>>>>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>>>>>> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
>>>>>
>>>>> Acked-by: Jan Beulich <jbeulich@suse.com>
>>>>
>>>> I have to object because the meaningless rename is going cause
>>>> significant pain in the rebase of the follow-on series for no improved
>>>> code clarity.
>>>
>>> Sorry, then an incremental patch undoing the rename that happened (with
>>> appropriate justification) will need proposing - the patch here has gone
>>> in already.
>>
>> Coming from a Linux background, ramdisk seemed more natural to me.  But 
>> looking at hvm_start_info, the fields are called module there.  And 
>> since we shouldn't tie this to the Linux naming, the more generic 
>> "module" name seemed fine to me.
> 
> Again, as I have stated, ramdisk is not a Linux only concept. In fact, 
> as Jan points out, initrd/initramfs are Linux specific implementations 
> of a ramdisk for which Xen doesn't even fully support. I am inclined to 
> ask the inverse of why hvm_start_info uses the name module. But that 
> aside, let's consider the fact that the field is only populated by the 
> device tree when a module type of BOOTMOD_RAMDISK is matched. And all 
> the uses of the field are when its value is stored into a local variable 
> called initrd.
> 
> Though the biggest irony is that generally obtuse abstraction are 
> routinely blocked unless there is a tangible future case. Yet none was 
> offered in the comment. Thus on that principle alone, a request for a 
> tangible future use should have been requested and provided for the 
> change to be considered.

Does it even need to be a _future_ use here? Aren't you working on
abstracting domain creation, suitable (in principle) for all architectures?
Isn't therefore a more generic name (as "module" is) preferable over a more
specific one?

Jan

Re: [PATCH v3 01/16] x86/boot: introduce boot domain

[Daniel P. Smith]

V/r,
Daniel P. Smith
Apertus Solutions, LLC

On 4/16/25 09:33, Jan Beulich wrote:
> On 16.04.2025 15:02, Daniel P. Smith wrote:
>> On 4/10/25 16:56, Jason Andryuk wrote:
>>> On 2025-04-10 11:01, Jan Beulich wrote:
>>>> On 10.04.2025 15:09, Daniel P. Smith wrote:
>>>>> On 4/9/25 02:24, Jan Beulich wrote:
>>>>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>>>>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>>>>>>
>>>>>>> To begin moving toward allowing the hypervisor to construct more
>>>>>>> than one
>>>>>>> domain at boot, a container is needed for a domain's build
>>>>>>> information.
>>>>>>> Introduce a new header, <xen/asm/bootdomain.h>, that contains the
>>>>>>> initial
>>>>>>> struct boot_domain that encapsulate the build information for a
>>>>>>> domain.
>>>>>>>
>>>>>>> Add a kernel and ramdisk boot module reference along with a struct
>>>>>>> domain
>>>>>>> reference to the new struct boot_domain. This allows a struct
>>>>>>> boot_domain
>>>>>>> reference to be the only parameter necessary to pass down through
>>>>>>> the domain
>>>>>>> construction call chain.
>>>>>>>
>>>>>>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>>>>>>> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
>>>>>>
>>>>>> Acked-by: Jan Beulich <jbeulich@suse.com>
>>>>>
>>>>> I have to object because the meaningless rename is going cause
>>>>> significant pain in the rebase of the follow-on series for no improved
>>>>> code clarity.
>>>>
>>>> Sorry, then an incremental patch undoing the rename that happened (with
>>>> appropriate justification) will need proposing - the patch here has gone
>>>> in already.
>>>
>>> Coming from a Linux background, ramdisk seemed more natural to me.  But
>>> looking at hvm_start_info, the fields are called module there.  And
>>> since we shouldn't tie this to the Linux naming, the more generic
>>> "module" name seemed fine to me.
>>
>> Again, as I have stated, ramdisk is not a Linux only concept. In fact,
>> as Jan points out, initrd/initramfs are Linux specific implementations
>> of a ramdisk for which Xen doesn't even fully support. I am inclined to
>> ask the inverse of why hvm_start_info uses the name module. But that
>> aside, let's consider the fact that the field is only populated by the
>> device tree when a module type of BOOTMOD_RAMDISK is matched. And all
>> the uses of the field are when its value is stored into a local variable
>> called initrd.
>>
>> Though the biggest irony is that generally obtuse abstraction are
>> routinely blocked unless there is a tangible future case. Yet none was
>> offered in the comment. Thus on that principle alone, a request for a
>> tangible future use should have been requested and provided for the
>> change to be considered.
> 
> Does it even need to be a _future_ use here? Aren't you working on
> abstracting domain creation, suitable (in principle) for all architectures?
> Isn't therefore a more generic name (as "module" is) preferable over a more
> specific one?

Yes we are trying to build a future capability, but my point is let's 
consider all possible known OS's start up today. What other boot module 
could potentially be passed in that is exclusive of a ramdisk, thus 
allowing a multiplex of the field. And the answer is none. The other 
potential modules that could be passed in will need to be able to be 
coexist with a ramdisk module being passed. The immediate examples I can 
point to are, an SELinux policy file or a guest device tree. I'm not too 
familiar, perhaps a Zephyr guest may only take a guest DT, but a Linux 
guest may take an initrd and a DT.

v/r,
dps

Re: [PATCH v3 01/16] x86/boot: introduce boot domain

[Jan Beulich]

On 16.04.2025 16:00, Daniel P. Smith wrote:
> 
> 
> V/r,
> Daniel P. Smith
> Apertus Solutions, LLC
> 
> On 4/16/25 09:33, Jan Beulich wrote:
>> On 16.04.2025 15:02, Daniel P. Smith wrote:
>>> On 4/10/25 16:56, Jason Andryuk wrote:
>>>> On 2025-04-10 11:01, Jan Beulich wrote:
>>>>> On 10.04.2025 15:09, Daniel P. Smith wrote:
>>>>>> On 4/9/25 02:24, Jan Beulich wrote:
>>>>>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>>>>>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>>>>>>>
>>>>>>>> To begin moving toward allowing the hypervisor to construct more
>>>>>>>> than one
>>>>>>>> domain at boot, a container is needed for a domain's build
>>>>>>>> information.
>>>>>>>> Introduce a new header, <xen/asm/bootdomain.h>, that contains the
>>>>>>>> initial
>>>>>>>> struct boot_domain that encapsulate the build information for a
>>>>>>>> domain.
>>>>>>>>
>>>>>>>> Add a kernel and ramdisk boot module reference along with a struct
>>>>>>>> domain
>>>>>>>> reference to the new struct boot_domain. This allows a struct
>>>>>>>> boot_domain
>>>>>>>> reference to be the only parameter necessary to pass down through
>>>>>>>> the domain
>>>>>>>> construction call chain.
>>>>>>>>
>>>>>>>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>>>>>>>> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
>>>>>>>
>>>>>>> Acked-by: Jan Beulich <jbeulich@suse.com>
>>>>>>
>>>>>> I have to object because the meaningless rename is going cause
>>>>>> significant pain in the rebase of the follow-on series for no improved
>>>>>> code clarity.
>>>>>
>>>>> Sorry, then an incremental patch undoing the rename that happened (with
>>>>> appropriate justification) will need proposing - the patch here has gone
>>>>> in already.
>>>>
>>>> Coming from a Linux background, ramdisk seemed more natural to me.  But
>>>> looking at hvm_start_info, the fields are called module there.  And
>>>> since we shouldn't tie this to the Linux naming, the more generic
>>>> "module" name seemed fine to me.
>>>
>>> Again, as I have stated, ramdisk is not a Linux only concept. In fact,
>>> as Jan points out, initrd/initramfs are Linux specific implementations
>>> of a ramdisk for which Xen doesn't even fully support. I am inclined to
>>> ask the inverse of why hvm_start_info uses the name module. But that
>>> aside, let's consider the fact that the field is only populated by the
>>> device tree when a module type of BOOTMOD_RAMDISK is matched. And all
>>> the uses of the field are when its value is stored into a local variable
>>> called initrd.
>>>
>>> Though the biggest irony is that generally obtuse abstraction are
>>> routinely blocked unless there is a tangible future case. Yet none was
>>> offered in the comment. Thus on that principle alone, a request for a
>>> tangible future use should have been requested and provided for the
>>> change to be considered.
>>
>> Does it even need to be a _future_ use here? Aren't you working on
>> abstracting domain creation, suitable (in principle) for all architectures?
>> Isn't therefore a more generic name (as "module" is) preferable over a more
>> specific one?
> 
> Yes we are trying to build a future capability, but my point is let's 
> consider all possible known OS's start up today. What other boot module 
> could potentially be passed in that is exclusive of a ramdisk, thus 
> allowing a multiplex of the field. And the answer is none.

Is it? What if you are to start a nested Xen with its own kernel, initrd
and perhaps even an XSM policy "module"? Or anything else that is multi-
module capable (possibly but not necessarily because of having started
out as multiboot)?

Jan

Re: [PATCH v3 01/16] x86/boot: introduce boot domain

[Daniel P. Smith]

V/r,
Daniel P. Smith
Apertus Solutions, LLC

On 4/16/25 10:06, Jan Beulich wrote:
> On 16.04.2025 16:00, Daniel P. Smith wrote:
>>
>>
>> V/r,
>> Daniel P. Smith
>> Apertus Solutions, LLC
>>
>> On 4/16/25 09:33, Jan Beulich wrote:
>>> On 16.04.2025 15:02, Daniel P. Smith wrote:
>>>> On 4/10/25 16:56, Jason Andryuk wrote:
>>>>> On 2025-04-10 11:01, Jan Beulich wrote:
>>>>>> On 10.04.2025 15:09, Daniel P. Smith wrote:
>>>>>>> On 4/9/25 02:24, Jan Beulich wrote:
>>>>>>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>>>>>>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>>>>>>>>
>>>>>>>>> To begin moving toward allowing the hypervisor to construct more
>>>>>>>>> than one
>>>>>>>>> domain at boot, a container is needed for a domain's build
>>>>>>>>> information.
>>>>>>>>> Introduce a new header, <xen/asm/bootdomain.h>, that contains the
>>>>>>>>> initial
>>>>>>>>> struct boot_domain that encapsulate the build information for a
>>>>>>>>> domain.
>>>>>>>>>
>>>>>>>>> Add a kernel and ramdisk boot module reference along with a struct
>>>>>>>>> domain
>>>>>>>>> reference to the new struct boot_domain. This allows a struct
>>>>>>>>> boot_domain
>>>>>>>>> reference to be the only parameter necessary to pass down through
>>>>>>>>> the domain
>>>>>>>>> construction call chain.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>>>>>>>>> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
>>>>>>>>
>>>>>>>> Acked-by: Jan Beulich <jbeulich@suse.com>
>>>>>>>
>>>>>>> I have to object because the meaningless rename is going cause
>>>>>>> significant pain in the rebase of the follow-on series for no improved
>>>>>>> code clarity.
>>>>>>
>>>>>> Sorry, then an incremental patch undoing the rename that happened (with
>>>>>> appropriate justification) will need proposing - the patch here has gone
>>>>>> in already.
>>>>>
>>>>> Coming from a Linux background, ramdisk seemed more natural to me.  But
>>>>> looking at hvm_start_info, the fields are called module there.  And
>>>>> since we shouldn't tie this to the Linux naming, the more generic
>>>>> "module" name seemed fine to me.
>>>>
>>>> Again, as I have stated, ramdisk is not a Linux only concept. In fact,
>>>> as Jan points out, initrd/initramfs are Linux specific implementations
>>>> of a ramdisk for which Xen doesn't even fully support. I am inclined to
>>>> ask the inverse of why hvm_start_info uses the name module. But that
>>>> aside, let's consider the fact that the field is only populated by the
>>>> device tree when a module type of BOOTMOD_RAMDISK is matched. And all
>>>> the uses of the field are when its value is stored into a local variable
>>>> called initrd.
>>>>
>>>> Though the biggest irony is that generally obtuse abstraction are
>>>> routinely blocked unless there is a tangible future case. Yet none was
>>>> offered in the comment. Thus on that principle alone, a request for a
>>>> tangible future use should have been requested and provided for the
>>>> change to be considered.
>>>
>>> Does it even need to be a _future_ use here? Aren't you working on
>>> abstracting domain creation, suitable (in principle) for all architectures?
>>> Isn't therefore a more generic name (as "module" is) preferable over a more
>>> specific one?
>>
>> Yes we are trying to build a future capability, but my point is let's
>> consider all possible known OS's start up today. What other boot module
>> could potentially be passed in that is exclusive of a ramdisk, thus
>> allowing a multiplex of the field. And the answer is none.
> 
> Is it? What if you are to start a nested Xen with its own kernel, initrd
> and perhaps even an XSM policy "module"? Or anything else that is multi-
> module capable (possibly but not necessarily because of having started
> out as multiboot)?

As you said on the v2 thread, just one other OS also using such a 
concept doesn't mean much.

With that said, as I am sure you are well aware of, the nested example 
is a far more complicated situation which there is currently no good 
abstraction in use.

And btw, as far as ramdisk usage goes, I would be remised not to mention 
that another significantly large OS uses a ramdisk for boot, and that 
would be Windows. Windows leverages a ramdisk as configuration store 
between the winlaoder and tcbloader when doing a DRTM boot.

v/r,
dps

[PATCH v3 02/16] x86/boot: introduce domid field to struct boot_domain

[Alejandro Vallejo]

From: "Daniel P. Smith" <dpsmith@apertussolutions.com>

boot_domain stores the domid until it is used to create (and allocate)
struct domain. d->domain_id is not available early enough.

boot_domain domids are initialized to DOMID_INVALID. If not overridden
by device tree, domids of DOMID_INVALID are assigned a valid value. The
domid will be optionally parsed from the device tree configuration.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
Signed-off-by: Alejandro Vallejo <agarciav@amd.com>
---
v3:
  * Initialize all boot_info.domains domid fields
  * Re-write commit message to justify the new domid field.
  * Use "%pd vcpu 0" in error message
  * Move xen.h include (introduced in v1) inside ifdef guards
---
 xen/arch/x86/include/asm/boot-domain.h |  4 ++++
 xen/arch/x86/setup.c                   | 10 +++++-----
 2 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/include/asm/boot-domain.h b/xen/arch/x86/include/asm/boot-domain.h
index 5e1e1a0b61..fcbedda0f0 100644
--- a/xen/arch/x86/include/asm/boot-domain.h
+++ b/xen/arch/x86/include/asm/boot-domain.h
@@ -8,7 +8,11 @@
 #ifndef __XEN_X86_BOOTDOMAIN_H__
 #define __XEN_X86_BOOTDOMAIN_H__
 
+#include <public/xen.h>
+
 struct boot_domain {
+    domid_t domid;
+
     struct boot_module *kernel;
     struct boot_module *module;
 
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index ddb10ea03d..585316f1ca 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -295,6 +295,7 @@ static const char *cmdline_cook(const char *p, const char *loader_name);
 struct boot_info __initdata xen_boot_info = {
     .loader = "unknown",
     .cmdline = "",
+    .domains = { [0 ... MAX_NR_BOOTDOMS - 1] = { .domid = DOMID_INVALID } },
 };
 
 static struct boot_info *__init multiboot_fill_boot_info(
@@ -994,7 +995,6 @@ static struct domain *__init create_dom0(struct boot_info *bi)
     };
     struct boot_domain *bd = &bi->domains[0];
     struct domain *d;
-    domid_t domid;
 
     if ( opt_dom0_pvh )
     {
@@ -1010,15 +1010,15 @@ static struct domain *__init create_dom0(struct boot_info *bi)
         dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
 
     /* Create initial domain.  Not d0 for pvshim. */
-    domid = get_initial_domain_id();
-    d = domain_create(domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
+    bd->domid = get_initial_domain_id();
+    d = domain_create(bd->domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
     if ( IS_ERR(d) )
-        panic("Error creating d%u: %ld\n", domid, PTR_ERR(d));
+        panic("Error creating d%u: %ld\n", bd->domid, PTR_ERR(d));
 
     init_dom0_cpuid_policy(d);
 
     if ( alloc_dom0_vcpu0(d) == NULL )
-        panic("Error creating d%uv0\n", domid);
+        panic("Error creating %pd vcpu 0\n", d);
 
     /* Grab the DOM0 command line. */
     if ( bd->kernel->cmdline_pa || bi->kextra )
-- 
2.43.0

Re: [PATCH v3 02/16] x86/boot: introduce domid field to struct boot_domain

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> @@ -1010,15 +1010,15 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>          dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
>  
>      /* Create initial domain.  Not d0 for pvshim. */
> -    domid = get_initial_domain_id();
> -    d = domain_create(domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
> +    bd->domid = get_initial_domain_id();
> +    d = domain_create(bd->domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
>      if ( IS_ERR(d) )
> -        panic("Error creating d%u: %ld\n", domid, PTR_ERR(d));
> +        panic("Error creating d%u: %ld\n", bd->domid, PTR_ERR(d));
>  
>      init_dom0_cpuid_policy(d);
>  
>      if ( alloc_dom0_vcpu0(d) == NULL )
> -        panic("Error creating d%uv0\n", domid);
> +        panic("Error creating %pd vcpu 0\n", d);

And why exactly is this not %pdv0? That's what would be possible to catch by
grep-ing for what vsnprintf() would emit for %pv. Preferably with that adjusted
(which can be done while committing):
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan

Re: [PATCH v3 02/16] x86/boot: introduce domid field to struct boot_domain

[Alejandro Vallejo]

On Wed Apr 9, 2025 at 7:34 AM BST, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> @@ -1010,15 +1010,15 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>>          dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
>>  
>>      /* Create initial domain.  Not d0 for pvshim. */
>> -    domid = get_initial_domain_id();
>> -    d = domain_create(domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
>> +    bd->domid = get_initial_domain_id();
>> +    d = domain_create(bd->domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
>>      if ( IS_ERR(d) )
>> -        panic("Error creating d%u: %ld\n", domid, PTR_ERR(d));
>> +        panic("Error creating d%u: %ld\n", bd->domid, PTR_ERR(d));
>>  
>>      init_dom0_cpuid_policy(d);
>>  
>>      if ( alloc_dom0_vcpu0(d) == NULL )
>> -        panic("Error creating d%uv0\n", domid);
>> +        panic("Error creating %pd vcpu 0\n", d);
>
> And why exactly is this not %pdv0?

Likely to avoid what looks like a highly cryptic format specifier. But I
agree your option seems nicer.

> That's what would be possible to catch by
> grep-ing for what vsnprintf() would emit for %pv. Preferably with that adjusted
> (which can be done while committing):

Yes, please.

> Acked-by: Jan Beulich <jbeulich@suse.com>

Thanks

>
> Jan

Cheers,
Alejandro

Re: [PATCH v3 02/16] x86/boot: introduce domid field to struct boot_domain

[Jason Andryuk]

On 2025-04-09 06:33, Alejandro Vallejo wrote:
> On Wed Apr 9, 2025 at 7:34 AM BST, Jan Beulich wrote:
>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>> @@ -1010,15 +1010,15 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>>>           dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
>>>   
>>>       /* Create initial domain.  Not d0 for pvshim. */
>>> -    domid = get_initial_domain_id();
>>> -    d = domain_create(domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
>>> +    bd->domid = get_initial_domain_id();
>>> +    d = domain_create(bd->domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
>>>       if ( IS_ERR(d) )
>>> -        panic("Error creating d%u: %ld\n", domid, PTR_ERR(d));
>>> +        panic("Error creating d%u: %ld\n", bd->domid, PTR_ERR(d));
>>>   
>>>       init_dom0_cpuid_policy(d);
>>>   
>>>       if ( alloc_dom0_vcpu0(d) == NULL )
>>> -        panic("Error creating d%uv0\n", domid);
>>> +        panic("Error creating %pd vcpu 0\n", d);
>>
>> And why exactly is this not %pdv0?
> 
> Likely to avoid what looks like a highly cryptic format specifier. But I
> agree your option seems nicer.

Hi, Jan.  Sorry, I made this change.  In earlier feedback you wrote:

 > That said, since vsprintf.c:print_vcpu() calls print_domain(), using
 > %pd is certainly an option here (inconsistencies would arise if %pv
 > and %pd presented domain IDs in [perhaps just slightly] different
 > ways).

I took that to mean you thought "faking" %pv is undesirable, so I just 
wrote in in a different form to avoid the potential inconsistency.  I'm 
fine with %pdv0.

>> Acked-by: Jan Beulich <jbeulich@suse.com>

Thanks,
Jason

[PATCH v3 03/16] x86/boot: add cmdline to struct boot_domain

[Alejandro Vallejo]

From: "Daniel P. Smith" <dpsmith@apertussolutions.com>

Add a container for the "cooked" command line for a domain. This
provides for the backing memory to be directly associated with the
domain being constructed.  This is done in anticipation that the domain
construction path may need to be invoked multiple times, thus ensuring
each instance had a distinct memory allocation.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
Signed-off-by: Alejandro Vallejo <agarciav@amd.com>
---
No changes on ACPI cmdline handling on PVH, as it's orthogonal to the
purpose of this patch.

v3:
  * s/xfree/XFREE/ on failed construct_dom0() to avoid a dangling
cmdline ptr.
  * Re-flow hvm_copy_to_guest_phys() into a multi-line call.
  * s/bd->cmdline != NULL/b->cmdline/ (to homogenise with the previous
    cmdline pointer check)
---
 xen/arch/x86/hvm/dom0_build.c          | 12 +++----
 xen/arch/x86/include/asm/boot-domain.h |  1 +
 xen/arch/x86/pv/dom0_build.c           |  4 +--
 xen/arch/x86/setup.c                   | 50 +++++++++++++++++++-------
 4 files changed, 47 insertions(+), 20 deletions(-)

diff --git a/xen/arch/x86/hvm/dom0_build.c b/xen/arch/x86/hvm/dom0_build.c
index 2a094b3145..ebad5a49b8 100644
--- a/xen/arch/x86/hvm/dom0_build.c
+++ b/xen/arch/x86/hvm/dom0_build.c
@@ -653,7 +653,6 @@ static int __init pvh_load_kernel(
     void *image_start = image_base + image->headroom;
     unsigned long image_len = image->size;
     unsigned long initrd_len = initrd ? initrd->size : 0;
-    const char *cmdline = image->cmdline_pa ? __va(image->cmdline_pa) : NULL;
     const char *initrd_cmdline = NULL;
     struct elf_binary elf;
     struct elf_dom_parms parms;
@@ -736,8 +735,8 @@ static int __init pvh_load_kernel(
             initrd = NULL;
     }
 
-    if ( cmdline )
-        extra_space += elf_round_up(&elf, strlen(cmdline) + 1);
+    if ( bd->cmdline )
+        extra_space += elf_round_up(&elf, strlen(bd->cmdline) + 1);
 
     last_addr = find_memory(d, &elf, extra_space);
     if ( last_addr == INVALID_PADDR )
@@ -778,9 +777,10 @@ static int __init pvh_load_kernel(
     /* Free temporary buffers. */
     free_boot_modules();
 
-    if ( cmdline != NULL )
+    if ( bd->cmdline )
     {
-        rc = hvm_copy_to_guest_phys(last_addr, cmdline, strlen(cmdline) + 1, v);
+        rc = hvm_copy_to_guest_phys(last_addr, bd->cmdline,
+                                    strlen(bd->cmdline) + 1, v);
         if ( rc )
         {
             printk("Unable to copy guest command line\n");
@@ -791,7 +791,7 @@ static int __init pvh_load_kernel(
          * Round up to 32/64 bits (depending on the guest kernel bitness) so
          * the modlist/start_info is aligned.
          */
-        last_addr += elf_round_up(&elf, strlen(cmdline) + 1);
+        last_addr += elf_round_up(&elf, strlen(bd->cmdline) + 1);
     }
     if ( initrd != NULL )
     {
diff --git a/xen/arch/x86/include/asm/boot-domain.h b/xen/arch/x86/include/asm/boot-domain.h
index fcbedda0f0..d7c6042e25 100644
--- a/xen/arch/x86/include/asm/boot-domain.h
+++ b/xen/arch/x86/include/asm/boot-domain.h
@@ -15,6 +15,7 @@ struct boot_domain {
 
     struct boot_module *kernel;
     struct boot_module *module;
+    const char *cmdline;
 
     struct domain *d;
 };
diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c
index b485eea05f..e1b78d47c2 100644
--- a/xen/arch/x86/pv/dom0_build.c
+++ b/xen/arch/x86/pv/dom0_build.c
@@ -972,8 +972,8 @@ static int __init dom0_construct(const struct boot_domain *bd)
     }
 
     memset(si->cmd_line, 0, sizeof(si->cmd_line));
-    if ( image->cmdline_pa )
-        strlcpy((char *)si->cmd_line, __va(image->cmdline_pa), sizeof(si->cmd_line));
+    if ( bd->cmdline )
+        strlcpy((char *)si->cmd_line, bd->cmdline, sizeof(si->cmd_line));
 
 #ifdef CONFIG_VIDEO
     if ( !pv_shim && fill_console_start_info((void *)(si + 1)) )
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 585316f1ca..83cb66e309 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -978,10 +978,30 @@ static unsigned int __init copy_bios_e820(struct e820entry *map, unsigned int li
     return n;
 }
 
-static struct domain *__init create_dom0(struct boot_info *bi)
+static size_t __init domain_cmdline_size(
+    struct boot_info *bi, struct boot_domain *bd)
 {
-    static char __initdata cmdline[MAX_GUEST_CMDLINE];
+    size_t s = bi->kextra ? strlen(bi->kextra) : 0;
+
+    s += bd->kernel->cmdline_pa ? strlen(__va(bd->kernel->cmdline_pa)) : 0;
 
+    if ( s == 0 )
+        return s;
+
+    /*
+     * Certain parameters from the Xen command line may be added to the dom0
+     * command line. Add additional space for the possible cases along with one
+     * extra char to hold \0.
+     */
+    s += strlen(" noapic") + strlen(" acpi=") + sizeof(acpi_param) + 1;
+
+    return s;
+}
+
+static struct domain *__init create_dom0(struct boot_info *bi)
+{
+    char *cmdline = NULL;
+    size_t cmdline_size;
     struct xen_domctl_createdomain dom0_cfg = {
         .flags = IS_ENABLED(CONFIG_TBOOT) ? XEN_DOMCTL_CDF_s3_integrity : 0,
         .max_evtchn_port = -1,
@@ -1020,20 +1040,24 @@ static struct domain *__init create_dom0(struct boot_info *bi)
     if ( alloc_dom0_vcpu0(d) == NULL )
         panic("Error creating %pd vcpu 0\n", d);
 
-    /* Grab the DOM0 command line. */
-    if ( bd->kernel->cmdline_pa || bi->kextra )
+    cmdline_size = domain_cmdline_size(bi, bd);
+    if ( cmdline_size )
     {
+        if ( !(cmdline = xzalloc_array(char, cmdline_size)) )
+            panic("Error allocating cmdline buffer for %pd\n", d);
+
         if ( bd->kernel->cmdline_pa )
-            safe_strcpy(cmdline,
-                        cmdline_cook(__va(bd->kernel->cmdline_pa), bi->loader));
+            strlcpy(cmdline,
+                    cmdline_cook(__va(bd->kernel->cmdline_pa), bi->loader),
+                    cmdline_size);
 
         if ( bi->kextra )
             /* kextra always includes exactly one leading space. */
-            safe_strcat(cmdline, bi->kextra);
+            strlcat(cmdline, bi->kextra, cmdline_size);
 
         /* Append any extra parameters. */
         if ( skip_ioapic_setup && !strstr(cmdline, "noapic") )
-            safe_strcat(cmdline, " noapic");
+            strlcat(cmdline, " noapic", cmdline_size);
 
         if ( (strlen(acpi_param) == 0) && acpi_disabled )
         {
@@ -1043,17 +1067,19 @@ static struct domain *__init create_dom0(struct boot_info *bi)
 
         if ( (strlen(acpi_param) != 0) && !strstr(cmdline, "acpi=") )
         {
-            safe_strcat(cmdline, " acpi=");
-            safe_strcat(cmdline, acpi_param);
+            strlcat(cmdline, " acpi=", cmdline_size);
+            strlcat(cmdline, acpi_param, cmdline_size);
         }
-
-        bd->kernel->cmdline_pa = __pa(cmdline);
+        bd->kernel->cmdline_pa = 0;
+        bd->cmdline = cmdline;
     }
 
     bd->d = d;
     if ( construct_dom0(bd) != 0 )
         panic("Could not construct domain 0\n");
 
+    XFREE(cmdline);
+
     return d;
 }
 
-- 
2.43.0

Re: [PATCH v3 03/16] x86/boot: add cmdline to struct boot_domain

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
> 
> Add a container for the "cooked" command line for a domain. This
> provides for the backing memory to be directly associated with the
> domain being constructed.  This is done in anticipation that the domain
> construction path may need to be invoked multiple times, thus ensuring
> each instance had a distinct memory allocation.
> 
> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
> Signed-off-by: Alejandro Vallejo <agarciav@amd.com>
> ---
> No changes on ACPI cmdline handling on PVH, as it's orthogonal to the
> purpose of this patch.
> 
> v3:
>   * s/xfree/XFREE/ on failed construct_dom0() to avoid a dangling
> cmdline ptr.
>   * Re-flow hvm_copy_to_guest_phys() into a multi-line call.
>   * s/bd->cmdline != NULL/b->cmdline/ (to homogenise with the previous
>     cmdline pointer check)
> ---
>  xen/arch/x86/hvm/dom0_build.c          | 12 +++----
>  xen/arch/x86/include/asm/boot-domain.h |  1 +
>  xen/arch/x86/pv/dom0_build.c           |  4 +--
>  xen/arch/x86/setup.c                   | 50 +++++++++++++++++++-------
>  4 files changed, 47 insertions(+), 20 deletions(-)
> 
> diff --git a/xen/arch/x86/hvm/dom0_build.c b/xen/arch/x86/hvm/dom0_build.c
> index 2a094b3145..ebad5a49b8 100644
> --- a/xen/arch/x86/hvm/dom0_build.c
> +++ b/xen/arch/x86/hvm/dom0_build.c
> @@ -653,7 +653,6 @@ static int __init pvh_load_kernel(
>      void *image_start = image_base + image->headroom;
>      unsigned long image_len = image->size;
>      unsigned long initrd_len = initrd ? initrd->size : 0;
> -    const char *cmdline = image->cmdline_pa ? __va(image->cmdline_pa) : NULL;
>      const char *initrd_cmdline = NULL;
>      struct elf_binary elf;
>      struct elf_dom_parms parms;
> @@ -736,8 +735,8 @@ static int __init pvh_load_kernel(
>              initrd = NULL;
>      }
>  
> -    if ( cmdline )
> -        extra_space += elf_round_up(&elf, strlen(cmdline) + 1);
> +    if ( bd->cmdline )
> +        extra_space += elf_round_up(&elf, strlen(bd->cmdline) + 1);
>  
>      last_addr = find_memory(d, &elf, extra_space);
>      if ( last_addr == INVALID_PADDR )
> @@ -778,9 +777,10 @@ static int __init pvh_load_kernel(
>      /* Free temporary buffers. */
>      free_boot_modules();
>  
> -    if ( cmdline != NULL )
> +    if ( bd->cmdline )
>      {
> -        rc = hvm_copy_to_guest_phys(last_addr, cmdline, strlen(cmdline) + 1, v);
> +        rc = hvm_copy_to_guest_phys(last_addr, bd->cmdline,
> +                                    strlen(bd->cmdline) + 1, v);
>          if ( rc )
>          {
>              printk("Unable to copy guest command line\n");
> @@ -791,7 +791,7 @@ static int __init pvh_load_kernel(
>           * Round up to 32/64 bits (depending on the guest kernel bitness) so
>           * the modlist/start_info is aligned.
>           */
> -        last_addr += elf_round_up(&elf, strlen(cmdline) + 1);
> +        last_addr += elf_round_up(&elf, strlen(bd->cmdline) + 1);
>      }
>      if ( initrd != NULL )
>      {

Perhaps better introduce a local variable cmdline_len? That would allow the first
if() to go away (but of course not its body).

> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -978,10 +978,30 @@ static unsigned int __init copy_bios_e820(struct e820entry *map, unsigned int li
>      return n;
>  }
>  
> -static struct domain *__init create_dom0(struct boot_info *bi)
> +static size_t __init domain_cmdline_size(
> +    struct boot_info *bi, struct boot_domain *bd)

const for both? And perhaps s/domain/dom0/ in the function name?

>  {
> -    static char __initdata cmdline[MAX_GUEST_CMDLINE];
> +    size_t s = bi->kextra ? strlen(bi->kextra) : 0;
> +
> +    s += bd->kernel->cmdline_pa ? strlen(__va(bd->kernel->cmdline_pa)) : 0;
>  
> +    if ( s == 0 )
> +        return s;

While this retains prior behavior, that prior behavior was certainly odd (and
pretty likely not meant to be like that).

> @@ -1043,17 +1067,19 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>  
>          if ( (strlen(acpi_param) != 0) && !strstr(cmdline, "acpi=") )
>          {
> -            safe_strcat(cmdline, " acpi=");
> -            safe_strcat(cmdline, acpi_param);
> +            strlcat(cmdline, " acpi=", cmdline_size);
> +            strlcat(cmdline, acpi_param, cmdline_size);
>          }
> -
> -        bd->kernel->cmdline_pa = __pa(cmdline);
> +        bd->kernel->cmdline_pa = 0;
> +        bd->cmdline = cmdline;
>      }
>  
>      bd->d = d;
>      if ( construct_dom0(bd) != 0 )
>          panic("Could not construct domain 0\n");
>  
> +    XFREE(cmdline);

While this tidies the local variable, what about bd->cmdline? As it stands this
gives the impression that you're freeing a pointer here which may still be used
through passing bd elsewhere.

Jan

Re: [PATCH v3 03/16] x86/boot: add cmdline to struct boot_domain

[Alejandro Vallejo]

On Wed Apr 9, 2025 at 7:48 AM BST, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>> 
>> Add a container for the "cooked" command line for a domain. This
>> provides for the backing memory to be directly associated with the
>> domain being constructed.  This is done in anticipation that the domain
>> construction path may need to be invoked multiple times, thus ensuring
>> each instance had a distinct memory allocation.
>> 
>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>> Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
>> Signed-off-by: Alejandro Vallejo <agarciav@amd.com>
>> ---
>> No changes on ACPI cmdline handling on PVH, as it's orthogonal to the
>> purpose of this patch.
>> 
>> v3:
>>   * s/xfree/XFREE/ on failed construct_dom0() to avoid a dangling
>> cmdline ptr.
>>   * Re-flow hvm_copy_to_guest_phys() into a multi-line call.
>>   * s/bd->cmdline != NULL/b->cmdline/ (to homogenise with the previous
>>     cmdline pointer check)
>> ---
>>  xen/arch/x86/hvm/dom0_build.c          | 12 +++----
>>  xen/arch/x86/include/asm/boot-domain.h |  1 +
>>  xen/arch/x86/pv/dom0_build.c           |  4 +--
>>  xen/arch/x86/setup.c                   | 50 +++++++++++++++++++-------
>>  4 files changed, 47 insertions(+), 20 deletions(-)
>> 
>> diff --git a/xen/arch/x86/hvm/dom0_build.c b/xen/arch/x86/hvm/dom0_build.c
>> index 2a094b3145..ebad5a49b8 100644
>> --- a/xen/arch/x86/hvm/dom0_build.c
>> +++ b/xen/arch/x86/hvm/dom0_build.c
>> @@ -653,7 +653,6 @@ static int __init pvh_load_kernel(
>>      void *image_start = image_base + image->headroom;
>>      unsigned long image_len = image->size;
>>      unsigned long initrd_len = initrd ? initrd->size : 0;
>> -    const char *cmdline = image->cmdline_pa ? __va(image->cmdline_pa) : NULL;
>>      const char *initrd_cmdline = NULL;
>>      struct elf_binary elf;
>>      struct elf_dom_parms parms;
>> @@ -736,8 +735,8 @@ static int __init pvh_load_kernel(
>>              initrd = NULL;
>>      }
>>  
>> -    if ( cmdline )
>> -        extra_space += elf_round_up(&elf, strlen(cmdline) + 1);
>> +    if ( bd->cmdline )
>> +        extra_space += elf_round_up(&elf, strlen(bd->cmdline) + 1);
>>  
>>      last_addr = find_memory(d, &elf, extra_space);
>>      if ( last_addr == INVALID_PADDR )
>> @@ -778,9 +777,10 @@ static int __init pvh_load_kernel(
>>      /* Free temporary buffers. */
>>      free_boot_modules();
>>  
>> -    if ( cmdline != NULL )
>> +    if ( bd->cmdline )
>>      {
>> -        rc = hvm_copy_to_guest_phys(last_addr, cmdline, strlen(cmdline) + 1, v);
>> +        rc = hvm_copy_to_guest_phys(last_addr, bd->cmdline,
>> +                                    strlen(bd->cmdline) + 1, v);
>>          if ( rc )
>>          {
>>              printk("Unable to copy guest command line\n");
>> @@ -791,7 +791,7 @@ static int __init pvh_load_kernel(
>>           * Round up to 32/64 bits (depending on the guest kernel bitness) so
>>           * the modlist/start_info is aligned.
>>           */
>> -        last_addr += elf_round_up(&elf, strlen(cmdline) + 1);
>> +        last_addr += elf_round_up(&elf, strlen(bd->cmdline) + 1);
>>      }
>>      if ( initrd != NULL )
>>      {
>
> Perhaps better introduce a local variable cmdline_len? That would allow the first
> if() to go away (but of course not its body).

I'd agree if the function body was smaller, but it has 16 locals
already. It's already quite hard to know what's going on, so I'd rather
not make the situation worse.

>
>> --- a/xen/arch/x86/setup.c
>> +++ b/xen/arch/x86/setup.c
>> @@ -978,10 +978,30 @@ static unsigned int __init copy_bios_e820(struct e820entry *map, unsigned int li
>>      return n;
>>  }
>>  
>> -static struct domain *__init create_dom0(struct boot_info *bi)
>> +static size_t __init domain_cmdline_size(
>> +    struct boot_info *bi, struct boot_domain *bd)
>
> const for both? And perhaps s/domain/dom0/ in the function name?
>
>>  {
>> -    static char __initdata cmdline[MAX_GUEST_CMDLINE];
>> +    size_t s = bi->kextra ? strlen(bi->kextra) : 0;
>> +
>> +    s += bd->kernel->cmdline_pa ? strlen(__va(bd->kernel->cmdline_pa)) : 0;
>>  
>> +    if ( s == 0 )
>> +        return s;
>
> While this retains prior behavior, that prior behavior was certainly odd (and
> pretty likely not meant to be like that).

What part of it? How would you propose it to behave? Do you mean that if
no cmdline is passed some ought to be allocated in case we want to
override it?

Either way, such a functional change is better suited for a different
patch that does just that, plus properly handling the acpi adjustments
for PVH dom0.

>
>> @@ -1043,17 +1067,19 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>>  
>>          if ( (strlen(acpi_param) != 0) && !strstr(cmdline, "acpi=") )
>>          {
>> -            safe_strcat(cmdline, " acpi=");
>> -            safe_strcat(cmdline, acpi_param);
>> +            strlcat(cmdline, " acpi=", cmdline_size);
>> +            strlcat(cmdline, acpi_param, cmdline_size);
>>          }
>> -
>> -        bd->kernel->cmdline_pa = __pa(cmdline);
>> +        bd->kernel->cmdline_pa = 0;
>> +        bd->cmdline = cmdline;
>>      }
>>  
>>      bd->d = d;
>>      if ( construct_dom0(bd) != 0 )
>>          panic("Could not construct domain 0\n");
>>  
>> +    XFREE(cmdline);
>
> While this tidies the local variable, what about bd->cmdline? As it stands this
> gives the impression that you're freeing a pointer here which may still be used
> through passing bd elsewhere.

That ought to have been bd->cmdline indeed.

>
> Jan

Cheers,
Alejandro

Re: [PATCH v3 03/16] x86/boot: add cmdline to struct boot_domain

[Alejandro Vallejo]

On Wed Apr 9, 2025 at 12:11 PM BST, Alejandro Vallejo wrote:
> On Wed Apr 9, 2025 at 7:48 AM BST, Jan Beulich wrote:
>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>> --- a/xen/arch/x86/setup.c
>>> +++ b/xen/arch/x86/setup.c
>>> @@ -978,10 +978,30 @@ static unsigned int __init copy_bios_e820(struct e820entry *map, unsigned int li
>>>      return n;
>>>  }
>>>  
>>> -static struct domain *__init create_dom0(struct boot_info *bi)
>>> +static size_t __init domain_cmdline_size(
>>> +    struct boot_info *bi, struct boot_domain *bd)
>>
>> const for both? And perhaps s/domain/dom0/ in the function name?

(missed this one)

Sure to the const pointers. But as the hyperlaunch effort progresses the
point is to turn all this into a more generic domain builders rather
than having dom0-specific stuff. Changing the name like that here to
adjust it in a few patches down the line doesn't seem worth the effort.

>> While this tidies the local variable, what about bd->cmdline? As it stands this
>> gives the impression that you're freeing a pointer here which may still be used
>> through passing bd elsewhere.
>
> That ought to have been bd->cmdline indeed.
>

Actually, it can't be. It's a "const char *", so XFREE() chokes on it.
I'll turn it into

    XFREE(cmdline);
    bd->cmdline = NULL;

instead.

Cheers,
Alejandro

Re: [PATCH v3 03/16] x86/boot: add cmdline to struct boot_domain

[Jan Beulich]

On 09.04.2025 13:28, Alejandro Vallejo wrote:
> On Wed Apr 9, 2025 at 12:11 PM BST, Alejandro Vallejo wrote:
>> On Wed Apr 9, 2025 at 7:48 AM BST, Jan Beulich wrote:
>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>> --- a/xen/arch/x86/setup.c
>>>> +++ b/xen/arch/x86/setup.c
>>>> @@ -978,10 +978,30 @@ static unsigned int __init copy_bios_e820(struct e820entry *map, unsigned int li
>>>>      return n;
>>>>  }
>>>>  
>>>> -static struct domain *__init create_dom0(struct boot_info *bi)
>>>> +static size_t __init domain_cmdline_size(
>>>> +    struct boot_info *bi, struct boot_domain *bd)
>>>
>>> const for both? And perhaps s/domain/dom0/ in the function name?
> 
> (missed this one)
> 
> Sure to the const pointers. But as the hyperlaunch effort progresses the
> point is to turn all this into a more generic domain builders rather
> than having dom0-specific stuff. Changing the name like that here to
> adjust it in a few patches down the line doesn't seem worth the effort.

Oh, if the function is going to fine further uses, that's likely okay.
("likely" because we've seen too many abandoned series, where we then
ended up with pieces that were never gaining their intended final
purpose.)

>>> While this tidies the local variable, what about bd->cmdline? As it stands this
>>> gives the impression that you're freeing a pointer here which may still be used
>>> through passing bd elsewhere.
>>
>> That ought to have been bd->cmdline indeed.
>>
> 
> Actually, it can't be. It's a "const char *", so XFREE() chokes on it.
> I'll turn it into
> 
>     XFREE(cmdline);
>     bd->cmdline = NULL;
> 
> instead.

At which point it can be xfree() again, seeing that the function ends
right afterwards?

Jan

Re: [PATCH v3 03/16] x86/boot: add cmdline to struct boot_domain

[Jan Beulich]

On 09.04.2025 13:11, Alejandro Vallejo wrote:
> On Wed Apr 9, 2025 at 7:48 AM BST, Jan Beulich wrote:
>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>> --- a/xen/arch/x86/hvm/dom0_build.c
>>> +++ b/xen/arch/x86/hvm/dom0_build.c
>>> @@ -653,7 +653,6 @@ static int __init pvh_load_kernel(
>>>      void *image_start = image_base + image->headroom;
>>>      unsigned long image_len = image->size;
>>>      unsigned long initrd_len = initrd ? initrd->size : 0;
>>> -    const char *cmdline = image->cmdline_pa ? __va(image->cmdline_pa) : NULL;
>>>      const char *initrd_cmdline = NULL;
>>>      struct elf_binary elf;
>>>      struct elf_dom_parms parms;
>>> @@ -736,8 +735,8 @@ static int __init pvh_load_kernel(
>>>              initrd = NULL;
>>>      }
>>>  
>>> -    if ( cmdline )
>>> -        extra_space += elf_round_up(&elf, strlen(cmdline) + 1);
>>> +    if ( bd->cmdline )
>>> +        extra_space += elf_round_up(&elf, strlen(bd->cmdline) + 1);
>>>  
>>>      last_addr = find_memory(d, &elf, extra_space);
>>>      if ( last_addr == INVALID_PADDR )
>>> @@ -778,9 +777,10 @@ static int __init pvh_load_kernel(
>>>      /* Free temporary buffers. */
>>>      free_boot_modules();
>>>  
>>> -    if ( cmdline != NULL )
>>> +    if ( bd->cmdline )
>>>      {
>>> -        rc = hvm_copy_to_guest_phys(last_addr, cmdline, strlen(cmdline) + 1, v);
>>> +        rc = hvm_copy_to_guest_phys(last_addr, bd->cmdline,
>>> +                                    strlen(bd->cmdline) + 1, v);
>>>          if ( rc )
>>>          {
>>>              printk("Unable to copy guest command line\n");
>>> @@ -791,7 +791,7 @@ static int __init pvh_load_kernel(
>>>           * Round up to 32/64 bits (depending on the guest kernel bitness) so
>>>           * the modlist/start_info is aligned.
>>>           */
>>> -        last_addr += elf_round_up(&elf, strlen(cmdline) + 1);
>>> +        last_addr += elf_round_up(&elf, strlen(bd->cmdline) + 1);
>>>      }
>>>      if ( initrd != NULL )
>>>      {
>>
>> Perhaps better introduce a local variable cmdline_len? That would allow the first
>> if() to go away (but of course not its body).
> 
> I'd agree if the function body was smaller, but it has 16 locals
> already. It's already quite hard to know what's going on, so I'd rather
> not make the situation worse.

You wouldn't: You'd replace one local var by another.

>>> --- a/xen/arch/x86/setup.c
>>> +++ b/xen/arch/x86/setup.c
>>> @@ -978,10 +978,30 @@ static unsigned int __init copy_bios_e820(struct e820entry *map, unsigned int li
>>>      return n;
>>>  }
>>>  
>>> -static struct domain *__init create_dom0(struct boot_info *bi)
>>> +static size_t __init domain_cmdline_size(
>>> +    struct boot_info *bi, struct boot_domain *bd)
>>
>> const for both? And perhaps s/domain/dom0/ in the function name?
>>
>>>  {
>>> -    static char __initdata cmdline[MAX_GUEST_CMDLINE];
>>> +    size_t s = bi->kextra ? strlen(bi->kextra) : 0;
>>> +
>>> +    s += bd->kernel->cmdline_pa ? strlen(__va(bd->kernel->cmdline_pa)) : 0;
>>>  
>>> +    if ( s == 0 )
>>> +        return s;
>>
>> While this retains prior behavior, that prior behavior was certainly odd (and
>> pretty likely not meant to be like that).
> 
> What part of it? How would you propose it to behave? Do you mean that if
> no cmdline is passed some ought to be allocated in case we want to
> override it?

"noapic" and "acpi=" want appending (if so intended) irrespective of there
being a non-empty command line already.

> Either way, such a functional change is better suited for a different
> patch that does just that, plus properly handling the acpi adjustments
> for PVH dom0.

Maybe. It's always odd to see issues live on when changes are made in their
area. For backportability, yes, the fix may want to be separate (and first).

Jan

Re: [PATCH v3 03/16] x86/boot: add cmdline to struct boot_domain

[Denis Mukhin]

On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:

> 
> 
> From: "Daniel P. Smith" dpsmith@apertussolutions.com
> 
> 
> Add a container for the "cooked" command line for a domain. This
> provides for the backing memory to be directly associated with the
> domain being constructed. This is done in anticipation that the domain
> construction path may need to be invoked multiple times, thus ensuring
> each instance had a distinct memory allocation.
> 
> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
> 
> Signed-off-by: Jason Andryuk jason.andryuk@amd.com
> 
> Signed-off-by: Alejandro Vallejo agarciav@amd.com
> 
> ---
> No changes on ACPI cmdline handling on PVH, as it's orthogonal to the
> purpose of this patch.
> 
> v3:
> * s/xfree/XFREE/ on failed construct_dom0() to avoid a dangling
> cmdline ptr.
> * Re-flow hvm_copy_to_guest_phys() into a multi-line call.
> * s/bd->cmdline != NULL/b->cmdline/ (to homogenise with the previous
> 
> cmdline pointer check)
> ---
> xen/arch/x86/hvm/dom0_build.c | 12 +++----
> xen/arch/x86/include/asm/boot-domain.h | 1 +
> xen/arch/x86/pv/dom0_build.c | 4 +--
> xen/arch/x86/setup.c | 50 +++++++++++++++++++-------
> 4 files changed, 47 insertions(+), 20 deletions(-)
> 
> diff --git a/xen/arch/x86/hvm/dom0_build.c b/xen/arch/x86/hvm/dom0_build.c
> index 2a094b3145..ebad5a49b8 100644
> --- a/xen/arch/x86/hvm/dom0_build.c
> +++ b/xen/arch/x86/hvm/dom0_build.c
> @@ -653,7 +653,6 @@ static int __init pvh_load_kernel(
> void *image_start = image_base + image->headroom;
> 
> unsigned long image_len = image->size;
> 
> unsigned long initrd_len = initrd ? initrd->size : 0;
> 
> - const char *cmdline = image->cmdline_pa ? __va(image->cmdline_pa) : NULL;
> 
> const char *initrd_cmdline = NULL;
> struct elf_binary elf;
> struct elf_dom_parms parms;
> @@ -736,8 +735,8 @@ static int __init pvh_load_kernel(
> initrd = NULL;
> }
> 
> - if ( cmdline )
> - extra_space += elf_round_up(&elf, strlen(cmdline) + 1);
> + if ( bd->cmdline )
> 
> + extra_space += elf_round_up(&elf, strlen(bd->cmdline) + 1);

I think it makes sense to use local variable to cache the results of 

  strlen(bd->cmdline) + 1

since the construct is used multiple times within pvh_load_kernel()

E.g. cmdline_len similarly to {image,initrd}_len.

> 
> 
> last_addr = find_memory(d, &elf, extra_space);
> if ( last_addr == INVALID_PADDR )
> @@ -778,9 +777,10 @@ static int __init pvh_load_kernel(
> /* Free temporary buffers. */
> free_boot_modules();
> 
> - if ( cmdline != NULL )
> + if ( bd->cmdline )
> 
> {
> - rc = hvm_copy_to_guest_phys(last_addr, cmdline, strlen(cmdline) + 1, v);
> + rc = hvm_copy_to_guest_phys(last_addr, bd->cmdline,
> 
> + strlen(bd->cmdline) + 1, v);
> 
> if ( rc )
> {
> printk("Unable to copy guest command line\n");
> @@ -791,7 +791,7 @@ static int __init pvh_load_kernel(
> * Round up to 32/64 bits (depending on the guest kernel bitness) so
> * the modlist/start_info is aligned.
> */
> - last_addr += elf_round_up(&elf, strlen(cmdline) + 1);
> + last_addr += elf_round_up(&elf, strlen(bd->cmdline) + 1);
> 
> }
> if ( initrd != NULL )
> {
> diff --git a/xen/arch/x86/include/asm/boot-domain.h b/xen/arch/x86/include/asm/boot-domain.h
> index fcbedda0f0..d7c6042e25 100644
> --- a/xen/arch/x86/include/asm/boot-domain.h
> +++ b/xen/arch/x86/include/asm/boot-domain.h
> @@ -15,6 +15,7 @@ struct boot_domain {
> 
> struct boot_module *kernel;
> struct boot_module *module;
> + const char *cmdline;
> 
> struct domain *d;
> };
> diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c
> index b485eea05f..e1b78d47c2 100644
> --- a/xen/arch/x86/pv/dom0_build.c
> +++ b/xen/arch/x86/pv/dom0_build.c
> @@ -972,8 +972,8 @@ static int __init dom0_construct(const struct boot_domain *bd)
> }
> 
> memset(si->cmd_line, 0, sizeof(si->cmd_line));
> 
> - if ( image->cmdline_pa )
> 
> - strlcpy((char *)si->cmd_line, __va(image->cmdline_pa), sizeof(si->cmd_line));
> 
> + if ( bd->cmdline )
> 
> + strlcpy((char *)si->cmd_line, bd->cmdline, sizeof(si->cmd_line));
> 
> 
> #ifdef CONFIG_VIDEO
> if ( !pv_shim && fill_console_start_info((void *)(si + 1)) )
> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
> index 585316f1ca..83cb66e309 100644
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -978,10 +978,30 @@ static unsigned int __init copy_bios_e820(struct e820entry *map, unsigned int li
> return n;
> }
> 
> -static struct domain *__init create_dom0(struct boot_info *bi)
> +static size_t __init domain_cmdline_size(
> + struct boot_info *bi, struct boot_domain *bd)
> {
> - static char __initdata cmdline[MAX_GUEST_CMDLINE];
> + size_t s = bi->kextra ? strlen(bi->kextra) : 0;
> 
> +
> + s += bd->kernel->cmdline_pa ? strlen(__va(bd->kernel->cmdline_pa)) : 0;
> 
> 
> + if ( s == 0 )
> + return s;
> +
> + /*
> + * Certain parameters from the Xen command line may be added to the dom0
> + * command line. Add additional space for the possible cases along with one
> + * extra char to hold \0.
> + */
> + s += strlen(" noapic") + strlen(" acpi=") + sizeof(acpi_param) + 1;
> +
> + return s;
> +}
> +
> +static struct domain *__init create_dom0(struct boot_info *bi)
> +{
> + char *cmdline = NULL;
> + size_t cmdline_size;
> struct xen_domctl_createdomain dom0_cfg = {
> .flags = IS_ENABLED(CONFIG_TBOOT) ? XEN_DOMCTL_CDF_s3_integrity : 0,
> .max_evtchn_port = -1,
> @@ -1020,20 +1040,24 @@ static struct domain *__init create_dom0(struct boot_info bi)
> if ( alloc_dom0_vcpu0(d) == NULL )
> panic("Error creating %pd vcpu 0\n", d);
> 
> - / Grab the DOM0 command line. */
> - if ( bd->kernel->cmdline_pa || bi->kextra )
> 
> + cmdline_size = domain_cmdline_size(bi, bd);
> + if ( cmdline_size )
> {
> + if ( !(cmdline = xzalloc_array(char, cmdline_size)) )
> + panic("Error allocating cmdline buffer for %pd\n", d);
> +
> if ( bd->kernel->cmdline_pa )
> 
> - safe_strcpy(cmdline,
> - cmdline_cook(__va(bd->kernel->cmdline_pa), bi->loader));
> 
> + strlcpy(cmdline,
> + cmdline_cook(__va(bd->kernel->cmdline_pa), bi->loader),
> 
> + cmdline_size);
> 
> if ( bi->kextra )
> 
> /* kextra always includes exactly one leading space. */
> - safe_strcat(cmdline, bi->kextra);
> 
> + strlcat(cmdline, bi->kextra, cmdline_size);
> 
> 
> /* Append any extra parameters. */
> if ( skip_ioapic_setup && !strstr(cmdline, "noapic") )
> - safe_strcat(cmdline, " noapic");
> + strlcat(cmdline, " noapic", cmdline_size);
> 
> if ( (strlen(acpi_param) == 0) && acpi_disabled )
> {
> @@ -1043,17 +1067,19 @@ static struct domain *__init create_dom0(struct boot_info *bi)
> 
> if ( (strlen(acpi_param) != 0) && !strstr(cmdline, "acpi=") )
> {
> - safe_strcat(cmdline, " acpi=");
> - safe_strcat(cmdline, acpi_param);
> + strlcat(cmdline, " acpi=", cmdline_size);
> + strlcat(cmdline, acpi_param, cmdline_size);
> }
> -
> - bd->kernel->cmdline_pa = __pa(cmdline);
> 
> + bd->kernel->cmdline_pa = 0;
> 
> + bd->cmdline = cmdline;
> 
> }
> 
> bd->d = d;
> 
> if ( construct_dom0(bd) != 0 )
> panic("Could not construct domain 0\n");
> 
> + XFREE(cmdline);
> +
> return d;
> }
> 
> --
> 2.43.0

Re: [PATCH v3 03/16] x86/boot: add cmdline to struct boot_domain

[Alejandro Vallejo]

On Wed Apr 9, 2025 at 10:05 PM BST, Denis Mukhin wrote:
> On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:
>
>> 
>> 
>> From: "Daniel P. Smith" dpsmith@apertussolutions.com
>> 
>> 
>> Add a container for the "cooked" command line for a domain. This
>> provides for the backing memory to be directly associated with the
>> domain being constructed. This is done in anticipation that the domain
>> construction path may need to be invoked multiple times, thus ensuring
>> each instance had a distinct memory allocation.
>> 
>> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
>> 
>> Signed-off-by: Jason Andryuk jason.andryuk@amd.com
>> 
>> Signed-off-by: Alejandro Vallejo agarciav@amd.com
>> 
>> ---
>> No changes on ACPI cmdline handling on PVH, as it's orthogonal to the
>> purpose of this patch.
>> 
>> v3:
>> * s/xfree/XFREE/ on failed construct_dom0() to avoid a dangling
>> cmdline ptr.
>> * Re-flow hvm_copy_to_guest_phys() into a multi-line call.
>> * s/bd->cmdline != NULL/b->cmdline/ (to homogenise with the previous
>> 
>> cmdline pointer check)
>> ---
>> xen/arch/x86/hvm/dom0_build.c | 12 +++----
>> xen/arch/x86/include/asm/boot-domain.h | 1 +
>> xen/arch/x86/pv/dom0_build.c | 4 +--
>> xen/arch/x86/setup.c | 50 +++++++++++++++++++-------
>> 4 files changed, 47 insertions(+), 20 deletions(-)
>> 
>> diff --git a/xen/arch/x86/hvm/dom0_build.c b/xen/arch/x86/hvm/dom0_build.c
>> index 2a094b3145..ebad5a49b8 100644
>> --- a/xen/arch/x86/hvm/dom0_build.c
>> +++ b/xen/arch/x86/hvm/dom0_build.c
>> @@ -653,7 +653,6 @@ static int __init pvh_load_kernel(
>> void *image_start = image_base + image->headroom;
>> 
>> unsigned long image_len = image->size;
>> 
>> unsigned long initrd_len = initrd ? initrd->size : 0;
>> 
>> - const char *cmdline = image->cmdline_pa ? __va(image->cmdline_pa) : NULL;
>> 
>> const char *initrd_cmdline = NULL;
>> struct elf_binary elf;
>> struct elf_dom_parms parms;
>> @@ -736,8 +735,8 @@ static int __init pvh_load_kernel(
>> initrd = NULL;
>> }
>> 
>> - if ( cmdline )
>> - extra_space += elf_round_up(&elf, strlen(cmdline) + 1);
>> + if ( bd->cmdline )
>> 
>> + extra_space += elf_round_up(&elf, strlen(bd->cmdline) + 1);
>
> I think it makes sense to use local variable to cache the results of 
>
>   strlen(bd->cmdline) + 1
>
> since the construct is used multiple times within pvh_load_kernel()
>
> E.g. cmdline_len similarly to {image,initrd}_len.
>

Seeing how you're the second person to propose the same thing, I'll bite
and just do it. I really dislike the wall of locals in
pvh_load_kernel(), but oh well...

Cheers,
Alejandro

[PATCH v3 04/16] kconfig: introduce option to independently enable libfdt

[Alejandro Vallejo]

From: "Daniel P. Smith" <dpsmith@apertussolutions.com>

Currently, the inclusion of libfdt is controlled by the CONFIG_HAS_DEVICE_TREE
kconfig flag. This flag also changes behavior in a few places, such as boot
module processing for XSM. To support the ability to include libfdt without
changing these behaviors, introduce CONFIG_LIBFDT. The inclusion of
libfdt is then moved under CONFIG_LIBFDT.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
---
v3:
    * Use CONFIG_LIBFDT instead of CONFIG_HAS_DEVICET_TREE
---
 xen/common/Kconfig  | 4 ++++
 xen/common/Makefile | 2 +-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index 06ae9751aa..00f74b1e32 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -72,6 +72,7 @@ config HAS_COMPAT
 
 config HAS_DEVICE_TREE
 	bool
+	select LIBFDT
 
 config HAS_DIT # Data Independent Timing
 	bool
@@ -106,6 +107,9 @@ config HAS_UBSAN
 config HAS_VMAP
 	bool
 
+config LIBFDT
+	bool
+
 config MEM_ACCESS_ALWAYS_ON
 	bool
 
diff --git a/xen/common/Makefile b/xen/common/Makefile
index 9da8a7244d..5a9041b0ee 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -79,7 +79,7 @@ obj-y += sched/
 obj-$(CONFIG_UBSAN) += ubsan/
 
 obj-$(CONFIG_NEEDS_LIBELF) += libelf/
-obj-$(CONFIG_HAS_DEVICE_TREE) += libfdt/
+obj-$(CONFIG_LIBFDT) += libfdt/
 
 CONF_FILE := $(if $(patsubst /%,,$(KCONFIG_CONFIG)),$(objtree)/)$(KCONFIG_CONFIG)
 $(obj)/config.gz: $(CONF_FILE)
-- 
2.43.0

Re: [PATCH v3 04/16] kconfig: introduce option to independently enable libfdt

[Denis Mukhin]

On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:

> 
> 
> From: "Daniel P. Smith" dpsmith@apertussolutions.com
> 
> 
> Currently, the inclusion of libfdt is controlled by the CONFIG_HAS_DEVICE_TREE
> kconfig flag. This flag also changes behavior in a few places, such as boot
> module processing for XSM. To support the ability to include libfdt without
> changing these behaviors, introduce CONFIG_LIBFDT. The inclusion of
> libfdt is then moved under CONFIG_LIBFDT.
> 
> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
> 
> Signed-off-by: Jason Andryuk jason.andryuk@amd.com

Reviewed-by: Denis Mukhin <dmukhin@ford.com>

> 
> ---
> v3:
> * Use CONFIG_LIBFDT instead of CONFIG_HAS_DEVICET_TREE
> ---
> xen/common/Kconfig | 4 ++++
> xen/common/Makefile | 2 +-
> 2 files changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/common/Kconfig b/xen/common/Kconfig
> index 06ae9751aa..00f74b1e32 100644
> --- a/xen/common/Kconfig
> +++ b/xen/common/Kconfig
> @@ -72,6 +72,7 @@ config HAS_COMPAT
> 
> config HAS_DEVICE_TREE
> bool
> + select LIBFDT
> 
> config HAS_DIT # Data Independent Timing
> bool
> @@ -106,6 +107,9 @@ config HAS_UBSAN
> config HAS_VMAP
> bool
> 
> +config LIBFDT
> + bool
> +
> config MEM_ACCESS_ALWAYS_ON
> bool
> 
> diff --git a/xen/common/Makefile b/xen/common/Makefile
> index 9da8a7244d..5a9041b0ee 100644
> --- a/xen/common/Makefile
> +++ b/xen/common/Makefile
> @@ -79,7 +79,7 @@ obj-y += sched/
> obj-$(CONFIG_UBSAN) += ubsan/
> 
> obj-$(CONFIG_NEEDS_LIBELF) += libelf/
> -obj-$(CONFIG_HAS_DEVICE_TREE) += libfdt/
> +obj-$(CONFIG_LIBFDT) += libfdt/
> 
> CONF_FILE := $(if $(patsubst /%,,$(KCONFIG_CONFIG)),$(objtree)/)$(KCONFIG_CONFIG)
> $(obj)/config.gz: $(CONF_FILE)
> --
> 2.43.0

Re: [PATCH v3 04/16] kconfig: introduce option to independently enable libfdt

[Alejandro Vallejo]

On Wed Apr 9, 2025 at 7:57 PM BST, Denis Mukhin wrote:
> On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:
>
>> 
>> 
>> From: "Daniel P. Smith" dpsmith@apertussolutions.com
>> 
>> 
>> Currently, the inclusion of libfdt is controlled by the CONFIG_HAS_DEVICE_TREE
>> kconfig flag. This flag also changes behavior in a few places, such as boot
>> module processing for XSM. To support the ability to include libfdt without
>> changing these behaviors, introduce CONFIG_LIBFDT. The inclusion of
>> libfdt is then moved under CONFIG_LIBFDT.
>> 
>> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
>> 
>> Signed-off-by: Jason Andryuk jason.andryuk@amd.com
>
> Reviewed-by: Denis Mukhin <dmukhin@ford.com>

Thanks

Cheers,
Alejandro

Re: [PATCH v3 04/16] kconfig: introduce option to independently enable libfdt

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
> 
> Currently, the inclusion of libfdt is controlled by the CONFIG_HAS_DEVICE_TREE
> kconfig flag. This flag also changes behavior in a few places, such as boot
> module processing for XSM. To support the ability to include libfdt without
> changing these behaviors, introduce CONFIG_LIBFDT. The inclusion of
> libfdt is then moved under CONFIG_LIBFDT.
> 
> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

Looks independent of earlier patches, and hence may be possible to go in right
away?

Jan

Re: [PATCH v3 04/16] kconfig: introduce option to independently enable libfdt

[Alejandro Vallejo]

On Thu Apr 10, 2025 at 10:04 AM BST, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>> 
>> Currently, the inclusion of libfdt is controlled by the CONFIG_HAS_DEVICE_TREE
>> kconfig flag. This flag also changes behavior in a few places, such as boot
>> module processing for XSM. To support the ability to include libfdt without
>> changing these behaviors, introduce CONFIG_LIBFDT. The inclusion of
>> libfdt is then moved under CONFIG_LIBFDT.
>> 
>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>> Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
>
> Acked-by: Jan Beulich <jbeulich@suse.com>

Thanks

>
> Looks independent of earlier patches, and hence may be possible to go in right
> away?
>
> Jan

Yes, it's independent of those.

Cheers,
Alejandro

[PATCH v3 05/16] kconfig: introduce domain builder config option

[Alejandro Vallejo]

From: "Daniel P. Smith" <dpsmith@apertussolutions.com>

Hyperlaunch domain builder will be the consolidated boot time domain
building logic framework. Introduces the config option to enable this
domain builder to eventually turn on the ability to load the domain
configuration via a flattened device tree.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
---
v3:
    * Minor s/_/-/ in the "source" line
---
 xen/arch/x86/Kconfig                |  2 ++
 xen/arch/x86/domain-builder/Kconfig | 15 +++++++++++++++
 2 files changed, 17 insertions(+)
 create mode 100644 xen/arch/x86/domain-builder/Kconfig

diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index de2fa37f08..a31002324e 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -321,6 +321,8 @@ config UCODE_SCAN_DEFAULT
 	  Enable if you have a Linux-based dom0 with microcode attached to the
 	  initramfs.
 
+source "arch/x86/domain-builder/Kconfig"
+
 endmenu
 
 source "common/Kconfig"
diff --git a/xen/arch/x86/domain-builder/Kconfig b/xen/arch/x86/domain-builder/Kconfig
new file mode 100644
index 0000000000..8ed493c3b5
--- /dev/null
+++ b/xen/arch/x86/domain-builder/Kconfig
@@ -0,0 +1,15 @@
+
+menu "Domain Builder Features"
+
+config DOMAIN_BUILDER
+	bool "Domain builder (UNSUPPORTED)" if UNSUPPORTED
+	select LIB_DEVICE_TREE
+	help
+	  Enables the domain builder capability to configure boot domain
+	  construction using a flattened device tree.
+
+	  This feature is currently experimental.
+
+	  If unsure, say N.
+
+endmenu
-- 
2.43.0

Re: [PATCH v3 05/16] kconfig: introduce domain builder config option

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> --- /dev/null
> +++ b/xen/arch/x86/domain-builder/Kconfig

This really looks to be the first patch where it needs settling on whether
all of this is to live under arch/x86/. If it is to, the reasons need writing
down somewhere - maybe not here, but at least in the cover letter.

> @@ -0,0 +1,15 @@
> +
> +menu "Domain Builder Features"
> +
> +config DOMAIN_BUILDER
> +	bool "Domain builder (UNSUPPORTED)" if UNSUPPORTED
> +	select LIB_DEVICE_TREE

Was this meant to be LIBFDT?

Jan

Re: [PATCH v3 05/16] kconfig: introduce domain builder config option

[Alejandro Vallejo]

On Thu Apr 10, 2025 at 10:08 AM BST, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> --- /dev/null
>> +++ b/xen/arch/x86/domain-builder/Kconfig
>
> This really looks to be the first patch where it needs settling on whether
> all of this is to live under arch/x86/. If it is to, the reasons need writing
> down somewhere - maybe not here, but at least in the cover letter.

I need to think about it. I haven't yet reviewed all the history about
the series and the related discussions on the matter. Moving this to
common code ought to be simple enough, but merging dom0less into it
seems like a pretty big undertaking. Presumably you merely mean the code
location and the scope of the Kconfig?

>
>> @@ -0,0 +1,15 @@
>> +
>> +menu "Domain Builder Features"
>> +
>> +config DOMAIN_BUILDER
>> +	bool "Domain builder (UNSUPPORTED)" if UNSUPPORTED
>> +	select LIB_DEVICE_TREE
>
> Was this meant to be LIBFDT?
>
> Jan

Yes, the next patch turns it into LIBFDT. I spent more than I should've
wondering "How does the final series even work at all?" before looking
ahead and noticing this select was adjusted on the wrong patch.

Cheers,
Alejandro

Re: [PATCH v3 05/16] kconfig: introduce domain builder config option

[Jan Beulich]

On 10.04.2025 14:52, Alejandro Vallejo wrote:
> On Thu Apr 10, 2025 at 10:08 AM BST, Jan Beulich wrote:
>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>> --- /dev/null
>>> +++ b/xen/arch/x86/domain-builder/Kconfig
>>
>> This really looks to be the first patch where it needs settling on whether
>> all of this is to live under arch/x86/. If it is to, the reasons need writing
>> down somewhere - maybe not here, but at least in the cover letter.
> 
> I need to think about it. I haven't yet reviewed all the history about
> the series and the related discussions on the matter. Moving this to
> common code ought to be simple enough, but merging dom0less into it
> seems like a pretty big undertaking. Presumably you merely mean the code
> location and the scope of the Kconfig?

Code location only: Yes, definitely. Kconfig: The scope would presumably still
be x86-only, until e.g. Arm was enabled, too.

Jan

[PATCH v3 06/16] x86/hyperlaunch: introduce the domain builder

[Alejandro Vallejo]

From: "Daniel P. Smith" <dpsmith@apertussolutions.com>

Introduce the domain builder which is capable of consuming a device tree as the
first boot module. If it finds a device tree as the first boot module, it will
set its type to BOOTMOD_FDT. This change only detects the boot module and
continues to boot with slight change to the boot convention that the dom0
kernel is no longer first boot module but is the second.

No functional change intended.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
---
v3:
    * Move obj-y += domain-builder/
    * Remove blank line in Makefile
    * const in has_hyperlaunch_fdt()
    * CONFIG_LIBFDT rename
    * Use boot_info forward declaration
    * Rename domainbuilder.h to domain-builder.h
    * Add fdt NULL check
---
 xen/arch/x86/Makefile                     |  1 +
 xen/arch/x86/domain-builder/Kconfig       |  2 +-
 xen/arch/x86/domain-builder/Makefile      |  2 +
 xen/arch/x86/domain-builder/core.c        | 57 +++++++++++++++++++++++
 xen/arch/x86/domain-builder/fdt.c         | 37 +++++++++++++++
 xen/arch/x86/domain-builder/fdt.h         | 21 +++++++++
 xen/arch/x86/include/asm/bootinfo.h       |  3 ++
 xen/arch/x86/include/asm/domain-builder.h |  8 ++++
 xen/arch/x86/setup.c                      | 17 ++++---
 9 files changed, 141 insertions(+), 7 deletions(-)
 create mode 100644 xen/arch/x86/domain-builder/Makefile
 create mode 100644 xen/arch/x86/domain-builder/core.c
 create mode 100644 xen/arch/x86/domain-builder/fdt.c
 create mode 100644 xen/arch/x86/domain-builder/fdt.h
 create mode 100644 xen/arch/x86/include/asm/domain-builder.h

diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index f59c9665fd..deae31d627 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -1,6 +1,7 @@
 obj-y += acpi/
 obj-y += boot/
 obj-y += cpu/
+obj-y += domain-builder/
 obj-y += efi/
 obj-y += genapic/
 obj-$(CONFIG_GUEST) += guest/
diff --git a/xen/arch/x86/domain-builder/Kconfig b/xen/arch/x86/domain-builder/Kconfig
index 8ed493c3b5..51d549c20d 100644
--- a/xen/arch/x86/domain-builder/Kconfig
+++ b/xen/arch/x86/domain-builder/Kconfig
@@ -3,7 +3,7 @@ menu "Domain Builder Features"
 
 config DOMAIN_BUILDER
 	bool "Domain builder (UNSUPPORTED)" if UNSUPPORTED
-	select LIB_DEVICE_TREE
+	select LIBFDT
 	help
 	  Enables the domain builder capability to configure boot domain
 	  construction using a flattened device tree.
diff --git a/xen/arch/x86/domain-builder/Makefile b/xen/arch/x86/domain-builder/Makefile
new file mode 100644
index 0000000000..b10cd56b28
--- /dev/null
+++ b/xen/arch/x86/domain-builder/Makefile
@@ -0,0 +1,2 @@
+obj-$(CONFIG_DOMAIN_BUILDER) += fdt.init.o
+obj-y += core.init.o
diff --git a/xen/arch/x86/domain-builder/core.c b/xen/arch/x86/domain-builder/core.c
new file mode 100644
index 0000000000..d6ae94f45c
--- /dev/null
+++ b/xen/arch/x86/domain-builder/core.c
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2024, Apertus Solutions, LLC
+ */
+#include <xen/err.h>
+#include <xen/init.h>
+#include <xen/kconfig.h>
+#include <xen/lib.h>
+
+#include <asm/bootinfo.h>
+
+#include "fdt.h"
+
+void __init builder_init(struct boot_info *bi)
+{
+    if ( IS_ENABLED(CONFIG_DOMAIN_BUILDER) )
+    {
+        int ret;
+
+        switch ( ret = has_hyperlaunch_fdt(bi) )
+        {
+        case 0:
+            printk("Hyperlaunch device tree detected\n");
+            bi->hyperlaunch_enabled = true;
+            bi->mods[0].type = BOOTMOD_FDT;
+            break;
+
+        case -EINVAL:
+            printk("Hyperlaunch device tree was not detected\n");
+            bi->hyperlaunch_enabled = false;
+            break;
+
+        case -ENOENT:
+        case -ENODATA:
+            printk("Device tree found, but not hyperlaunch (%d)\n", ret);
+            bi->hyperlaunch_enabled = false;
+            bi->mods[0].type = BOOTMOD_FDT;
+            break;
+
+        default:
+            printk("Unknown error (%d) occured checking for hyperlaunch device tree\n",
+                   ret);
+            bi->hyperlaunch_enabled = false;
+            break;
+        }
+    }
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
new file mode 100644
index 0000000000..aaf8c1cc16
--- /dev/null
+++ b/xen/arch/x86/domain-builder/fdt.c
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2024, Apertus Solutions, LLC
+ */
+#include <xen/err.h>
+#include <xen/init.h>
+#include <xen/lib.h>
+#include <xen/libfdt/libfdt.h>
+
+#include <asm/bootinfo.h>
+#include <asm/page.h>
+#include <asm/setup.h>
+
+#include "fdt.h"
+
+int __init has_hyperlaunch_fdt(const struct boot_info *bi)
+{
+    int ret = 0;
+    const void *fdt = bootstrap_map_bm(&bi->mods[HYPERLAUNCH_MODULE_IDX]);
+
+    if ( !fdt || fdt_check_header(fdt) < 0 )
+        ret = -EINVAL;
+
+    bootstrap_unmap();
+
+    return ret;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/x86/domain-builder/fdt.h b/xen/arch/x86/domain-builder/fdt.h
new file mode 100644
index 0000000000..8e62cd843e
--- /dev/null
+++ b/xen/arch/x86/domain-builder/fdt.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
+#ifndef __XEN_X86_FDT_H__
+#define __XEN_X86_FDT_H__
+
+#include <xen/init.h>
+
+struct boot_info;
+
+/* hyperlaunch fdt is required to be module 0 */
+#define HYPERLAUNCH_MODULE_IDX 0
+
+#ifdef CONFIG_DOMAIN_BUILDER
+int has_hyperlaunch_fdt(const struct boot_info *bi);
+#else
+static inline int __init has_hyperlaunch_fdt(const struct boot_info *bi)
+{
+    return -EINVAL;
+}
+#endif
+
+#endif /* __XEN_X86_FDT_H__ */
diff --git a/xen/arch/x86/include/asm/bootinfo.h b/xen/arch/x86/include/asm/bootinfo.h
index 3afc214c17..82c2650fcf 100644
--- a/xen/arch/x86/include/asm/bootinfo.h
+++ b/xen/arch/x86/include/asm/bootinfo.h
@@ -27,6 +27,7 @@ enum bootmod_type {
     BOOTMOD_RAMDISK,
     BOOTMOD_MICROCODE,
     BOOTMOD_XSM_POLICY,
+    BOOTMOD_FDT,
 };
 
 struct boot_module {
@@ -80,6 +81,8 @@ struct boot_info {
     paddr_t memmap_addr;
     size_t memmap_length;
 
+    bool hyperlaunch_enabled;
+
     unsigned int nr_modules;
     struct boot_module mods[MAX_NR_BOOTMODS + 1];
     struct boot_domain domains[MAX_NR_BOOTDOMS];
diff --git a/xen/arch/x86/include/asm/domain-builder.h b/xen/arch/x86/include/asm/domain-builder.h
new file mode 100644
index 0000000000..b6d9ba94de
--- /dev/null
+++ b/xen/arch/x86/include/asm/domain-builder.h
@@ -0,0 +1,8 @@
+#ifndef __XEN_X86_DOMBUILDER_H__
+#define __XEN_X86_DOMBUILDER_H__
+
+struct boot_info;
+
+void builder_init(struct boot_info *bi);
+
+#endif
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 83cb66e309..e5d78bcb48 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -36,6 +36,7 @@
 #include <asm/bzimage.h>
 #include <asm/cpu-policy.h>
 #include <asm/desc.h>
+#include <asm/domain-builder.h>
 #include <asm/e820.h>
 #include <asm/edd.h>
 #include <asm/genapic.h>
@@ -1281,9 +1282,12 @@ void asmlinkage __init noreturn __start_xen(void)
                bi->nr_modules);
     }
 
-    /* Dom0 kernel is always first */
-    bi->mods[0].type = BOOTMOD_KERNEL;
-    bi->domains[0].kernel = &bi->mods[0];
+    builder_init(bi);
+
+    /* Find first unknown boot module to use as Dom0 kernel */
+    i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
+    bi->mods[i].type = BOOTMOD_KERNEL;
+    bi->domains[0].kernel = &bi->mods[i];
 
     if ( pvh_boot )
     {
@@ -1466,8 +1470,9 @@ void asmlinkage __init noreturn __start_xen(void)
         xen->size  = __2M_rwdata_end - _stext;
     }
 
-    bi->mods[0].headroom =
-        bzimage_headroom(bootstrap_map_bm(&bi->mods[0]), bi->mods[0].size);
+    bi->domains[0].kernel->headroom =
+        bzimage_headroom(bootstrap_map_bm(bi->domains[0].kernel),
+                         bi->domains[0].kernel->size);
     bootstrap_unmap();
 
 #ifndef highmem_start
@@ -1591,7 +1596,7 @@ void asmlinkage __init noreturn __start_xen(void)
 #endif
     }
 
-    if ( bi->mods[0].headroom && !bi->mods[0].relocated )
+    if ( bi->domains[0].kernel->headroom && !bi->domains[0].kernel->relocated )
         panic("Not enough memory to relocate the dom0 kernel image\n");
     for ( i = 0; i < bi->nr_modules; ++i )
     {
-- 
2.43.0

Re: [PATCH v3 06/16] x86/hyperlaunch: introduce the domain builder

[Denis Mukhin]

On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:

> 
> 
> From: "Daniel P. Smith" dpsmith@apertussolutions.com
> 
> 
> Introduce the domain builder which is capable of consuming a device tree as the
> first boot module. If it finds a device tree as the first boot module, it will
> set its type to BOOTMOD_FDT. This change only detects the boot module and
> continues to boot with slight change to the boot convention that the dom0
> kernel is no longer first boot module but is the second.
> 
> No functional change intended.
> 
> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
> 
> Signed-off-by: Jason Andryuk jason.andryuk@amd.com
> 
> ---
> v3:
> * Move obj-y += domain-builder/
> * Remove blank line in Makefile
> * const in has_hyperlaunch_fdt()
> * CONFIG_LIBFDT rename
> * Use boot_info forward declaration
> * Rename domainbuilder.h to domain-builder.h
> * Add fdt NULL check
> ---
> xen/arch/x86/Makefile | 1 +
> xen/arch/x86/domain-builder/Kconfig | 2 +-
> xen/arch/x86/domain-builder/Makefile | 2 +
> xen/arch/x86/domain-builder/core.c | 57 +++++++++++++++++++++++
> xen/arch/x86/domain-builder/fdt.c | 37 +++++++++++++++
> xen/arch/x86/domain-builder/fdt.h | 21 +++++++++

I have a general question.

Wouldn't that make sense to use arch-independent placeholder for domain builder
code right from the starting point?

For example something like xen/common/domain-builder ?

My understanding is that there's a lot of code in the domain builder which
can be potentially shared/re-used with non-x86 architectures.

Also, that seems to align with the intention to have common arch-independent
subsystems in the code base:

   https://docs.google.com/presentation/d/1q9cjJZLUxUo1YzMpCxVHuL-ZOGoFaO9haHfRBK4i4Uc/edit?slide=id.g32afc87aef4_0_18#slide=id.g32afc87aef4_0_18

> xen/arch/x86/include/asm/bootinfo.h | 3 ++
> xen/arch/x86/include/asm/domain-builder.h | 8 ++++
> xen/arch/x86/setup.c | 17 ++++---
> 9 files changed, 141 insertions(+), 7 deletions(-)
> create mode 100644 xen/arch/x86/domain-builder/Makefile
> create mode 100644 xen/arch/x86/domain-builder/core.c
> create mode 100644 xen/arch/x86/domain-builder/fdt.c
> create mode 100644 xen/arch/x86/domain-builder/fdt.h
> create mode 100644 xen/arch/x86/include/asm/domain-builder.h
> 
> diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
> index f59c9665fd..deae31d627 100644
> --- a/xen/arch/x86/Makefile
> +++ b/xen/arch/x86/Makefile
> @@ -1,6 +1,7 @@
> obj-y += acpi/
> obj-y += boot/
> obj-y += cpu/
> +obj-y += domain-builder/
> obj-y += efi/
> obj-y += genapic/
> obj-$(CONFIG_GUEST) += guest/
> diff --git a/xen/arch/x86/domain-builder/Kconfig b/xen/arch/x86/domain-builder/Kconfig
> index 8ed493c3b5..51d549c20d 100644
> --- a/xen/arch/x86/domain-builder/Kconfig
> +++ b/xen/arch/x86/domain-builder/Kconfig
> @@ -3,7 +3,7 @@ menu "Domain Builder Features"
> 
> config DOMAIN_BUILDER
> bool "Domain builder (UNSUPPORTED)" if UNSUPPORTED
> - select LIB_DEVICE_TREE
> + select LIBFDT
> help
> Enables the domain builder capability to configure boot domain
> construction using a flattened device tree.
> diff --git a/xen/arch/x86/domain-builder/Makefile b/xen/arch/x86/domain-builder/Makefile
> new file mode 100644
> index 0000000000..b10cd56b28
> --- /dev/null
> +++ b/xen/arch/x86/domain-builder/Makefile
> @@ -0,0 +1,2 @@
> +obj-$(CONFIG_DOMAIN_BUILDER) += fdt.init.o
> +obj-y += core.init.o
> diff --git a/xen/arch/x86/domain-builder/core.c b/xen/arch/x86/domain-builder/core.c
> new file mode 100644
> index 0000000000..d6ae94f45c
> --- /dev/null
> +++ b/xen/arch/x86/domain-builder/core.c
> @@ -0,0 +1,57 @@
> +/* SPDX-License-Identifier: GPL-2.0-only /
> +/
> + * Copyright (C) 2024, Apertus Solutions, LLC
> + */
> +#include <xen/err.h>
> 
> +#include <xen/init.h>
> 
> +#include <xen/kconfig.h>
> 
> +#include <xen/lib.h>
> 
> +
> +#include <asm/bootinfo.h>
> 
> +
> +#include "fdt.h"
> +
> +void __init builder_init(struct boot_info *bi)
> +{
> + if ( IS_ENABLED(CONFIG_DOMAIN_BUILDER) )
> + {
> + int ret;
> +
> + switch ( ret = has_hyperlaunch_fdt(bi) )
> + {
> + case 0:
> + printk("Hyperlaunch device tree detected\n");
> + bi->hyperlaunch_enabled = true;
> 
> + bi->mods[0].type = BOOTMOD_FDT;
> 
> + break;
> +
> + case -EINVAL:
> + printk("Hyperlaunch device tree was not detected\n");
> + bi->hyperlaunch_enabled = false;
> 
> + break;
> +
> + case -ENOENT:
> + case -ENODATA:
> + printk("Device tree found, but not hyperlaunch (%d)\n", ret);
> + bi->hyperlaunch_enabled = false;
> 
> + bi->mods[0].type = BOOTMOD_FDT;
> 
> + break;
> +
> + default:
> + printk("Unknown error (%d) occured checking for hyperlaunch device tree\n",
> + ret);
> + bi->hyperlaunch_enabled = false;
> 
> + break;
> + }
> + }
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + /
> diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
> new file mode 100644
> index 0000000000..aaf8c1cc16
> --- /dev/null
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -0,0 +1,37 @@
> +/ SPDX-License-Identifier: GPL-2.0-only /
> +/
> + * Copyright (C) 2024, Apertus Solutions, LLC
> + */
> +#include <xen/err.h>
> 
> +#include <xen/init.h>
> 
> +#include <xen/lib.h>
> 
> +#include <xen/libfdt/libfdt.h>
> 
> +
> +#include <asm/bootinfo.h>
> 
> +#include <asm/page.h>
> 
> +#include <asm/setup.h>
> 
> +
> +#include "fdt.h"
> +
> +int __init has_hyperlaunch_fdt(const struct boot_info *bi)
> +{
> + int ret = 0;
> + const void *fdt = bootstrap_map_bm(&bi->mods[HYPERLAUNCH_MODULE_IDX]);
> 
> +
> + if ( !fdt || fdt_check_header(fdt) < 0 )
> + ret = -EINVAL;
> +
> + bootstrap_unmap();
> +
> + return ret;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + /
> diff --git a/xen/arch/x86/domain-builder/fdt.h b/xen/arch/x86/domain-builder/fdt.h
> new file mode 100644
> index 0000000000..8e62cd843e
> --- /dev/null
> +++ b/xen/arch/x86/domain-builder/fdt.h
> @@ -0,0 +1,21 @@
> +/ SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-2-Clause) */
> +#ifndef XEN_X86_FDT_H
> +#define XEN_X86_FDT_H
> +
> +#include <xen/init.h>
> 
> +
> +struct boot_info;
> +
> +/* hyperlaunch fdt is required to be module 0 */
> +#define HYPERLAUNCH_MODULE_IDX 0
> +
> +#ifdef CONFIG_DOMAIN_BUILDER
> +int has_hyperlaunch_fdt(const struct boot_info *bi);
> +#else
> +static inline int __init has_hyperlaunch_fdt(const struct boot_info bi)
> +{
> + return -EINVAL;
> +}
> +#endif
> +
> +#endif / XEN_X86_FDT_H */
> diff --git a/xen/arch/x86/include/asm/bootinfo.h b/xen/arch/x86/include/asm/bootinfo.h
> index 3afc214c17..82c2650fcf 100644
> --- a/xen/arch/x86/include/asm/bootinfo.h
> +++ b/xen/arch/x86/include/asm/bootinfo.h
> @@ -27,6 +27,7 @@ enum bootmod_type {
> BOOTMOD_RAMDISK,
> BOOTMOD_MICROCODE,
> BOOTMOD_XSM_POLICY,
> + BOOTMOD_FDT,
> };
> 
> struct boot_module {
> @@ -80,6 +81,8 @@ struct boot_info {
> paddr_t memmap_addr;
> size_t memmap_length;
> 
> + bool hyperlaunch_enabled;
> +
> unsigned int nr_modules;
> struct boot_module mods[MAX_NR_BOOTMODS + 1];
> struct boot_domain domains[MAX_NR_BOOTDOMS];
> diff --git a/xen/arch/x86/include/asm/domain-builder.h b/xen/arch/x86/include/asm/domain-builder.h
> new file mode 100644
> index 0000000000..b6d9ba94de
> --- /dev/null
> +++ b/xen/arch/x86/include/asm/domain-builder.h
> @@ -0,0 +1,8 @@
> +#ifndef XEN_X86_DOMBUILDER_H
> +#define XEN_X86_DOMBUILDER_H
> +
> +struct boot_info;
> +
> +void builder_init(struct boot_info *bi);
> +
> +#endif
> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
> index 83cb66e309..e5d78bcb48 100644
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -36,6 +36,7 @@
> #include <asm/bzimage.h>
> 
> #include <asm/cpu-policy.h>
> 
> #include <asm/desc.h>
> 
> +#include <asm/domain-builder.h>
> 
> #include <asm/e820.h>
> 
> #include <asm/edd.h>
> 
> #include <asm/genapic.h>
> 
> @@ -1281,9 +1282,12 @@ void asmlinkage __init noreturn __start_xen(void)
> bi->nr_modules);
> 
> }
> 
> - /* Dom0 kernel is always first */
> - bi->mods[0].type = BOOTMOD_KERNEL;
> 
> - bi->domains[0].kernel = &bi->mods[0];
> 
> + builder_init(bi);
> +
> + /* Find first unknown boot module to use as Dom0 kernel */
> + i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
> + bi->mods[i].type = BOOTMOD_KERNEL;
> 
> + bi->domains[0].kernel = &bi->mods[i];
> 
> 
> if ( pvh_boot )
> {
> @@ -1466,8 +1470,9 @@ void asmlinkage __init noreturn __start_xen(void)
> xen->size = __2M_rwdata_end - _stext;
> 
> }
> 
> - bi->mods[0].headroom =
> 
> - bzimage_headroom(bootstrap_map_bm(&bi->mods[0]), bi->mods[0].size);
> 
> + bi->domains[0].kernel->headroom =
> 
> + bzimage_headroom(bootstrap_map_bm(bi->domains[0].kernel),
> 
> + bi->domains[0].kernel->size);
> 
> bootstrap_unmap();
> 
> #ifndef highmem_start
> @@ -1591,7 +1596,7 @@ void asmlinkage __init noreturn __start_xen(void)
> #endif
> }
> 
> - if ( bi->mods[0].headroom && !bi->mods[0].relocated )
> 
> + if ( bi->domains[0].kernel->headroom && !bi->domains[0].kernel->relocated )
> 
> panic("Not enough memory to relocate the dom0 kernel image\n");
> for ( i = 0; i < bi->nr_modules; ++i )
> 
> {
> --
> 2.43.0

Re: [PATCH v3 06/16] x86/hyperlaunch: introduce the domain builder

[Jan Beulich]

On 09.04.2025 20:53, Denis Mukhin wrote:
> On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:
> 
>>
>>
>> From: "Daniel P. Smith" dpsmith@apertussolutions.com
>>
>>
>> Introduce the domain builder which is capable of consuming a device tree as the
>> first boot module. If it finds a device tree as the first boot module, it will
>> set its type to BOOTMOD_FDT. This change only detects the boot module and
>> continues to boot with slight change to the boot convention that the dom0
>> kernel is no longer first boot module but is the second.
>>
>> No functional change intended.
>>
>> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
>>
>> Signed-off-by: Jason Andryuk jason.andryuk@amd.com
>>
>> ---
>> v3:
>> * Move obj-y += domain-builder/
>> * Remove blank line in Makefile
>> * const in has_hyperlaunch_fdt()
>> * CONFIG_LIBFDT rename
>> * Use boot_info forward declaration
>> * Rename domainbuilder.h to domain-builder.h
>> * Add fdt NULL check
>> ---
>> xen/arch/x86/Makefile | 1 +
>> xen/arch/x86/domain-builder/Kconfig | 2 +-
>> xen/arch/x86/domain-builder/Makefile | 2 +
>> xen/arch/x86/domain-builder/core.c | 57 +++++++++++++++++++++++
>> xen/arch/x86/domain-builder/fdt.c | 37 +++++++++++++++
>> xen/arch/x86/domain-builder/fdt.h | 21 +++++++++
> 
> I have a general question.
> 
> Wouldn't that make sense to use arch-independent placeholder for domain builder
> code right from the starting point?
> 
> For example something like xen/common/domain-builder ?
> 
> My understanding is that there's a lot of code in the domain builder which
> can be potentially shared/re-used with non-x86 architectures.

And indeed this point was already raised before.

Jan

Re: [PATCH v3 06/16] x86/hyperlaunch: introduce the domain builder

[Alejandro Vallejo]

On Wed Apr 9, 2025 at 7:53 PM BST, Denis Mukhin wrote:
> On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:
>
>> 
>> 
>> From: "Daniel P. Smith" dpsmith@apertussolutions.com
>> 
>> 
>> Introduce the domain builder which is capable of consuming a device tree as the
>> first boot module. If it finds a device tree as the first boot module, it will
>> set its type to BOOTMOD_FDT. This change only detects the boot module and
>> continues to boot with slight change to the boot convention that the dom0
>> kernel is no longer first boot module but is the second.
>> 
>> No functional change intended.
>> 
>> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
>> 
>> Signed-off-by: Jason Andryuk jason.andryuk@amd.com
>> 
>> ---
>> v3:
>> * Move obj-y += domain-builder/
>> * Remove blank line in Makefile
>> * const in has_hyperlaunch_fdt()
>> * CONFIG_LIBFDT rename
>> * Use boot_info forward declaration
>> * Rename domainbuilder.h to domain-builder.h
>> * Add fdt NULL check
>> ---
>> xen/arch/x86/Makefile | 1 +
>> xen/arch/x86/domain-builder/Kconfig | 2 +-
>> xen/arch/x86/domain-builder/Makefile | 2 +
>> xen/arch/x86/domain-builder/core.c | 57 +++++++++++++++++++++++
>> xen/arch/x86/domain-builder/fdt.c | 37 +++++++++++++++
>> xen/arch/x86/domain-builder/fdt.h | 21 +++++++++
>
> I have a general question.
>
> Wouldn't that make sense to use arch-independent placeholder for domain builder
> code right from the starting point?
>
> For example something like xen/common/domain-builder ?
>
> My understanding is that there's a lot of code in the domain builder which
> can be potentially shared/re-used with non-x86 architectures.
>
> Also, that seems to align with the intention to have common arch-independent
> subsystems in the code base:
>
>    https://docs.google.com/presentation/d/1q9cjJZLUxUo1YzMpCxVHuL-ZOGoFaO9haHfRBK4i4Uc/edit?slide=id.g32afc87aef4_0_18#slide=id.g32afc87aef4_0_18

  https://lore.kernel.org/xen-devel/D92Z1LECR2VA.3FW5H7ZIRVXU7@amd.com/

In principle, yes. I'll try to move it, but I suspect making it coexist
with dom0less will prove tricky.

>> diff --git a/xen/arch/x86/domain-builder/core.c b/xen/arch/x86/domain-builder/core.c
>> new file mode 100644
>> index 0000000000..d6ae94f45c
>> --- /dev/null
>> +++ b/xen/arch/x86/domain-builder/core.c
>> @@ -0,0 +1,57 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only /
>> +/
>> + * Copyright (C) 2024, Apertus Solutions, LLC
>> + */
>> +#include <xen/err.h>
>> 
>> +#include <xen/init.h>
>> 
>> +#include <xen/kconfig.h>
>> 
>> +#include <xen/lib.h>
>> 
>> +
>> +#include <asm/bootinfo.h>
>> 
>> +
>> +#include "fdt.h"
>> +
>> +void __init builder_init(struct boot_info *bi)
>> +{
>> + if ( IS_ENABLED(CONFIG_DOMAIN_BUILDER) )
>> + {
>> + int ret;
>> +
>> + switch ( ret = has_hyperlaunch_fdt(bi) )
>> + {
>> + case 0:
>> + printk("Hyperlaunch device tree detected\n");
>> + bi->hyperlaunch_enabled = true;

FYI, your email client seems to be collapsing the indentation of the
quoted patches. Is that intentional?

Cheers,
Alejandro

Re: [PATCH v3 06/16] x86/hyperlaunch: introduce the domain builder

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> --- a/xen/arch/x86/Makefile
> +++ b/xen/arch/x86/Makefile
> @@ -1,6 +1,7 @@
>  obj-y += acpi/
>  obj-y += boot/
>  obj-y += cpu/
> +obj-y += domain-builder/

I continue to be irritated that this isn't obj-$(DOMAIN_BUILDER). The
sole function in core.c has "if ( IS_ENABLED(CONFIG_DOMAIN_BUILDER) )" around
its entire body (which is unhelpful at least from an indentation pov), and
the sole function in fdt.c is only used by the one in core.c. builder_init()
clearly could have an inline stub alternative as of this patch. If future
patches change that picture, imo that would need saying here to justify this
unconditional descend into the new subdir.

> --- a/xen/arch/x86/domain-builder/Kconfig
> +++ b/xen/arch/x86/domain-builder/Kconfig
> @@ -3,7 +3,7 @@ menu "Domain Builder Features"
>  
>  config DOMAIN_BUILDER
>  	bool "Domain builder (UNSUPPORTED)" if UNSUPPORTED
> -	select LIB_DEVICE_TREE
> +	select LIBFDT

That's what the earlier patch meant to be doing?

> --- /dev/null
> +++ b/xen/arch/x86/domain-builder/Makefile
> @@ -0,0 +1,2 @@
> +obj-$(CONFIG_DOMAIN_BUILDER) += fdt.init.o
> +obj-y += core.init.o

*.init.o want enlisting into obj-bin-y.

> --- /dev/null
> +++ b/xen/arch/x86/domain-builder/core.c
> @@ -0,0 +1,57 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2024, Apertus Solutions, LLC
> + */
> +#include <xen/err.h>
> +#include <xen/init.h>
> +#include <xen/kconfig.h>
> +#include <xen/lib.h>
> +
> +#include <asm/bootinfo.h>
> +
> +#include "fdt.h"
> +
> +void __init builder_init(struct boot_info *bi)
> +{
> +    if ( IS_ENABLED(CONFIG_DOMAIN_BUILDER) )
> +    {
> +        int ret;
> +
> +        switch ( ret = has_hyperlaunch_fdt(bi) )
> +        {
> +        case 0:
> +            printk("Hyperlaunch device tree detected\n");
> +            bi->hyperlaunch_enabled = true;
> +            bi->mods[0].type = BOOTMOD_FDT;
> +            break;
> +
> +        case -EINVAL:
> +            printk("Hyperlaunch device tree was not detected\n");
> +            bi->hyperlaunch_enabled = false;
> +            break;

For people not using hyperlaunch (which for the time being is likely going
to be a majority) this log line will be little more than spam. I'd like it
to be considered to be dropped.

> --- /dev/null
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -0,0 +1,37 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2024, Apertus Solutions, LLC
> + */
> +#include <xen/err.h>

Was xen/errno.h meant here?

> +#include <xen/init.h>
> +#include <xen/lib.h>

I don't see anything used from there.

> +#include <xen/libfdt/libfdt.h>
> +
> +#include <asm/bootinfo.h>
> +#include <asm/page.h>

How does page.h come into play here?

> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -36,6 +36,7 @@
>  #include <asm/bzimage.h>
>  #include <asm/cpu-policy.h>
>  #include <asm/desc.h>
> +#include <asm/domain-builder.h>
>  #include <asm/e820.h>
>  #include <asm/edd.h>
>  #include <asm/genapic.h>
> @@ -1281,9 +1282,12 @@ void asmlinkage __init noreturn __start_xen(void)
>                 bi->nr_modules);
>      }
>  
> -    /* Dom0 kernel is always first */
> -    bi->mods[0].type = BOOTMOD_KERNEL;
> -    bi->domains[0].kernel = &bi->mods[0];
> +    builder_init(bi);
> +
> +    /* Find first unknown boot module to use as Dom0 kernel */
> +    i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);

For this I think it would be quite desirable if in a prereq patch i's type
was (finally) changed to unsigned int, which is how it's used effectively
everywhere, with one loop requiring a little bit of adjustment.

> +    bi->mods[i].type = BOOTMOD_KERNEL;

Overrunning the array if MAX_NR_BOOTMODS + 1 was returned.

Jan

[PATCH v3 07/16] x86/hyperlaunch: initial support for hyperlaunch device tree

[Alejandro Vallejo]

From: "Daniel P. Smith" <dpsmith@apertussolutions.com>

Add the ability to detect both a formal hyperlaunch device tree or a dom0less
device tree. If the hyperlaunch device tree is found, then count the number of
domain entries, reporting an error if more than one is found.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
---
const not added to walk_hyperlaunch_fdt bi since it is updated while
walking the device tree.

v3:
    * Add a blank between declaration and statement
    * Add const to void *fdt
---
 xen/arch/x86/domain-builder/core.c  | 15 +++++++
 xen/arch/x86/domain-builder/fdt.c   | 64 +++++++++++++++++++++++++++++
 xen/arch/x86/domain-builder/fdt.h   |  5 +++
 xen/arch/x86/include/asm/bootinfo.h |  1 +
 4 files changed, 85 insertions(+)

diff --git a/xen/arch/x86/domain-builder/core.c b/xen/arch/x86/domain-builder/core.c
index d6ae94f45c..c50eff34fb 100644
--- a/xen/arch/x86/domain-builder/core.c
+++ b/xen/arch/x86/domain-builder/core.c
@@ -44,6 +44,21 @@ void __init builder_init(struct boot_info *bi)
             break;
         }
     }
+
+    if ( bi->hyperlaunch_enabled )
+    {
+        int ret;
+
+        printk(XENLOG_INFO "Hyperlaunch configuration:\n");
+        if ( (ret = walk_hyperlaunch_fdt(bi)) < 0 )
+        {
+            printk(XENLOG_INFO "  walk of device tree failed (%d)\n", ret);
+            bi->hyperlaunch_enabled = false;
+            return;
+        }
+
+        printk(XENLOG_INFO "  Number of domains: %d\n", bi->nr_domains);
+    }
 }
 
 /*
diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
index aaf8c1cc16..4c5b7747f5 100644
--- a/xen/arch/x86/domain-builder/fdt.c
+++ b/xen/arch/x86/domain-builder/fdt.c
@@ -13,6 +13,36 @@
 
 #include "fdt.h"
 
+static int __init find_hyperlaunch_node(const void *fdt)
+{
+    int hv_node = fdt_path_offset(fdt, "/chosen/hypervisor");
+
+    if ( hv_node >= 0 )
+    {
+        /* Anything other than zero indicates no match */
+        if ( fdt_node_check_compatible(fdt, hv_node, "hypervisor,xen") )
+            return -ENODATA;
+        else
+            return hv_node;
+    }
+    else
+    {
+        /* Look for dom0less config */
+        int node, chosen_node = fdt_path_offset(fdt, "/chosen");
+
+        if ( chosen_node < 0 )
+            return -ENOENT;
+
+        fdt_for_each_subnode(node, fdt, chosen_node)
+        {
+            if ( fdt_node_check_compatible(fdt, node, "xen,domain") == 0 )
+                return chosen_node;
+        }
+    }
+
+    return -ENODATA;
+}
+
 int __init has_hyperlaunch_fdt(const struct boot_info *bi)
 {
     int ret = 0;
@@ -20,7 +50,41 @@ int __init has_hyperlaunch_fdt(const struct boot_info *bi)
 
     if ( !fdt || fdt_check_header(fdt) < 0 )
         ret = -EINVAL;
+    else
+        ret = find_hyperlaunch_node(fdt);
+
+    bootstrap_unmap();
+
+    return ret < 0 ? ret : 0;
+}
+
+int __init walk_hyperlaunch_fdt(struct boot_info *bi)
+{
+    int ret = 0, hv_node, node;
+    const void *fdt = bootstrap_map_bm(&bi->mods[HYPERLAUNCH_MODULE_IDX]);
+
+    if ( unlikely(!fdt) )
+        return -EINVAL;
+
+    hv_node = find_hyperlaunch_node(fdt);
+    if ( hv_node < 0 )
+    {
+        ret = hv_node;
+        goto err_out;
+    }
+
+    fdt_for_each_subnode(node, fdt, hv_node)
+    {
+        ret = fdt_node_check_compatible(fdt, node, "xen,domain");
+        if ( ret == 0 )
+            bi->nr_domains++;
+    }
+
+    /* Until multi-domain construction is added, throw an error */
+    if ( !bi->nr_domains || bi->nr_domains > 1 )
+        printk(XENLOG_ERR "Hyperlaunch only supports dom0 construction\n");
 
+ err_out:
     bootstrap_unmap();
 
     return ret;
diff --git a/xen/arch/x86/domain-builder/fdt.h b/xen/arch/x86/domain-builder/fdt.h
index 8e62cd843e..1849656571 100644
--- a/xen/arch/x86/domain-builder/fdt.h
+++ b/xen/arch/x86/domain-builder/fdt.h
@@ -11,11 +11,16 @@ struct boot_info;
 
 #ifdef CONFIG_DOMAIN_BUILDER
 int has_hyperlaunch_fdt(const struct boot_info *bi);
+int walk_hyperlaunch_fdt(struct boot_info *bi);
 #else
 static inline int __init has_hyperlaunch_fdt(const struct boot_info *bi)
 {
     return -EINVAL;
 }
+static inline int __init walk_hyperlaunch_fdt(struct boot_info *bi)
+{
+    return -EINVAL;
+}
 #endif
 
 #endif /* __XEN_X86_FDT_H__ */
diff --git a/xen/arch/x86/include/asm/bootinfo.h b/xen/arch/x86/include/asm/bootinfo.h
index 82c2650fcf..1e3d582e45 100644
--- a/xen/arch/x86/include/asm/bootinfo.h
+++ b/xen/arch/x86/include/asm/bootinfo.h
@@ -84,6 +84,7 @@ struct boot_info {
     bool hyperlaunch_enabled;
 
     unsigned int nr_modules;
+    unsigned int nr_domains;
     struct boot_module mods[MAX_NR_BOOTMODS + 1];
     struct boot_domain domains[MAX_NR_BOOTDOMS];
 };
-- 
2.43.0

Re: [PATCH v3 07/16] x86/hyperlaunch: initial support for hyperlaunch device tree

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> --- a/xen/arch/x86/domain-builder/core.c
> +++ b/xen/arch/x86/domain-builder/core.c
> @@ -44,6 +44,21 @@ void __init builder_init(struct boot_info *bi)
>              break;
>          }
>      }
> +
> +    if ( bi->hyperlaunch_enabled )
> +    {

Not knowing what else if going to appear here and in what shape, could the
if() here be avoided by making case blocks in the earlier switch setting
the field to false (which is kind of redundant anyway with it starting out
false) use "return" instead of "break"? The the setting of the field to
true could also be centralized below the switch().

> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -13,6 +13,36 @@
>  
>  #include "fdt.h"
>  
> +static int __init find_hyperlaunch_node(const void *fdt)
> +{
> +    int hv_node = fdt_path_offset(fdt, "/chosen/hypervisor");
> +
> +    if ( hv_node >= 0 )
> +    {
> +        /* Anything other than zero indicates no match */
> +        if ( fdt_node_check_compatible(fdt, hv_node, "hypervisor,xen") )
> +            return -ENODATA;
> +        else
> +            return hv_node;

Could I talk you into omitting such unnecessary "else"?

> @@ -20,7 +50,41 @@ int __init has_hyperlaunch_fdt(const struct boot_info *bi)
>  
>      if ( !fdt || fdt_check_header(fdt) < 0 )
>          ret = -EINVAL;
> +    else
> +        ret = find_hyperlaunch_node(fdt);
> +
> +    bootstrap_unmap();
> +
> +    return ret < 0 ? ret : 0;
> +}
> +
> +int __init walk_hyperlaunch_fdt(struct boot_info *bi)
> +{
> +    int ret = 0, hv_node, node;
> +    const void *fdt = bootstrap_map_bm(&bi->mods[HYPERLAUNCH_MODULE_IDX]);
> +
> +    if ( unlikely(!fdt) )
> +        return -EINVAL;
> +
> +    hv_node = find_hyperlaunch_node(fdt);
> +    if ( hv_node < 0 )
> +    {
> +        ret = hv_node;
> +        goto err_out;
> +    }
> +
> +    fdt_for_each_subnode(node, fdt, hv_node)
> +    {
> +        ret = fdt_node_check_compatible(fdt, node, "xen,domain");
> +        if ( ret == 0 )
> +            bi->nr_domains++;
> +    }
> +
> +    /* Until multi-domain construction is added, throw an error */
> +    if ( !bi->nr_domains || bi->nr_domains > 1 )

Simply "!= 1"?

> --- a/xen/arch/x86/domain-builder/fdt.h
> +++ b/xen/arch/x86/domain-builder/fdt.h
> @@ -11,11 +11,16 @@ struct boot_info;
>  
>  #ifdef CONFIG_DOMAIN_BUILDER
>  int has_hyperlaunch_fdt(const struct boot_info *bi);
> +int walk_hyperlaunch_fdt(struct boot_info *bi);
>  #else
>  static inline int __init has_hyperlaunch_fdt(const struct boot_info *bi)
>  {
>      return -EINVAL;
>  }
> +static inline int __init walk_hyperlaunch_fdt(struct boot_info *bi)
> +{
> +    return -EINVAL;
> +}

There's no need for this stub (nor the has_hyperlaunch_fdt() one, as I
notice only now) - even with present arrangements calling code is guarded
such that DCE will take care of eliminating the call, and hence having a
declaration suffices.

Jan

Re: [PATCH v3 07/16] x86/hyperlaunch: initial support for hyperlaunch device tree

[Alejandro Vallejo]

On Thu Apr 10, 2025 at 11:10 AM BST, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> --- a/xen/arch/x86/domain-builder/core.c
>> +++ b/xen/arch/x86/domain-builder/core.c
>> @@ -44,6 +44,21 @@ void __init builder_init(struct boot_info *bi)
>>              break;
>>          }
>>      }
>> +
>> +    if ( bi->hyperlaunch_enabled )
>> +    {
>
> Not knowing what else if going to appear here and in what shape, could the
> if() here be avoided by making case blocks in the earlier switch setting
> the field to false (which is kind of redundant anyway with it starting out
> false) use "return" instead of "break"? The the setting of the field to
> true could also be centralized below the switch().

Looking ahead in the patchlist, not much. There's an else clause for
non-hyperlaunch with code picked from setup.c . It could very well stay
there and prevent core.c from being included at all, removing the if
guards here as well when the file is no longer included.

>
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -13,6 +13,36 @@
>>  
>>  #include "fdt.h"
>>  
>> +static int __init find_hyperlaunch_node(const void *fdt)
>> +{
>> +    int hv_node = fdt_path_offset(fdt, "/chosen/hypervisor");
>> +
>> +    if ( hv_node >= 0 )
>> +    {
>> +        /* Anything other than zero indicates no match */
>> +        if ( fdt_node_check_compatible(fdt, hv_node, "hypervisor,xen") )
>> +            return -ENODATA;
>> +        else
>> +            return hv_node;
>
> Could I talk you into omitting such unnecessary "else"?

Yes, sure.

>
>> @@ -20,7 +50,41 @@ int __init has_hyperlaunch_fdt(const struct boot_info *bi)
>>  
>>      if ( !fdt || fdt_check_header(fdt) < 0 )
>>          ret = -EINVAL;
>> +    else
>> +        ret = find_hyperlaunch_node(fdt);
>> +
>> +    bootstrap_unmap();
>> +
>> +    return ret < 0 ? ret : 0;
>> +}
>> +
>> +int __init walk_hyperlaunch_fdt(struct boot_info *bi)
>> +{
>> +    int ret = 0, hv_node, node;
>> +    const void *fdt = bootstrap_map_bm(&bi->mods[HYPERLAUNCH_MODULE_IDX]);
>> +
>> +    if ( unlikely(!fdt) )
>> +        return -EINVAL;
>> +
>> +    hv_node = find_hyperlaunch_node(fdt);
>> +    if ( hv_node < 0 )
>> +    {
>> +        ret = hv_node;
>> +        goto err_out;
>> +    }
>> +
>> +    fdt_for_each_subnode(node, fdt, hv_node)
>> +    {
>> +        ret = fdt_node_check_compatible(fdt, node, "xen,domain");
>> +        if ( ret == 0 )
>> +            bi->nr_domains++;
>> +    }
>> +
>> +    /* Until multi-domain construction is added, throw an error */
>> +    if ( !bi->nr_domains || bi->nr_domains > 1 )
>
> Simply "!= 1"?

That would be simpler, yes :)

It's all temporary until the restriction is lifted later on, but will do.

>
>> --- a/xen/arch/x86/domain-builder/fdt.h
>> +++ b/xen/arch/x86/domain-builder/fdt.h
>> @@ -11,11 +11,16 @@ struct boot_info;
>>  
>>  #ifdef CONFIG_DOMAIN_BUILDER
>>  int has_hyperlaunch_fdt(const struct boot_info *bi);
>> +int walk_hyperlaunch_fdt(struct boot_info *bi);
>>  #else
>>  static inline int __init has_hyperlaunch_fdt(const struct boot_info *bi)
>>  {
>>      return -EINVAL;
>>  }
>> +static inline int __init walk_hyperlaunch_fdt(struct boot_info *bi)
>> +{
>> +    return -EINVAL;
>> +}
>
> There's no need for this stub (nor the has_hyperlaunch_fdt() one, as I
> notice only now) - even with present arrangements calling code is guarded
> such that DCE will take care of eliminating the call, and hence having a
> declaration suffices.
>
> Jan

There's some refactoring to do for other helpers later on, but sure. It
should be fine.

Cheers,
Alejandro

[PATCH v3 08/16] x86/hyperlaunch: Add helpers to locate multiboot modules

[Alejandro Vallejo]

Hyperlaunch mandates either a reg or module-index DT prop on nodes that
contain `multiboot,module" under their "compatible" prop. This patch
introduces a helper to generically find such index, appending the module
to the list of modules if it wasn't already (i.e: because it's given via
the "reg" prop).

Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
Signed-off-by: Alejandro Vallejo <agarciav@amd.com>
---
v3:
    * New on v3.
    * Subsumes much of the dup code between kernel/initrd patches.
    * Changes previous behaviour in v2 to look into "reg" and
      "module-index" props, rather than just index.
    * Use addr_cells/size_cells rather than size_size
---
 xen/arch/x86/domain-builder/fdt.c   | 142 ++++++++++++++++++++++++++++
 xen/arch/x86/domain-builder/fdt.h   |   2 +
 xen/include/xen/libfdt/libfdt-xen.h |  57 +++++++++++
 3 files changed, 201 insertions(+)

diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
index 4c5b7747f5..9ebc8fd0e4 100644
--- a/xen/arch/x86/domain-builder/fdt.c
+++ b/xen/arch/x86/domain-builder/fdt.c
@@ -13,6 +13,148 @@
 
 #include "fdt.h"
 
+/*
+ * Unpacks a "reg" property into its address and size constituents.
+ *
+ * @param prop          Pointer to an FDT "reg" property.
+ * @param address_cells Number of 4-octet cells that make up an "address".
+ * @param size_cells    Number of 4-octet cells that make up a "size".
+ * @param p_addr[out]   Address encoded in the property.
+ * @param p_size[out]   Size encoded in the property.
+ * @returns             -EINVAL on malformed property, 0 otherwise.
+ */
+static int __init read_fdt_prop_as_reg(const struct fdt_property *prop,
+                                       int address_cells, int size_cells,
+                                       uint64_t *p_addr, uint64_t *p_size)
+{
+    const fdt32_t *cell = (const fdt32_t *)prop->data;
+    uint64_t addr, size;
+
+    if ( fdt32_to_cpu(prop->len) !=
+         (address_cells + size_cells) * sizeof(*cell) )
+    {
+        printk("  Cannot read reg %lu+%lu from prop len %u\n",
+            address_cells * sizeof(*cell), size_cells * sizeof(*cell),
+            fdt32_to_cpu(prop->len));
+        return -EINVAL;
+    }
+
+    switch ( address_cells ) {
+    case 1:
+        addr = fdt32_to_cpu(*cell);
+        break;
+    case 2:
+        addr = fdt64_to_cpu(*(const fdt64_t *)cell);
+        break;
+    default:
+        printk("  unsupported sized address_cells\n");
+        return -EINVAL;
+    }
+
+    cell += address_cells;
+    switch ( size_cells ) {
+    case 1:
+        size = fdt32_to_cpu(*cell);
+        break;
+    case 2:
+        size = fdt64_to_cpu(*(const fdt64_t *)cell);
+        break;
+    default:
+        printk("  unsupported sized size_cells\n");
+        return -EINVAL;
+    }
+
+    *p_addr = addr;
+    *p_size = size;
+
+    return 0;
+}
+
+/*
+ * Locate a multiboot module given its node offset in the FDT.
+ *
+ * The module location may be given via either FDT property:
+ *     * reg = <address, size>
+ *         * Mutates `bi` to append the module.
+ *     * module-index = <idx>
+ *         * Leaves `bi` unchanged.
+ *
+ * @param fdt           Pointer to the full FDT.
+ * @param node          Offset for the module node.
+ * @param address_cells Number of 4-octet cells that make up an "address".
+ * @param size_cells    Number of 4-octet cells that make up a "size".
+ * @param bi[inout]     Xen's representation of the boot parameters.
+ * @return              -EINVAL on malformed nodes, otherwise
+ *                      index inside `bi->mods`
+ */
+int __init fdt_read_multiboot_module(const void *fdt, int node,
+                                     int address_cells, int size_cells,
+                                     struct boot_info *bi)
+{
+    const struct fdt_property *prop;
+    uint64_t addr, size;
+    int ret;
+    int idx;
+
+    ASSERT(!fdt_node_check_compatible(fdt, node, "multiboot,module"));
+
+    /* Location given as a `module-index` property. */
+    prop = fdt_get_property(fdt, node, "module-index", NULL);
+
+    if ( prop )
+    {
+        if ( fdt_get_property(fdt, node, "reg", NULL) )
+        {
+            printk("  Location of multiboot,module defined multiple times\n");
+            return -EINVAL;
+        }
+        return fdt_cell_as_u32((const fdt32_t *)prop->data);
+    }
+
+    /* Otherwise location given as a `reg` property. */
+    prop = fdt_get_property(fdt, node, "reg", NULL);
+
+    if ( !prop )
+    {
+        printk("  No location for multiboot,module\n");
+        return -EINVAL;
+    }
+    if ( fdt_get_property(fdt, node, "module-index", NULL) )
+    {
+        printk("  Location of multiboot,module defined multiple times\n");
+        return -EINVAL;
+    }
+
+    ret = read_fdt_prop_as_reg(prop, address_cells, size_cells, &addr, &size);
+
+    if ( ret < 0 )
+    {
+        printk("  Failed reading reg for multiboot,module\n");
+        return -EINVAL;
+    }
+
+    idx = bi->nr_modules + 1;
+    if ( idx > MAX_NR_BOOTMODS )
+    {
+        /*
+         * MAX_NR_BOOTMODS cannot exceed the max for MB1, represented by 32bits,
+         * thus the cast down to a u32 will be safe due to the prior check.
+         */
+        BUILD_BUG_ON(MAX_NR_BOOTMODS >= (uint64_t)UINT32_MAX);
+        printk("  idx %d exceeds maximum boot modules\n", idx);
+        return -EINVAL;
+    }
+
+    /* Append new module to the existing list */
+
+    bi->nr_modules = idx;
+    bi->mods[idx].start = addr;
+    bi->mods[idx].size = size;
+    printk("  module[%d]: addr %lx size %lx\n", idx, addr, size);
+
+    return idx;
+}
+
 static int __init find_hyperlaunch_node(const void *fdt)
 {
     int hv_node = fdt_path_offset(fdt, "/chosen/hypervisor");
diff --git a/xen/arch/x86/domain-builder/fdt.h b/xen/arch/x86/domain-builder/fdt.h
index 1849656571..e8769dc51c 100644
--- a/xen/arch/x86/domain-builder/fdt.h
+++ b/xen/arch/x86/domain-builder/fdt.h
@@ -3,6 +3,8 @@
 #define __XEN_X86_FDT_H__
 
 #include <xen/init.h>
+#include <xen/libfdt/libfdt.h>
+#include <xen/libfdt/libfdt-xen.h>
 
 struct boot_info;
 
diff --git a/xen/include/xen/libfdt/libfdt-xen.h b/xen/include/xen/libfdt/libfdt-xen.h
index a5340bc9f4..2259c09a6a 100644
--- a/xen/include/xen/libfdt/libfdt-xen.h
+++ b/xen/include/xen/libfdt/libfdt-xen.h
@@ -13,6 +13,63 @@
 
 #include <xen/libfdt/libfdt.h>
 
+static inline int __init fdt_cell_as_u32(const fdt32_t *cell)
+{
+    return fdt32_to_cpu(*cell);
+}
+
+static inline uint64_t  __init fdt_cell_as_u64(const fdt32_t *cell)
+{
+    return ((uint64_t)fdt32_to_cpu(cell[0]) << 32) | fdt32_to_cpu(cell[1]);
+}
+
+/*
+ * Property: reg
+ *
+ * Defined in Section 2.3.6 of the Device Tree Specification is the "reg"
+ * standard property. The property is a prop-encoded-array that is encoded as
+ * an arbitrary number of (address, size) pairs.  We only extract a single
+ * pair since that is what is used in practice.
+ */
+static inline int __init fdt_get_reg_prop(
+    const void *fdt, int node, unsigned int addr_cells, unsigned int size_cells,
+    uint64_t *addr, uint64_t *size)
+{
+    int ret;
+    const struct fdt_property *prop;
+    fdt32_t *cell;
+
+    /* FDT spec max size is 4 (128bit int), but largest arch int size is 64 */
+    if ( size_cells > 2 || addr_cells > 2 )
+        return -EINVAL;
+
+    prop = fdt_get_property(fdt, node, "reg", &ret);
+    if ( !prop || ret < sizeof(u32) )
+        return ret < 0 ? ret : -EINVAL;
+
+    if ( fdt32_to_cpu(prop->len) !=
+	 ((size_cells + addr_cells) * sizeof(*cell)) )
+        return -EINVAL;
+
+    cell = (fdt32_t *)prop->data;
+
+    /* read address field */
+    if ( addr_cells == 1 )
+        *addr = fdt_cell_as_u32(cell);
+    else
+        *addr = fdt_cell_as_u64(cell);
+
+    cell += addr_cells;
+
+    /* read size field */
+    if ( size_cells == 1 )
+        *size = fdt_cell_as_u32(cell);
+    else
+        *size = fdt_cell_as_u64(cell);
+
+    return 0;
+}
+
 static inline int fdt_get_mem_rsv_paddr(const void *fdt, int n,
                                         paddr_t *address,
                                         paddr_t *size)
-- 
2.43.0

Re: [PATCH v3 08/16] x86/hyperlaunch: Add helpers to locate multiboot modules

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -13,6 +13,148 @@
>  
>  #include "fdt.h"
>  
> +/*
> + * Unpacks a "reg" property into its address and size constituents.
> + *
> + * @param prop          Pointer to an FDT "reg" property.
> + * @param address_cells Number of 4-octet cells that make up an "address".
> + * @param size_cells    Number of 4-octet cells that make up a "size".
> + * @param p_addr[out]   Address encoded in the property.
> + * @param p_size[out]   Size encoded in the property.
> + * @returns             -EINVAL on malformed property, 0 otherwise.
> + */
> +static int __init read_fdt_prop_as_reg(const struct fdt_property *prop,
> +                                       int address_cells, int size_cells,
> +                                       uint64_t *p_addr, uint64_t *p_size)
> +{
> +    const fdt32_t *cell = (const fdt32_t *)prop->data;
> +    uint64_t addr, size;
> +
> +    if ( fdt32_to_cpu(prop->len) !=
> +         (address_cells + size_cells) * sizeof(*cell) )
> +    {
> +        printk("  Cannot read reg %lu+%lu from prop len %u\n",
> +            address_cells * sizeof(*cell), size_cells * sizeof(*cell),
> +            fdt32_to_cpu(prop->len));
> +        return -EINVAL;
> +    }
> +
> +    switch ( address_cells ) {

Nit: Brace on its own line please.

> +    case 1:
> +        addr = fdt32_to_cpu(*cell);
> +        break;
> +    case 2:
> +        addr = fdt64_to_cpu(*(const fdt64_t *)cell);
> +        break;
> +    default:
> +        printk("  unsupported sized address_cells\n");

Depending on how likely this or ...

> +        return -EINVAL;
> +    }
> +
> +    cell += address_cells;
> +    switch ( size_cells ) {
> +    case 1:
> +        size = fdt32_to_cpu(*cell);
> +        break;
> +    case 2:
> +        size = fdt64_to_cpu(*(const fdt64_t *)cell);
> +        break;
> +    default:
> +        printk("  unsupported sized size_cells\n");

... this path is to be hit, perhaps also log the bogus size? Then again, this
being passed in, isn't it an internal error if the wrong size makes it here?
I.e. rather use ASSERT_UNREACHABLE()?

> +        return -EINVAL;
> +    }
> +
> +    *p_addr = addr;
> +    *p_size = size;
> +
> +    return 0;
> +}

The function as a whole looks somewhat similar to fdt_get_reg_prop(). What's
the deal?

> +/*
> + * Locate a multiboot module given its node offset in the FDT.
> + *
> + * The module location may be given via either FDT property:
> + *     * reg = <address, size>
> + *         * Mutates `bi` to append the module.
> + *     * module-index = <idx>
> + *         * Leaves `bi` unchanged.
> + *
> + * @param fdt           Pointer to the full FDT.
> + * @param node          Offset for the module node.
> + * @param address_cells Number of 4-octet cells that make up an "address".
> + * @param size_cells    Number of 4-octet cells that make up a "size".
> + * @param bi[inout]     Xen's representation of the boot parameters.
> + * @return              -EINVAL on malformed nodes, otherwise
> + *                      index inside `bi->mods`
> + */
> +int __init fdt_read_multiboot_module(const void *fdt, int node,
> +                                     int address_cells, int size_cells,
> +                                     struct boot_info *bi)

Functions without callers and non-static ones without declarations are
disliked by Misra.

> +{
> +    const struct fdt_property *prop;
> +    uint64_t addr, size;
> +    int ret;
> +    int idx;
> +
> +    ASSERT(!fdt_node_check_compatible(fdt, node, "multiboot,module"));
> +
> +    /* Location given as a `module-index` property. */
> +    prop = fdt_get_property(fdt, node, "module-index", NULL);
> +
> +    if ( prop )
> +    {
> +        if ( fdt_get_property(fdt, node, "reg", NULL) )
> +        {
> +            printk("  Location of multiboot,module defined multiple times\n");
> +            return -EINVAL;
> +        }
> +        return fdt_cell_as_u32((const fdt32_t *)prop->data);

No concerns here of there being less than 4 bytes of data?

> +    }
> +
> +    /* Otherwise location given as a `reg` property. */
> +    prop = fdt_get_property(fdt, node, "reg", NULL);
> +
> +    if ( !prop )
> +    {
> +        printk("  No location for multiboot,module\n");
> +        return -EINVAL;
> +    }
> +    if ( fdt_get_property(fdt, node, "module-index", NULL) )
> +    {
> +        printk("  Location of multiboot,module defined multiple times\n");
> +        return -EINVAL;
> +    }
> +
> +    ret = read_fdt_prop_as_reg(prop, address_cells, size_cells, &addr, &size);
> +
> +    if ( ret < 0 )
> +    {
> +        printk("  Failed reading reg for multiboot,module\n");
> +        return -EINVAL;
> +    }
> +
> +    idx = bi->nr_modules + 1;

This at least looks like an off-by-one. If the addition of 1 is really
intended, I think it needs commenting on.

> +    if ( idx > MAX_NR_BOOTMODS )
> +    {
> +        /*
> +         * MAX_NR_BOOTMODS cannot exceed the max for MB1, represented by 32bits,
> +         * thus the cast down to a u32 will be safe due to the prior check.
> +         */
> +        BUILD_BUG_ON(MAX_NR_BOOTMODS >= (uint64_t)UINT32_MAX);

Because of idx being a signed quantity, isn't INT_MAX the required upper
bound? The latest then the somewhat odd cast should also be possible to
drop.

> +        printk("  idx %d exceeds maximum boot modules\n", idx);

Perhaps include STR(MAX_NR_BOOTMODS) as well?

> --- a/xen/include/xen/libfdt/libfdt-xen.h
> +++ b/xen/include/xen/libfdt/libfdt-xen.h
> @@ -13,6 +13,63 @@
>  
>  #include <xen/libfdt/libfdt.h>
>  
> +static inline int __init fdt_cell_as_u32(const fdt32_t *cell)

Why plain int here, but ...

> +{
> +    return fdt32_to_cpu(*cell);
> +}
> +
> +static inline uint64_t  __init fdt_cell_as_u64(const fdt32_t *cell)

... a fixed-width and unsigned type here? Question is whether the former
helper is really warranted.

Also nit: Stray double blank.

> +{
> +    return ((uint64_t)fdt32_to_cpu(cell[0]) << 32) | fdt32_to_cpu(cell[1]);

That is - uniformly big endian?

> +}

Marking such relatively generic inline functions __init is also somewhat
risky. 

> +/*
> + * Property: reg
> + *
> + * Defined in Section 2.3.6 of the Device Tree Specification is the "reg"
> + * standard property. The property is a prop-encoded-array that is encoded as
> + * an arbitrary number of (address, size) pairs.  We only extract a single
> + * pair since that is what is used in practice.
> + */
> +static inline int __init fdt_get_reg_prop(
> +    const void *fdt, int node, unsigned int addr_cells, unsigned int size_cells,
> +    uint64_t *addr, uint64_t *size)
> +{
> +    int ret;
> +    const struct fdt_property *prop;
> +    fdt32_t *cell;
> +
> +    /* FDT spec max size is 4 (128bit int), but largest arch int size is 64 */
> +    if ( size_cells > 2 || addr_cells > 2 )
> +        return -EINVAL;
> +
> +    prop = fdt_get_property(fdt, node, "reg", &ret);
> +    if ( !prop || ret < sizeof(u32) )

No uses of u32 et al in new code please. Question anyway is whether this isn't
meant to be sizeof(*cell) like you have it ...

> +        return ret < 0 ? ret : -EINVAL;
> +
> +    if ( fdt32_to_cpu(prop->len) !=
> +	 ((size_cells + addr_cells) * sizeof(*cell)) )

... here. Or maybe it's to be sizeof(prop->len)?

Also nit: Hard tab slipped in.

> +        return -EINVAL;
> +
> +    cell = (fdt32_t *)prop->data;
> +
> +    /* read address field */
> +    if ( addr_cells == 1 )
> +        *addr = fdt_cell_as_u32(cell);
> +    else
> +        *addr = fdt_cell_as_u64(cell);
> +
> +    cell += addr_cells;
> +
> +    /* read size field */
> +    if ( size_cells == 1 )
> +        *size = fdt_cell_as_u32(cell);
> +    else
> +        *size = fdt_cell_as_u64(cell);
> +
> +    return 0;
> +}

Does this really want/need to be an inline function?

Jan

Re: [PATCH v3 08/16] x86/hyperlaunch: Add helpers to locate multiboot modules

[Alejandro Vallejo]

On Thu Apr 10, 2025 at 11:42 AM BST, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -13,6 +13,148 @@
>>  
>>  #include "fdt.h"
>>  
>> +/*
>> + * Unpacks a "reg" property into its address and size constituents.
>> + *
>> + * @param prop          Pointer to an FDT "reg" property.
>> + * @param address_cells Number of 4-octet cells that make up an "address".
>> + * @param size_cells    Number of 4-octet cells that make up a "size".
>> + * @param p_addr[out]   Address encoded in the property.
>> + * @param p_size[out]   Size encoded in the property.
>> + * @returns             -EINVAL on malformed property, 0 otherwise.
>> + */
>> +static int __init read_fdt_prop_as_reg(const struct fdt_property *prop,
>> +                                       int address_cells, int size_cells,
>> +                                       uint64_t *p_addr, uint64_t *p_size)
>> +{
>> +    const fdt32_t *cell = (const fdt32_t *)prop->data;
>> +    uint64_t addr, size;
>> +
>> +    if ( fdt32_to_cpu(prop->len) !=
>> +         (address_cells + size_cells) * sizeof(*cell) )
>> +    {
>> +        printk("  Cannot read reg %lu+%lu from prop len %u\n",
>> +            address_cells * sizeof(*cell), size_cells * sizeof(*cell),
>> +            fdt32_to_cpu(prop->len));
>> +        return -EINVAL;
>> +    }
>> +
>> +    switch ( address_cells ) {
>
> Nit: Brace on its own line please.

Sure

>
>> +    case 1:
>> +        addr = fdt32_to_cpu(*cell);
>> +        break;
>> +    case 2:
>> +        addr = fdt64_to_cpu(*(const fdt64_t *)cell);
>> +        break;
>> +    default:
>> +        printk("  unsupported sized address_cells\n");
>
> Depending on how likely this or ...
>
>> +        return -EINVAL;
>> +    }
>> +
>> +    cell += address_cells;
>> +    switch ( size_cells ) {
>> +    case 1:
>> +        size = fdt32_to_cpu(*cell);
>> +        break;
>> +    case 2:
>> +        size = fdt64_to_cpu(*(const fdt64_t *)cell);
>> +        break;
>> +    default:
>> +        printk("  unsupported sized size_cells\n");
>
> ... this path is to be hit, perhaps also log the bogus size? Then again, this
> being passed in, isn't it an internal error if the wrong size makes it here?
> I.e. rather use ASSERT_UNREACHABLE()?

*_cells are DTB properties, so it's more of an input error.

Ack to log the sizes, will do.

>
>> +        return -EINVAL;
>> +    }
>> +
>> +    *p_addr = addr;
>> +    *p_size = size;
>> +
>> +    return 0;
>> +}
>
> The function as a whole looks somewhat similar to fdt_get_reg_prop(). What's
> the deal?

The latter shouldn't be there. It's leftover from code motion and a
merge.

>
>> +/*
>> + * Locate a multiboot module given its node offset in the FDT.
>> + *
>> + * The module location may be given via either FDT property:
>> + *     * reg = <address, size>
>> + *         * Mutates `bi` to append the module.
>> + *     * module-index = <idx>
>> + *         * Leaves `bi` unchanged.
>> + *
>> + * @param fdt           Pointer to the full FDT.
>> + * @param node          Offset for the module node.
>> + * @param address_cells Number of 4-octet cells that make up an "address".
>> + * @param size_cells    Number of 4-octet cells that make up a "size".
>> + * @param bi[inout]     Xen's representation of the boot parameters.
>> + * @return              -EINVAL on malformed nodes, otherwise
>> + *                      index inside `bi->mods`
>> + */
>> +int __init fdt_read_multiboot_module(const void *fdt, int node,
>> +                                     int address_cells, int size_cells,
>> +                                     struct boot_info *bi)
>
> Functions without callers and non-static ones without declarations are
> disliked by Misra.

Can't do much about it if I want them to stand alone in a single patch.
Otherwise the following ones become quite unwieldy to look at. All I can
say is that this function becomes static and with a caller on the next
patch.

>
>> +{
>> +    const struct fdt_property *prop;
>> +    uint64_t addr, size;
>> +    int ret;
>> +    int idx;
>> +
>> +    ASSERT(!fdt_node_check_compatible(fdt, node, "multiboot,module"));
>> +
>> +    /* Location given as a `module-index` property. */
>> +    prop = fdt_get_property(fdt, node, "module-index", NULL);
>> +
>> +    if ( prop )
>> +    {
>> +        if ( fdt_get_property(fdt, node, "reg", NULL) )
>> +        {
>> +            printk("  Location of multiboot,module defined multiple times\n");
>> +            return -EINVAL;
>> +        }
>> +        return fdt_cell_as_u32((const fdt32_t *)prop->data);
>
> No concerns here of there being less than 4 bytes of data?

v4 moves the property accessors earlier so this is a safe access.

>
>> +    }
>> +
>> +    /* Otherwise location given as a `reg` property. */
>> +    prop = fdt_get_property(fdt, node, "reg", NULL);
>> +
>> +    if ( !prop )
>> +    {
>> +        printk("  No location for multiboot,module\n");
>> +        return -EINVAL;
>> +    }
>> +    if ( fdt_get_property(fdt, node, "module-index", NULL) )
>> +    {
>> +        printk("  Location of multiboot,module defined multiple times\n");
>> +        return -EINVAL;
>> +    }
>> +
>> +    ret = read_fdt_prop_as_reg(prop, address_cells, size_cells, &addr, &size);
>> +
>> +    if ( ret < 0 )
>> +    {
>> +        printk("  Failed reading reg for multiboot,module\n");
>> +        return -EINVAL;
>> +    }
>> +
>> +    idx = bi->nr_modules + 1;
>
> This at least looks like an off-by-one. If the addition of 1 is really
> intended, I think it needs commenting on.

Seems to be, yes. The underlying array is a bit bizarre. It's sizes as
MAX_NR_BOOTMODS + 1, with the first one being the DTB itself. I guess
the intent was to take it into account, but bi->nr_modules is
initialised to the number of multiboot modules, so it SHOULD be already
taking it into account.

Also, the logic for bounds checking seems... off (because of the + 1 I
mentioned before). Or at least confusing, so I've moved to using
ARRAY_SIZE(bi->mods) rather than explicitly comparing against
MAX_NR_BOOTMODS.

The array is MAX_NR_BOOTMODS + 1 in length, so it's just more cognitive
load than I'm comfortable with.

>
>> +    if ( idx > MAX_NR_BOOTMODS )
>> +    {
>> +        /*
>> +         * MAX_NR_BOOTMODS cannot exceed the max for MB1, represented by 32bits,
>> +         * thus the cast down to a u32 will be safe due to the prior check.
>> +         */
>> +        BUILD_BUG_ON(MAX_NR_BOOTMODS >= (uint64_t)UINT32_MAX);
>
> Because of idx being a signed quantity, isn't INT_MAX the required upper
> bound? The latest then the somewhat odd cast should also be possible to
> drop.

It is, yes. Having a theoretical limit of 2**31-1 rather than 2**32-1 doesn't worry
me in the slightest.

>
>> +        printk("  idx %d exceeds maximum boot modules\n", idx);
>
> Perhaps include STR(MAX_NR_BOOTMODS) as well?
I'll print ARRAY_SIZE(bi->mods) instead. Otherwise it will be very
confusing.

>
>> --- a/xen/include/xen/libfdt/libfdt-xen.h
>> +++ b/xen/include/xen/libfdt/libfdt-xen.h
>> @@ -13,6 +13,63 @@kkk
>>  
>>  #include <xen/libfdt/libfdt.h>
>>  
>> +static inline int __init fdt_cell_as_u32(const fdt32_t *cell)
>
> Why plain int here, but ...
>
>> +{
>> +    return fdt32_to_cpu(*cell);
>> +}
>> +
>> +static inline uint64_t  __init fdt_cell_as_u64(const fdt32_t *cell)
>
> ... a fixed-width and unsigned type here? Question is whether the former
> helper is really warranted.
>
> Also nit: Stray double blank.
>
>> +{
>> +    return ((uint64_t)fdt32_to_cpu(cell[0]) << 32) | fdt32_to_cpu(cell[1]);
>
> That is - uniformly big endian?

These helpers are disappearing, so it doesn't matter. This is basically
an open coded:

  fdt64_to_cpu(*(const fdt64_t *)fdt32)

And, yes. DTBs are standardised as having big-endian properties, for
better or worse :/

>
>> +}
>
> Marking such relatively generic inline functions __init is also somewhat
> risky. 

They were originally in domain-builder/fdt.c and moved here as a result
of a request to have them on libfdt. libfdt proved to be somewhat
annoying because it would be hard to distinguish accessors for the
flattened and the unflattened tree.

I'd personally have them in domain-builder instead, where they are used.
Should they be needed somewhere else, we can always fator them out
somewhere else.

Thoughts?

>
>> +/*
>> + * Property: reg
>> + *
>> + * Defined in Section 2.3.6 of the Device Tree Specification is the "reg"
>> + * standard property. The property is a prop-encoded-array that is encoded as
>> + * an arbitrary number of (address, size) pairs.  We only extract a single
>> + * pair since that is what is used in practice.
>> + */
>> +static inline int __init fdt_get_reg_prop(
>> +    const void *fdt, int node, unsigned int addr_cells, unsigned int size_cells,
>> +    uint64_t *addr, uint64_t *size)
>> +{
>> +    int ret;
>> +    const struct fdt_property *prop;
>> +    fdt32_t *cell;
>> +
>> +    /* FDT spec max size is 4 (128bit int), but largest arch int size is 64 */
>> +    if ( size_cells > 2 || addr_cells > 2 )
>> +        return -EINVAL;
>> +
>> +    prop = fdt_get_property(fdt, node, "reg", &ret);
>> +    if ( !prop || ret < sizeof(u32) )
>
> No uses of u32 et al in new code please. Question anyway is whether this isn't
> meant to be sizeof(*cell) like you have it ...
>
>> +        return ret < 0 ? ret : -EINVAL;
>> +
>> +    if ( fdt32_to_cpu(prop->len) !=
>> +	 ((size_cells + addr_cells) * sizeof(*cell)) )
>
> ... here. Or maybe it's to be sizeof(prop->len)?
>
> Also nit: Hard tab slipped in.
>
>> +        return -EINVAL;
>> +
>> +    cell = (fdt32_t *)prop->data;
>> +
>> +    /* read address field */
>> +    if ( addr_cells == 1 )
>> +        *addr = fdt_cell_as_u32(cell);
>> +    else
>> +        *addr = fdt_cell_as_u64(cell);
>> +
>> +    cell += addr_cells;
>> +
>> +    /* read size field */
>> +    if ( size_cells == 1 )
>> +        *size = fdt_cell_as_u32(cell);
>> +    else
>> +        *size = fdt_cell_as_u64(cell);
>> +
>> +    return 0;
>> +}
>
> Does this really want/need to be an inline function?
>
> Jan

This function is gone in v4.

Cheers,
Alejandro

Re: [PATCH v3 08/16] x86/hyperlaunch: Add helpers to locate multiboot modules

[Jan Beulich]

On 14.04.2025 15:37, Alejandro Vallejo wrote:
> On Thu Apr 10, 2025 at 11:42 AM BST, Jan Beulich wrote:
>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>> +/*
>>> + * Locate a multiboot module given its node offset in the FDT.
>>> + *
>>> + * The module location may be given via either FDT property:
>>> + *     * reg = <address, size>
>>> + *         * Mutates `bi` to append the module.
>>> + *     * module-index = <idx>
>>> + *         * Leaves `bi` unchanged.
>>> + *
>>> + * @param fdt           Pointer to the full FDT.
>>> + * @param node          Offset for the module node.
>>> + * @param address_cells Number of 4-octet cells that make up an "address".
>>> + * @param size_cells    Number of 4-octet cells that make up a "size".
>>> + * @param bi[inout]     Xen's representation of the boot parameters.
>>> + * @return              -EINVAL on malformed nodes, otherwise
>>> + *                      index inside `bi->mods`
>>> + */
>>> +int __init fdt_read_multiboot_module(const void *fdt, int node,
>>> +                                     int address_cells, int size_cells,
>>> +                                     struct boot_info *bi)
>>
>> Functions without callers and non-static ones without declarations are
>> disliked by Misra.
> 
> Can't do much about it if I want them to stand alone in a single patch.
> Otherwise the following ones become quite unwieldy to look at. All I can
> say is that this function becomes static and with a caller on the next
> patch.

Which means you need to touch this again anyway. Perhaps we need a Misra
deviation for __maybe_unused functions / data, in which case you could
use that here and strip it along with making the function static. Cc-ing
Bugseng folks.

>>> +    /* Otherwise location given as a `reg` property. */
>>> +    prop = fdt_get_property(fdt, node, "reg", NULL);
>>> +
>>> +    if ( !prop )
>>> +    {
>>> +        printk("  No location for multiboot,module\n");
>>> +        return -EINVAL;
>>> +    }
>>> +    if ( fdt_get_property(fdt, node, "module-index", NULL) )
>>> +    {
>>> +        printk("  Location of multiboot,module defined multiple times\n");
>>> +        return -EINVAL;
>>> +    }
>>> +
>>> +    ret = read_fdt_prop_as_reg(prop, address_cells, size_cells, &addr, &size);
>>> +
>>> +    if ( ret < 0 )
>>> +    {
>>> +        printk("  Failed reading reg for multiboot,module\n");
>>> +        return -EINVAL;
>>> +    }
>>> +
>>> +    idx = bi->nr_modules + 1;
>>
>> This at least looks like an off-by-one. If the addition of 1 is really
>> intended, I think it needs commenting on.
> 
> Seems to be, yes. The underlying array is a bit bizarre. It's sizes as
> MAX_NR_BOOTMODS + 1, with the first one being the DTB itself. I guess
> the intent was to take it into account, but bi->nr_modules is
> initialised to the number of multiboot modules, so it SHOULD be already
> taking it into account.
> 
> Also, the logic for bounds checking seems... off (because of the + 1 I
> mentioned before). Or at least confusing, so I've moved to using
> ARRAY_SIZE(bi->mods) rather than explicitly comparing against
> MAX_NR_BOOTMODS.
> 
> The array is MAX_NR_BOOTMODS + 1 in length, so it's just more cognitive
> load than I'm comfortable with.

If I'm not mistaken the +1 is inherited from the modules array we had in
the past, where we wanted 1 extra slot for Xen itself. Hence before you
move to using ARRAY_SIZE() everywhere it needs to really be clear what
the +1 here is used for.

>>> --- a/xen/include/xen/libfdt/libfdt-xen.h
>>> +++ b/xen/include/xen/libfdt/libfdt-xen.h
>>> @@ -13,6 +13,63 @@kkk
>>>  
>>>  #include <xen/libfdt/libfdt.h>
>>>  
>>> +static inline int __init fdt_cell_as_u32(const fdt32_t *cell)
>>
>> Why plain int here, but ...
>>
>>> +{
>>> +    return fdt32_to_cpu(*cell);
>>> +}
>>> +
>>> +static inline uint64_t  __init fdt_cell_as_u64(const fdt32_t *cell)
>>
>> ... a fixed-width and unsigned type here? Question is whether the former
>> helper is really warranted.
>>
>> Also nit: Stray double blank.
>>
>>> +{
>>> +    return ((uint64_t)fdt32_to_cpu(cell[0]) << 32) | fdt32_to_cpu(cell[1]);
>>
>> That is - uniformly big endian?
> 
> These helpers are disappearing, so it doesn't matter. This is basically
> an open coded:
> 
>   fdt64_to_cpu(*(const fdt64_t *)fdt32)
> 
> And, yes. DTBs are standardised as having big-endian properties, for
> better or worse :/
> 
>>
>>> +}
>>
>> Marking such relatively generic inline functions __init is also somewhat
>> risky. 
> 
> They were originally in domain-builder/fdt.c and moved here as a result
> of a request to have them on libfdt. libfdt proved to be somewhat
> annoying because it would be hard to distinguish accessors for the
> flattened and the unflattened tree.
> 
> I'd personally have them in domain-builder instead, where they are used.
> Should they be needed somewhere else, we can always fator them out
> somewhere else.
> 
> Thoughts?

As long as they're needed only by domain-builder, it's probably fine to have
them just there.

Jan

Re: [PATCH v3 08/16] x86/hyperlaunch: Add helpers to locate multiboot modules

[Alejandro Vallejo]

On Mon Apr 14, 2025 at 4:05 PM BST, Jan Beulich wrote:
> On 14.04.2025 15:37, Alejandro Vallejo wrote:
>> On Thu Apr 10, 2025 at 11:42 AM BST, Jan Beulich wrote:
>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>> +/*
>>>> + * Locate a multiboot module given its node offset in the FDT.
>>>> + *
>>>> + * The module location may be given via either FDT property:
>>>> + *     * reg = <address, size>
>>>> + *         * Mutates `bi` to append the module.
>>>> + *     * module-index = <idx>
>>>> + *         * Leaves `bi` unchanged.
>>>> + *
>>>> + * @param fdt           Pointer to the full FDT.
>>>> + * @param node          Offset for the module node.
>>>> + * @param address_cells Number of 4-octet cells that make up an "address".
>>>> + * @param size_cells    Number of 4-octet cells that make up a "size".
>>>> + * @param bi[inout]     Xen's representation of the boot parameters.
>>>> + * @return              -EINVAL on malformed nodes, otherwise
>>>> + *                      index inside `bi->mods`
>>>> + */
>>>> +int __init fdt_read_multiboot_module(const void *fdt, int node,
>>>> +                                     int address_cells, int size_cells,
>>>> +                                     struct boot_info *bi)
>>>
>>> Functions without callers and non-static ones without declarations are
>>> disliked by Misra.
>> 
>> Can't do much about it if I want them to stand alone in a single patch.
>> Otherwise the following ones become quite unwieldy to look at. All I can
>> say is that this function becomes static and with a caller on the next
>> patch.
>
> Which means you need to touch this again anyway. Perhaps we need a Misra
> deviation for __maybe_unused functions / data, in which case you could
> use that here and strip it along with making the function static. Cc-ing
> Bugseng folks.

It's a transient violation, sure. Do we care about transient MISRA
violations though? I understand the importance of bisectability, but
AUIU MISRA compliance matters to the extent that that the tip is
compliant rather than the intermediate steps?

Another option would be to fold them this patch and the next together
after both get their R-by. As I said, I assumed you'd rather see them in
isolation for purposes of review.

>
>>>> +    /* Otherwise location given as a `reg` property. */
>>>> +    prop = fdt_get_property(fdt, node, "reg", NULL);
>>>> +
>>>> +    if ( !prop )
>>>> +    {
>>>> +        printk("  No location for multiboot,module\n");
>>>> +        return -EINVAL;
>>>> +    }
>>>> +    if ( fdt_get_property(fdt, node, "module-index", NULL) )
>>>> +    {
>>>> +        printk("  Location of multiboot,module defined multiple times\n");
>>>> +        return -EINVAL;
>>>> +    }
>>>> +
>>>> +    ret = read_fdt_prop_as_reg(prop, address_cells, size_cells, &addr, &size);
>>>> +
>>>> +    if ( ret < 0 )
>>>> +    {
>>>> +        printk("  Failed reading reg for multiboot,module\n");
>>>> +        return -EINVAL;
>>>> +    }
>>>> +
>>>> +    idx = bi->nr_modules + 1;
>>>
>>> This at least looks like an off-by-one. If the addition of 1 is really
>>> intended, I think it needs commenting on.
>> 
>> Seems to be, yes. The underlying array is a bit bizarre. It's sizes as
>> MAX_NR_BOOTMODS + 1, with the first one being the DTB itself. I guess
>> the intent was to take it into account, but bi->nr_modules is
>> initialised to the number of multiboot modules, so it SHOULD be already
>> taking it into account.
>> 
>> Also, the logic for bounds checking seems... off (because of the + 1 I
>> mentioned before). Or at least confusing, so I've moved to using
>> ARRAY_SIZE(bi->mods) rather than explicitly comparing against
>> MAX_NR_BOOTMODS.
>> 
>> The array is MAX_NR_BOOTMODS + 1 in length, so it's just more cognitive
>> load than I'm comfortable with.
>
> If I'm not mistaken the +1 is inherited from the modules array we had in
> the past, where we wanted 1 extra slot for Xen itself. Hence before you
> move to using ARRAY_SIZE() everywhere it needs to really be clear what
> the +1 here is used for.

Ew.  Ok, just looked at the code in multiboot_fill_boot_info and indeed
the arrangement is for all multiboot modules to be in front, and Xen to
be appended. But bi->nr_modules only lists multiboot modules, so
increasing that value is therefore not enough (or
next_boot_module_index() would fail).

I need to have a proper read on how this is all stitched together.  I
may simply swap BOOTMOD_XEN with the next entry on append. Though my
preference would be to _not_ have Xen as part of the module list to
begin with. Before boot_info that was probably a place as good as any,
but this would be much better off in a dedicated field.

I don't see much in terms of usage though. Why is it being added at all?

Cheers,
Alejandro

Re: [PATCH v3 08/16] x86/hyperlaunch: Add helpers to locate multiboot modules

[Jan Beulich]

On 14.04.2025 20:01, Alejandro Vallejo wrote:
> On Mon Apr 14, 2025 at 4:05 PM BST, Jan Beulich wrote:
>> On 14.04.2025 15:37, Alejandro Vallejo wrote:
>>> On Thu Apr 10, 2025 at 11:42 AM BST, Jan Beulich wrote:
>>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>>> +/*
>>>>> + * Locate a multiboot module given its node offset in the FDT.
>>>>> + *
>>>>> + * The module location may be given via either FDT property:
>>>>> + *     * reg = <address, size>
>>>>> + *         * Mutates `bi` to append the module.
>>>>> + *     * module-index = <idx>
>>>>> + *         * Leaves `bi` unchanged.
>>>>> + *
>>>>> + * @param fdt           Pointer to the full FDT.
>>>>> + * @param node          Offset for the module node.
>>>>> + * @param address_cells Number of 4-octet cells that make up an "address".
>>>>> + * @param size_cells    Number of 4-octet cells that make up a "size".
>>>>> + * @param bi[inout]     Xen's representation of the boot parameters.
>>>>> + * @return              -EINVAL on malformed nodes, otherwise
>>>>> + *                      index inside `bi->mods`
>>>>> + */
>>>>> +int __init fdt_read_multiboot_module(const void *fdt, int node,
>>>>> +                                     int address_cells, int size_cells,
>>>>> +                                     struct boot_info *bi)
>>>>
>>>> Functions without callers and non-static ones without declarations are
>>>> disliked by Misra.
>>>
>>> Can't do much about it if I want them to stand alone in a single patch.
>>> Otherwise the following ones become quite unwieldy to look at. All I can
>>> say is that this function becomes static and with a caller on the next
>>> patch.
>>
>> Which means you need to touch this again anyway. Perhaps we need a Misra
>> deviation for __maybe_unused functions / data, in which case you could
>> use that here and strip it along with making the function static. Cc-ing
>> Bugseng folks.
> 
> It's a transient violation, sure. Do we care about transient MISRA
> violations though? I understand the importance of bisectability, but
> AUIU MISRA compliance matters to the extent that that the tip is
> compliant rather than the intermediate steps?

Thing is that quite a few rules are blocking now. I haven't checked whether
the one here (already) is; if it isn't, we can't exclude it will be by the
time this patch is committed. If then the next patch isn't committed
together with it, we'd face a CI failure.

> Another option would be to fold them this patch and the next together
> after both get their R-by. As I said, I assumed you'd rather see them in
> isolation for purposes of review.

As it looks it's all plain code additions, so reviewability would merely
mildly suffer from patch size. But afaict there would be no loss of clarity.

>>>>> +    /* Otherwise location given as a `reg` property. */
>>>>> +    prop = fdt_get_property(fdt, node, "reg", NULL);
>>>>> +
>>>>> +    if ( !prop )
>>>>> +    {
>>>>> +        printk("  No location for multiboot,module\n");
>>>>> +        return -EINVAL;
>>>>> +    }
>>>>> +    if ( fdt_get_property(fdt, node, "module-index", NULL) )
>>>>> +    {
>>>>> +        printk("  Location of multiboot,module defined multiple times\n");
>>>>> +        return -EINVAL;
>>>>> +    }
>>>>> +
>>>>> +    ret = read_fdt_prop_as_reg(prop, address_cells, size_cells, &addr, &size);
>>>>> +
>>>>> +    if ( ret < 0 )
>>>>> +    {
>>>>> +        printk("  Failed reading reg for multiboot,module\n");
>>>>> +        return -EINVAL;
>>>>> +    }
>>>>> +
>>>>> +    idx = bi->nr_modules + 1;
>>>>
>>>> This at least looks like an off-by-one. If the addition of 1 is really
>>>> intended, I think it needs commenting on.
>>>
>>> Seems to be, yes. The underlying array is a bit bizarre. It's sizes as
>>> MAX_NR_BOOTMODS + 1, with the first one being the DTB itself. I guess
>>> the intent was to take it into account, but bi->nr_modules is
>>> initialised to the number of multiboot modules, so it SHOULD be already
>>> taking it into account.
>>>
>>> Also, the logic for bounds checking seems... off (because of the + 1 I
>>> mentioned before). Or at least confusing, so I've moved to using
>>> ARRAY_SIZE(bi->mods) rather than explicitly comparing against
>>> MAX_NR_BOOTMODS.
>>>
>>> The array is MAX_NR_BOOTMODS + 1 in length, so it's just more cognitive
>>> load than I'm comfortable with.
>>
>> If I'm not mistaken the +1 is inherited from the modules array we had in
>> the past, where we wanted 1 extra slot for Xen itself. Hence before you
>> move to using ARRAY_SIZE() everywhere it needs to really be clear what
>> the +1 here is used for.
> 
> Ew.  Ok, just looked at the code in multiboot_fill_boot_info and indeed
> the arrangement is for all multiboot modules to be in front, and Xen to
> be appended. But bi->nr_modules only lists multiboot modules, so
> increasing that value is therefore not enough (or
> next_boot_module_index() would fail).
> 
> I need to have a proper read on how this is all stitched together.  I
> may simply swap BOOTMOD_XEN with the next entry on append. Though my
> preference would be to _not_ have Xen as part of the module list to
> begin with. Before boot_info that was probably a place as good as any,
> but this would be much better off in a dedicated field.
> 
> I don't see much in terms of usage though. Why is it being added at all?

For hyperlaunch I fear it's you who needs to answer this question. For
pre-hyperlaunch it's (primarily?) for consider_modules(), iirc. See two
of the three comments ahead of its non-recursive invocations.

Jan

Re: [PATCH v3 08/16] x86/hyperlaunch: Add helpers to locate multiboot modules

[Alejandro Vallejo]

On Tue Apr 15, 2025 at 7:05 AM BST, Jan Beulich wrote:
> On 14.04.2025 20:01, Alejandro Vallejo wrote:
>> On Mon Apr 14, 2025 at 4:05 PM BST, Jan Beulich wrote:
>>> On 14.04.2025 15:37, Alejandro Vallejo wrote:
>>>> On Thu Apr 10, 2025 at 11:42 AM BST, Jan Beulich wrote:
>>>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>>>> +/*
>>>>>> + * Locate a multiboot module given its node offset in the FDT.
>>>>>> + *
>>>>>> + * The module location may be given via either FDT property:
>>>>>> + *     * reg = <address, size>
>>>>>> + *         * Mutates `bi` to append the module.
>>>>>> + *     * module-index = <idx>
>>>>>> + *         * Leaves `bi` unchanged.
>>>>>> + *
>>>>>> + * @param fdt           Pointer to the full FDT.
>>>>>> + * @param node          Offset for the module node.
>>>>>> + * @param address_cells Number of 4-octet cells that make up an "address".
>>>>>> + * @param size_cells    Number of 4-octet cells that make up a "size".
>>>>>> + * @param bi[inout]     Xen's representation of the boot parameters.
>>>>>> + * @return              -EINVAL on malformed nodes, otherwise
>>>>>> + *                      index inside `bi->mods`
>>>>>> + */
>>>>>> +int __init fdt_read_multiboot_module(const void *fdt, int node,
>>>>>> +                                     int address_cells, int size_cells,
>>>>>> +                                     struct boot_info *bi)
>>>>>
>>>>> Functions without callers and non-static ones without declarations are
>>>>> disliked by Misra.
>>>>
>>>> Can't do much about it if I want them to stand alone in a single patch.
>>>> Otherwise the following ones become quite unwieldy to look at. All I can
>>>> say is that this function becomes static and with a caller on the next
>>>> patch.
>>>
>>> Which means you need to touch this again anyway. Perhaps we need a Misra
>>> deviation for __maybe_unused functions / data, in which case you could
>>> use that here and strip it along with making the function static. Cc-ing
>>> Bugseng folks.
>> 
>> It's a transient violation, sure. Do we care about transient MISRA
>> violations though? I understand the importance of bisectability, but
>> AUIU MISRA compliance matters to the extent that that the tip is
>> compliant rather than the intermediate steps?
>
> Thing is that quite a few rules are blocking now. I haven't checked whether
> the one here (already) is; if it isn't, we can't exclude it will be by the
> time this patch is committed. If then the next patch isn't committed
> together with it, we'd face a CI failure.
>
>> Another option would be to fold them this patch and the next together
>> after both get their R-by. As I said, I assumed you'd rather see them in
>> isolation for purposes of review.
>
> As it looks it's all plain code additions, so reviewability would merely
> mildly suffer from patch size. But afaict there would be no loss of clarity.
>
>>>>>> +    /* Otherwise location given as a `reg` property. */
>>>>>> +    prop = fdt_get_property(fdt, node, "reg", NULL);
>>>>>> +
>>>>>> +    if ( !prop )
>>>>>> +    {
>>>>>> +        printk("  No location for multiboot,module\n");
>>>>>> +        return -EINVAL;
>>>>>> +    }
>>>>>> +    if ( fdt_get_property(fdt, node, "module-index", NULL) )
>>>>>> +    {
>>>>>> +        printk("  Location of multiboot,module defined multiple times\n");
>>>>>> +        return -EINVAL;
>>>>>> +    }
>>>>>> +
>>>>>> +    ret = read_fdt_prop_as_reg(prop, address_cells, size_cells, &addr, &size);
>>>>>> +
>>>>>> +    if ( ret < 0 )
>>>>>> +    {
>>>>>> +        printk("  Failed reading reg for multiboot,module\n");
>>>>>> +        return -EINVAL;
>>>>>> +    }
>>>>>> +
>>>>>> +    idx = bi->nr_modules + 1;
>>>>>
>>>>> This at least looks like an off-by-one. If the addition of 1 is really
>>>>> intended, I think it needs commenting on.
>>>>
>>>> Seems to be, yes. The underlying array is a bit bizarre. It's sizes as
>>>> MAX_NR_BOOTMODS + 1, with the first one being the DTB itself. I guess
>>>> the intent was to take it into account, but bi->nr_modules is
>>>> initialised to the number of multiboot modules, so it SHOULD be already
>>>> taking it into account.
>>>>
>>>> Also, the logic for bounds checking seems... off (because of the + 1 I
>>>> mentioned before). Or at least confusing, so I've moved to using
>>>> ARRAY_SIZE(bi->mods) rather than explicitly comparing against
>>>> MAX_NR_BOOTMODS.
>>>>
>>>> The array is MAX_NR_BOOTMODS + 1 in length, so it's just more cognitive
>>>> load than I'm comfortable with.
>>>
>>> If I'm not mistaken the +1 is inherited from the modules array we had in
>>> the past, where we wanted 1 extra slot for Xen itself. Hence before you
>>> move to using ARRAY_SIZE() everywhere it needs to really be clear what
>>> the +1 here is used for.
>> 
>> Ew.  Ok, just looked at the code in multiboot_fill_boot_info and indeed
>> the arrangement is for all multiboot modules to be in front, and Xen to
>> be appended. But bi->nr_modules only lists multiboot modules, so
>> increasing that value is therefore not enough (or
>> next_boot_module_index() would fail).
>> 
>> I need to have a proper read on how this is all stitched together.  I
>> may simply swap BOOTMOD_XEN with the next entry on append. Though my
>> preference would be to _not_ have Xen as part of the module list to
>> begin with. Before boot_info that was probably a place as good as any,
>> but this would be much better off in a dedicated field.
>> 
>> I don't see much in terms of usage though. Why is it being added at all?
>
> For hyperlaunch I fear it's you who needs to answer this question. For
> pre-hyperlaunch it's (primarily?) for consider_modules(), iirc. See two
> of the three comments ahead of its non-recursive invocations.
>
> Jan

There's no specific need for it on hyperlaunch AFAIK. Fixing
consider_modules to not require Xen being on the list of modules is easy
enough on both arm and x86 (it's a matter of passing the boot_info in
full rather than array + size), but I fear there may be more instances of
such checks.

I'll let it be for the time being and take a mental note to untangle
it later on. For this I'll simply ensure the append logic maintains Xen
at the back, as a sentinel of sorts for the module list, and document
that behaviour in the boot_info itself.

Cheers,
Alejandro

Re: [PATCH v3 08/16] x86/hyperlaunch: Add helpers to locate multiboot modules

[Nicola Vetrini]

On 2025-04-15 08:05, Jan Beulich wrote:
> On 14.04.2025 20:01, Alejandro Vallejo wrote:
>> On Mon Apr 14, 2025 at 4:05 PM BST, Jan Beulich wrote:
>>> On 14.04.2025 15:37, Alejandro Vallejo wrote:
>>>> On Thu Apr 10, 2025 at 11:42 AM BST, Jan Beulich wrote:
>>>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>>>> +/*
>>>>>> + * Locate a multiboot module given its node offset in the FDT.
>>>>>> + *
>>>>>> + * The module location may be given via either FDT property:
>>>>>> + *     * reg = <address, size>
>>>>>> + *         * Mutates `bi` to append the module.
>>>>>> + *     * module-index = <idx>
>>>>>> + *         * Leaves `bi` unchanged.
>>>>>> + *
>>>>>> + * @param fdt           Pointer to the full FDT.
>>>>>> + * @param node          Offset for the module node.
>>>>>> + * @param address_cells Number of 4-octet cells that make up an 
>>>>>> "address".
>>>>>> + * @param size_cells    Number of 4-octet cells that make up a 
>>>>>> "size".
>>>>>> + * @param bi[inout]     Xen's representation of the boot 
>>>>>> parameters.
>>>>>> + * @return              -EINVAL on malformed nodes, otherwise
>>>>>> + *                      index inside `bi->mods`
>>>>>> + */
>>>>>> +int __init fdt_read_multiboot_module(const void *fdt, int node,
>>>>>> +                                     int address_cells, int 
>>>>>> size_cells,
>>>>>> +                                     struct boot_info *bi)
>>>>> 
>>>>> Functions without callers and non-static ones without declarations 
>>>>> are
>>>>> disliked by Misra.
>>>> 
>>>> Can't do much about it if I want them to stand alone in a single 
>>>> patch.
>>>> Otherwise the following ones become quite unwieldy to look at. All I 
>>>> can
>>>> say is that this function becomes static and with a caller on the 
>>>> next
>>>> patch.
>>> 
>>> Which means you need to touch this again anyway. Perhaps we need a 
>>> Misra
>>> deviation for __maybe_unused functions / data, in which case you 
>>> could
>>> use that here and strip it along with making the function static. 
>>> Cc-ing
>>> Bugseng folks.
>> 
>> It's a transient violation, sure. Do we care about transient MISRA
>> violations though? I understand the importance of bisectability, but
>> AUIU MISRA compliance matters to the extent that that the tip is
>> compliant rather than the intermediate steps?
> 
> Thing is that quite a few rules are blocking now. I haven't checked 
> whether
> the one here (already) is; if it isn't, we can't exclude it will be by 
> the
> time this patch is committed. If then the next patch isn't committed
> together with it, we'd face a CI failure.
> 

It's Rule 8.4, and it is indeed blocking. To double check, a scan on a 
push containing this patch should trigger the failure.
You may transitively add an inline deviation comment or just a deviation 
with a configuration (I can help if needed), justified by the subsequent 
addition of static.

>> Another option would be to fold them this patch and the next together
>> after both get their R-by. As I said, I assumed you'd rather see them 
>> in
>> isolation for purposes of review.
> 
> As it looks it's all plain code additions, so reviewability would 
> merely
> mildly suffer from patch size. But afaict there would be no loss of 
> clarity.
> 
>>>>>> +    /* Otherwise location given as a `reg` property. */
>>>>>> +    prop = fdt_get_property(fdt, node, "reg", NULL);
>>>>>> +
>>>>>> +    if ( !prop )
>>>>>> +    {
>>>>>> +        printk("  No location for multiboot,module\n");
>>>>>> +        return -EINVAL;
>>>>>> +    }
>>>>>> +    if ( fdt_get_property(fdt, node, "module-index", NULL) )
>>>>>> +    {
>>>>>> +        printk("  Location of multiboot,module defined multiple 
>>>>>> times\n");
>>>>>> +        return -EINVAL;
>>>>>> +    }
>>>>>> +
>>>>>> +    ret = read_fdt_prop_as_reg(prop, address_cells, size_cells, 
>>>>>> &addr, &size);
>>>>>> +
>>>>>> +    if ( ret < 0 )
>>>>>> +    {
>>>>>> +        printk("  Failed reading reg for multiboot,module\n");
>>>>>> +        return -EINVAL;
>>>>>> +    }
>>>>>> +
>>>>>> +    idx = bi->nr_modules + 1;
>>>>> 
>>>>> This at least looks like an off-by-one. If the addition of 1 is 
>>>>> really
>>>>> intended, I think it needs commenting on.
>>>> 
>>>> Seems to be, yes. The underlying array is a bit bizarre. It's sizes 
>>>> as
>>>> MAX_NR_BOOTMODS + 1, with the first one being the DTB itself. I 
>>>> guess
>>>> the intent was to take it into account, but bi->nr_modules is
>>>> initialised to the number of multiboot modules, so it SHOULD be 
>>>> already
>>>> taking it into account.
>>>> 
>>>> Also, the logic for bounds checking seems... off (because of the + 1 
>>>> I
>>>> mentioned before). Or at least confusing, so I've moved to using
>>>> ARRAY_SIZE(bi->mods) rather than explicitly comparing against
>>>> MAX_NR_BOOTMODS.
>>>> 
>>>> The array is MAX_NR_BOOTMODS + 1 in length, so it's just more 
>>>> cognitive
>>>> load than I'm comfortable with.
>>> 
>>> If I'm not mistaken the +1 is inherited from the modules array we had 
>>> in
>>> the past, where we wanted 1 extra slot for Xen itself. Hence before 
>>> you
>>> move to using ARRAY_SIZE() everywhere it needs to really be clear 
>>> what
>>> the +1 here is used for.
>> 
>> Ew.  Ok, just looked at the code in multiboot_fill_boot_info and 
>> indeed
>> the arrangement is for all multiboot modules to be in front, and Xen 
>> to
>> be appended. But bi->nr_modules only lists multiboot modules, so
>> increasing that value is therefore not enough (or
>> next_boot_module_index() would fail).
>> 
>> I need to have a proper read on how this is all stitched together.  I
>> may simply swap BOOTMOD_XEN with the next entry on append. Though my
>> preference would be to _not_ have Xen as part of the module list to
>> begin with. Before boot_info that was probably a place as good as any,
>> but this would be much better off in a dedicated field.
>> 
>> I don't see much in terms of usage though. Why is it being added at 
>> all?
> 
> For hyperlaunch I fear it's you who needs to answer this question. For
> pre-hyperlaunch it's (primarily?) for consider_modules(), iirc. See two
> of the three comments ahead of its non-recursive invocations.
> 
> Jan

-- 
Nicola Vetrini, B.Sc.
Software Engineer
BUGSENG (https://bugseng.com)
LinkedIn: https://www.linkedin.com/in/nicola-vetrini-a42471253

Re: [PATCH v3 08/16] x86/hyperlaunch: Add helpers to locate multiboot modules

[Alejandro Vallejo]

On Wed Apr 16, 2025 at 5:55 PM BST, Nicola Vetrini wrote:
> On 2025-04-15 08:05, Jan Beulich wrote:
>> On 14.04.2025 20:01, Alejandro Vallejo wrote:
>>> On Mon Apr 14, 2025 at 4:05 PM BST, Jan Beulich wrote:
>>>> On 14.04.2025 15:37, Alejandro Vallejo wrote:
>>>>> On Thu Apr 10, 2025 at 11:42 AM BST, Jan Beulich wrote:
>>>>>> Functions without callers and non-static ones without declarations 
>>>>>> are
>>>>>> disliked by Misra.
>>>>> 
>>>>> Can't do much about it if I want them to stand alone in a single 
>>>>> patch.
>>>>> Otherwise the following ones become quite unwieldy to look at. All I 
>>>>> can
>>>>> say is that this function becomes static and with a caller on the 
>>>>> next
>>>>> patch.
>>>> 
>>>> Which means you need to touch this again anyway. Perhaps we need a 
>>>> Misra
>>>> deviation for __maybe_unused functions / data, in which case you 
>>>> could
>>>> use that here and strip it along with making the function static. 
>>>> Cc-ing
>>>> Bugseng folks.
>>> 
>>> It's a transient violation, sure. Do we care about transient MISRA
>>> violations though? I understand the importance of bisectability, but
>>> AUIU MISRA compliance matters to the extent that that the tip is
>>> compliant rather than the intermediate steps?
>> 
>> Thing is that quite a few rules are blocking now. I haven't checked 
>> whether
>> the one here (already) is; if it isn't, we can't exclude it will be by 
>> the
>> time this patch is committed. If then the next patch isn't committed
>> together with it, we'd face a CI failure.
>> 
>
> It's Rule 8.4, and it is indeed blocking. To double check, a scan on a 
> push containing this patch should trigger the failure.
> You may transitively add an inline deviation comment or just a deviation 
> with a configuration (I can help if needed), justified by the subsequent 
> addition of static.

Thanks for the context!

If I'm going to add something and remove it later might as well add the
missing declaration and remove it when static-ifying the function.
Particularly because I don't see a suitable x in SAFE-x-safe to use. (1
or 13 may work, but they hardly reflect what's being done.)

My bad for (mis)assuming transient states merely required bisectability
rather than full MISRA compliance. 

Cheers,
Alejandro

Re: [PATCH v3 08/16] x86/hyperlaunch: Add helpers to locate multiboot modules

[Nicola Vetrini]

On 2025-04-14 17:05, Jan Beulich wrote:
> On 14.04.2025 15:37, Alejandro Vallejo wrote:
>> On Thu Apr 10, 2025 at 11:42 AM BST, Jan Beulich wrote:
>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>> +/*
>>>> + * Locate a multiboot module given its node offset in the FDT.
>>>> + *
>>>> + * The module location may be given via either FDT property:
>>>> + *     * reg = <address, size>
>>>> + *         * Mutates `bi` to append the module.
>>>> + *     * module-index = <idx>
>>>> + *         * Leaves `bi` unchanged.
>>>> + *
>>>> + * @param fdt           Pointer to the full FDT.
>>>> + * @param node          Offset for the module node.
>>>> + * @param address_cells Number of 4-octet cells that make up an 
>>>> "address".
>>>> + * @param size_cells    Number of 4-octet cells that make up a 
>>>> "size".
>>>> + * @param bi[inout]     Xen's representation of the boot 
>>>> parameters.
>>>> + * @return              -EINVAL on malformed nodes, otherwise
>>>> + *                      index inside `bi->mods`
>>>> + */
>>>> +int __init fdt_read_multiboot_module(const void *fdt, int node,
>>>> +                                     int address_cells, int 
>>>> size_cells,
>>>> +                                     struct boot_info *bi)
>>> 
>>> Functions without callers and non-static ones without declarations 
>>> are
>>> disliked by Misra.
>> 
>> Can't do much about it if I want them to stand alone in a single 
>> patch.
>> Otherwise the following ones become quite unwieldy to look at. All I 
>> can
>> say is that this function becomes static and with a caller on the next
>> patch.
> 
> Which means you need to touch this again anyway. Perhaps we need a 
> Misra
> deviation for __maybe_unused functions / data, in which case you could
> use that here and strip it along with making the function static. 
> Cc-ing
> Bugseng folks.
> 

There is already an exception for __maybe_unused on labels (Rule 2.6). 
In principle it could be easily extended to encompass unused functions 
(which are verified by another rule), with a suitable rationale.

>>>> +    /* Otherwise location given as a `reg` property. */
>>>> +    prop = fdt_get_property(fdt, node, "reg", NULL);
>>>> +
>>>> +    if ( !prop )
>>>> +    {
>>>> +        printk("  No location for multiboot,module\n");
>>>> +        return -EINVAL;
>>>> +    }
>>>> +    if ( fdt_get_property(fdt, node, "module-index", NULL) )
>>>> +    {
>>>> +        printk("  Location of multiboot,module defined multiple 
>>>> times\n");
>>>> +        return -EINVAL;
>>>> +    }
>>>> +
>>>> +    ret = read_fdt_prop_as_reg(prop, address_cells, size_cells, 
>>>> &addr, &size);
>>>> +
>>>> +    if ( ret < 0 )
>>>> +    {
>>>> +        printk("  Failed reading reg for multiboot,module\n");
>>>> +        return -EINVAL;
>>>> +    }
>>>> +
>>>> +    idx = bi->nr_modules + 1;
>>> 
>>> This at least looks like an off-by-one. If the addition of 1 is 
>>> really
>>> intended, I think it needs commenting on.
>> 
>> Seems to be, yes. The underlying array is a bit bizarre. It's sizes as
>> MAX_NR_BOOTMODS + 1, with the first one being the DTB itself. I guess
>> the intent was to take it into account, but bi->nr_modules is
>> initialised to the number of multiboot modules, so it SHOULD be 
>> already
>> taking it into account.
>> 
>> Also, the logic for bounds checking seems... off (because of the + 1 I
>> mentioned before). Or at least confusing, so I've moved to using
>> ARRAY_SIZE(bi->mods) rather than explicitly comparing against
>> MAX_NR_BOOTMODS.
>> 
>> The array is MAX_NR_BOOTMODS + 1 in length, so it's just more 
>> cognitive
>> load than I'm comfortable with.
> 
> If I'm not mistaken the +1 is inherited from the modules array we had 
> in
> the past, where we wanted 1 extra slot for Xen itself. Hence before you
> move to using ARRAY_SIZE() everywhere it needs to really be clear what
> the +1 here is used for.
> 
>>>> --- a/xen/include/xen/libfdt/libfdt-xen.h
>>>> +++ b/xen/include/xen/libfdt/libfdt-xen.h
>>>> @@ -13,6 +13,63 @@kkk
>>>> 
>>>>  #include <xen/libfdt/libfdt.h>
>>>> 
>>>> +static inline int __init fdt_cell_as_u32(const fdt32_t *cell)
>>> 
>>> Why plain int here, but ...
>>> 
>>>> +{
>>>> +    return fdt32_to_cpu(*cell);
>>>> +}
>>>> +
>>>> +static inline uint64_t  __init fdt_cell_as_u64(const fdt32_t *cell)
>>> 
>>> ... a fixed-width and unsigned type here? Question is whether the 
>>> former
>>> helper is really warranted.
>>> 
>>> Also nit: Stray double blank.
>>> 
>>>> +{
>>>> +    return ((uint64_t)fdt32_to_cpu(cell[0]) << 32) | 
>>>> fdt32_to_cpu(cell[1]);
>>> 
>>> That is - uniformly big endian?
>> 
>> These helpers are disappearing, so it doesn't matter. This is 
>> basically
>> an open coded:
>> 
>>   fdt64_to_cpu(*(const fdt64_t *)fdt32)
>> 
>> And, yes. DTBs are standardised as having big-endian properties, for
>> better or worse :/
>> 
>>> 
>>>> +}
>>> 
>>> Marking such relatively generic inline functions __init is also 
>>> somewhat
>>> risky.
>> 
>> They were originally in domain-builder/fdt.c and moved here as a 
>> result
>> of a request to have them on libfdt. libfdt proved to be somewhat
>> annoying because it would be hard to distinguish accessors for the
>> flattened and the unflattened tree.
>> 
>> I'd personally have them in domain-builder instead, where they are 
>> used.
>> Should they be needed somewhere else, we can always fator them out
>> somewhere else.
>> 
>> Thoughts?
> 
> As long as they're needed only by domain-builder, it's probably fine to 
> have
> them just there.
> 
> Jan

-- 
Nicola Vetrini, B.Sc.
Software Engineer
BUGSENG (https://bugseng.com)
LinkedIn: https://www.linkedin.com/in/nicola-vetrini-a42471253

[PATCH v3 09/16] x86/hyperlaunch: locate dom0 kernel with hyperlaunch

[Alejandro Vallejo]

From: "Daniel P. Smith" <dpsmith@apertussolutions.com>

Look for a subnode of type `multiboot,kernel` within a domain node. If
found, locate it using the multiboot module helper to generically ensure
it lives in the module list. If the bootargs property is present and
there was not an MB1 string, then use the command line from the device
tree definition.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
Signed-off-by: Alejandro Vallejo <agarciav@amd.com>
---
v3:
    * Add const to fdt
    * Remove idx == NULL checks
    * Add BUILD_BUG_ON for MAX_NR_BOOTMODS fitting in a uint32_t
    * Remove trailing ) from printks
    * Return ENODATA for missing kernel
    * Re-work "max domains" warning and print limit
    * fdt_cell_as_u32/directly return values
    * Remove "pairs" looping from fdt_get_reg_prop() and only grab 1.
    * Use addr_cells and size_cells
---
 xen/arch/x86/domain-builder/core.c | 11 ++++++
 xen/arch/x86/domain-builder/fdt.c  | 57 ++++++++++++++++++++++++++++++
 xen/arch/x86/setup.c               |  5 ---
 3 files changed, 68 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/domain-builder/core.c b/xen/arch/x86/domain-builder/core.c
index c50eff34fb..eda7fa7a8f 100644
--- a/xen/arch/x86/domain-builder/core.c
+++ b/xen/arch/x86/domain-builder/core.c
@@ -59,6 +59,17 @@ void __init builder_init(struct boot_info *bi)
 
         printk(XENLOG_INFO "  Number of domains: %d\n", bi->nr_domains);
     }
+    else
+    {
+        unsigned int i;
+
+        /* Find first unknown boot module to use as Dom0 kernel */
+        printk("Falling back to using first boot module as dom0\n");
+        i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
+        bi->mods[i].type = BOOTMOD_KERNEL;
+        bi->domains[0].kernel = &bi->mods[i];
+        bi->nr_domains = 1;
+    }
 }
 
 /*
diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
index 9ebc8fd0e4..a037c8b6cb 100644
--- a/xen/arch/x86/domain-builder/fdt.c
+++ b/xen/arch/x86/domain-builder/fdt.c
@@ -155,6 +155,52 @@ int __init fdt_read_multiboot_module(const void *fdt, int node,
     return idx;
 }
 
+static int __init process_domain_node(
+    struct boot_info *bi, const void *fdt, int dom_node)
+{
+    int node;
+    struct boot_domain *bd = &bi->domains[bi->nr_domains];
+    const char *name = fdt_get_name(fdt, dom_node, NULL) ?: "unknown";
+    int address_cells = fdt_address_cells(fdt, dom_node);
+    int size_cells = fdt_size_cells(fdt, dom_node);
+
+    fdt_for_each_subnode(node, fdt, dom_node)
+    {
+        if ( fdt_node_check_compatible(fdt, node, "multiboot,kernel") == 0 )
+        {
+            int idx;
+
+            if ( bd->kernel )
+            {
+                printk(XENLOG_ERR "Duplicate kernel module for domain %s\n",
+                       name);
+                continue;
+            }
+
+            idx = fdt_read_multiboot_module(fdt, node, address_cells,
+                                            size_cells, bi);
+            if ( idx < 0 )
+            {
+                printk("  failed processing kernel module for domain %s\n",
+                       name);
+                return idx;
+            }
+
+            printk("  kernel: boot module %d\n", idx);
+            bi->mods[idx].type = BOOTMOD_KERNEL;
+            bd->kernel = &bi->mods[idx];
+        }
+    }
+
+    if ( !bd->kernel )
+    {
+        printk(XENLOG_ERR "ERR: no kernel assigned to domain\n");
+        return -ENODATA;
+    }
+
+    return 0;
+}
+
 static int __init find_hyperlaunch_node(const void *fdt)
 {
     int hv_node = fdt_path_offset(fdt, "/chosen/hypervisor");
@@ -217,9 +263,20 @@ int __init walk_hyperlaunch_fdt(struct boot_info *bi)
 
     fdt_for_each_subnode(node, fdt, hv_node)
     {
+        if ( bi->nr_domains >= MAX_NR_BOOTDOMS )
+        {
+            printk(XENLOG_WARNING
+                   "WARN: only creating first %u domains\n", MAX_NR_BOOTDOMS);
+            break;
+        }
+
         ret = fdt_node_check_compatible(fdt, node, "xen,domain");
         if ( ret == 0 )
+        {
+            if ( (ret = process_domain_node(bi, fdt, node)) < 0 )
+                break;
             bi->nr_domains++;
+        }
     }
 
     /* Until multi-domain construction is added, throw an error */
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index e5d78bcb48..00e8c8a2a3 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1284,11 +1284,6 @@ void asmlinkage __init noreturn __start_xen(void)
 
     builder_init(bi);
 
-    /* Find first unknown boot module to use as Dom0 kernel */
-    i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
-    bi->mods[i].type = BOOTMOD_KERNEL;
-    bi->domains[0].kernel = &bi->mods[i];
-
     if ( pvh_boot )
     {
         /* pvh_init() already filled in e820_raw */
-- 
2.43.0

Re: [PATCH v3 09/16] x86/hyperlaunch: locate dom0 kernel with hyperlaunch

[Denis Mukhin]

On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:

> 
> 
> From: "Daniel P. Smith" dpsmith@apertussolutions.com
> 
> 
> Look for a subnode of type `multiboot,kernel` within a domain node. If
> found, locate it using the multiboot module helper to generically ensure
> it lives in the module list. If the bootargs property is present and
> there was not an MB1 string, then use the command line from the device
> tree definition.
> 
> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
> 
> Signed-off-by: Jason Andryuk jason.andryuk@amd.com
> 
> Signed-off-by: Alejandro Vallejo agarciav@amd.com
> 
> ---
> v3:
> * Add const to fdt
> * Remove idx == NULL checks
> * Add BUILD_BUG_ON for MAX_NR_BOOTMODS fitting in a uint32_t
> * Remove trailing ) from printks
> * Return ENODATA for missing kernel
> * Re-work "max domains" warning and print limit
> * fdt_cell_as_u32/directly return values
> * Remove "pairs" looping from fdt_get_reg_prop() and only grab 1.
> * Use addr_cells and size_cells
> ---
> xen/arch/x86/domain-builder/core.c | 11 ++++++
> xen/arch/x86/domain-builder/fdt.c | 57 ++++++++++++++++++++++++++++++
> xen/arch/x86/setup.c | 5 ---
> 3 files changed, 68 insertions(+), 5 deletions(-)
> 
> diff --git a/xen/arch/x86/domain-builder/core.c b/xen/arch/x86/domain-builder/core.c
> index c50eff34fb..eda7fa7a8f 100644
> --- a/xen/arch/x86/domain-builder/core.c
> +++ b/xen/arch/x86/domain-builder/core.c
> @@ -59,6 +59,17 @@ void __init builder_init(struct boot_info *bi)
> 
> printk(XENLOG_INFO " Number of domains: %d\n", bi->nr_domains);
> 
> }
> + else
> + {
> + unsigned int i;
> +
> + /* Find first unknown boot module to use as Dom0 kernel */
> + printk("Falling back to using first boot module as dom0\n");
> + i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
> + bi->mods[i].type = BOOTMOD_KERNEL;
> 
> + bi->domains[0].kernel = &bi->mods[i];
> 
> + bi->nr_domains = 1;
> 
> + }
> }
> 
> /*
> diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
> index 9ebc8fd0e4..a037c8b6cb 100644
> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -155,6 +155,52 @@ int __init fdt_read_multiboot_module(const void *fdt, int node,
> return idx;
> }
> 
> +static int __init process_domain_node(
> + struct boot_info *bi, const void *fdt, int dom_node)
> +{
> + int node;
> + struct boot_domain *bd = &bi->domains[bi->nr_domains];
> 
> + const char *name = fdt_get_name(fdt, dom_node, NULL) ?: "unknown";
> + int address_cells = fdt_address_cells(fdt, dom_node);
> + int size_cells = fdt_size_cells(fdt, dom_node);
> +
> + fdt_for_each_subnode(node, fdt, dom_node)
> + {
> + if ( fdt_node_check_compatible(fdt, node, "multiboot,kernel") == 0 )
> + {
> + int idx;
> +
> + if ( bd->kernel )
> 
> + {
> + printk(XENLOG_ERR "Duplicate kernel module for domain %s\n",

Looks like it should be XENLOG_WARNING since the loop continues.

Also, I would use either Capitalized or lower case messages everywhere
for consistency.

> + name);
> + continue;
> + }
> +
> + idx = fdt_read_multiboot_module(fdt, node, address_cells,
> + size_cells, bi);
> + if ( idx < 0 )
> + {
> + printk(" failed processing kernel module for domain %s\n",

I think this printout should have XENLOG_ERR in it since it's on the
error code path.

> + name);
> + return idx;
> + }
> +
> + printk(" kernel: boot module %d\n", idx);
> + bi->mods[idx].type = BOOTMOD_KERNEL;
> 
> + bd->kernel = &bi->mods[idx];
> 
> + }
> + }
> +
> + if ( !bd->kernel )
> 
> + {
> + printk(XENLOG_ERR "ERR: no kernel assigned to domain\n");

Drop "ERR" since it is already XENLOG_ERR level?

> + return -ENODATA;
> + }
> +
> + return 0;
> +}
> +
> static int __init find_hyperlaunch_node(const void *fdt)
> {
> int hv_node = fdt_path_offset(fdt, "/chosen/hypervisor");
> @@ -217,9 +263,20 @@ int __init walk_hyperlaunch_fdt(struct boot_info *bi)
> 
> fdt_for_each_subnode(node, fdt, hv_node)
> {
> + if ( bi->nr_domains >= MAX_NR_BOOTDOMS )
> 
> + {
> + printk(XENLOG_WARNING
> + "WARN: only creating first %u domains\n", MAX_NR_BOOTDOMS);

Drop "WARN" since it is already XENLOG_WARNING level?

> + break;
> + }
> +
> ret = fdt_node_check_compatible(fdt, node, "xen,domain");
> if ( ret == 0 )
> + {
> + if ( (ret = process_domain_node(bi, fdt, node)) < 0 )
> + break;
> bi->nr_domains++;
> 
> + }
> }
> 
> /* Until multi-domain construction is added, throw an error /
> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
> index e5d78bcb48..00e8c8a2a3 100644
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -1284,11 +1284,6 @@ void asmlinkage __init noreturn __start_xen(void)
> 
> builder_init(bi);
> 
> - / Find first unknown boot module to use as Dom0 kernel */
> - i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
> - bi->mods[i].type = BOOTMOD_KERNEL;
> 
> - bi->domains[0].kernel = &bi->mods[i];
> 
> -
> if ( pvh_boot )
> {
> /* pvh_init() already filled in e820_raw */
> --
> 2.43.0

Re: [PATCH v3 09/16] x86/hyperlaunch: locate dom0 kernel with hyperlaunch

[Alejandro Vallejo]

On Wed Apr 9, 2025 at 10:24 PM BST, Denis Mukhin wrote:
> On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:
>
>> 
>> 
>> From: "Daniel P. Smith" dpsmith@apertussolutions.com
>> 
>> 
>> Look for a subnode of type `multiboot,kernel` within a domain node. If
>> found, locate it using the multiboot module helper to generically ensure
>> it lives in the module list. If the bootargs property is present and
>> there was not an MB1 string, then use the command line from the device
>> tree definition.
>> 
>> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
>> 
>> Signed-off-by: Jason Andryuk jason.andryuk@amd.com
>> 
>> Signed-off-by: Alejandro Vallejo agarciav@amd.com
>> 
>> ---
>> v3:
>> * Add const to fdt
>> * Remove idx == NULL checks
>> * Add BUILD_BUG_ON for MAX_NR_BOOTMODS fitting in a uint32_t
>> * Remove trailing ) from printks
>> * Return ENODATA for missing kernel
>> * Re-work "max domains" warning and print limit
>> * fdt_cell_as_u32/directly return values
>> * Remove "pairs" looping from fdt_get_reg_prop() and only grab 1.
>> * Use addr_cells and size_cells
>> ---
>> xen/arch/x86/domain-builder/core.c | 11 ++++++
>> xen/arch/x86/domain-builder/fdt.c | 57 ++++++++++++++++++++++++++++++
>> xen/arch/x86/setup.c | 5 ---
>> 3 files changed, 68 insertions(+), 5 deletions(-)
>> 
>> diff --git a/xen/arch/x86/domain-builder/core.c b/xen/arch/x86/domain-builder/core.c
>> index c50eff34fb..eda7fa7a8f 100644
>> --- a/xen/arch/x86/domain-builder/core.c
>> +++ b/xen/arch/x86/domain-builder/core.c
>> @@ -59,6 +59,17 @@ void __init builder_init(struct boot_info *bi)
>> 
>> printk(XENLOG_INFO " Number of domains: %d\n", bi->nr_domains);
>> 
>> }
>> + else
>> + {
>> + unsigned int i;
>> +
>> + /* Find first unknown boot module to use as Dom0 kernel */
>> + printk("Falling back to using first boot module as dom0\n");
>> + i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
>> + bi->mods[i].type = BOOTMOD_KERNEL;
>> 
>> + bi->domains[0].kernel = &bi->mods[i];
>> 
>> + bi->nr_domains = 1;
>> 
>> + }
>> }
>> 
>> /*
>> diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
>> index 9ebc8fd0e4..a037c8b6cb 100644
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -155,6 +155,52 @@ int __init fdt_read_multiboot_module(const void *fdt, int node,
>> return idx;
>> }
>> 
>> +static int __init process_domain_node(
>> + struct boot_info *bi, const void *fdt, int dom_node)
>> +{
>> + int node;
>> + struct boot_domain *bd = &bi->domains[bi->nr_domains];
>> 
>> + const char *name = fdt_get_name(fdt, dom_node, NULL) ?: "unknown";
>> + int address_cells = fdt_address_cells(fdt, dom_node);
>> + int size_cells = fdt_size_cells(fdt, dom_node);
>> +
>> + fdt_for_each_subnode(node, fdt, dom_node)
>> + {
>> + if ( fdt_node_check_compatible(fdt, node, "multiboot,kernel") == 0 )
>> + {
>> + int idx;
>> +
>> + if ( bd->kernel )
>> 
>> + {
>> + printk(XENLOG_ERR "Duplicate kernel module for domain %s\n",
>
> Looks like it should be XENLOG_WARNING since the loop continues.

Fair point.

>
> Also, I would use either Capitalized or lower case messages everywhere
> for consistency.

That's related to those leading spaces. The lowercases end up
immediately under the configuration message so it's easier to bind them
visually as "hyperlaunch-related".

    (XEN) Hyperlaunch configuration:
    (XEN)   something
    (XEN)   failed processing kernel module for domain %s

>
>> + name);
>> + continue;
>> + }
>> +
>> + idx = fdt_read_multiboot_module(fdt, node, address_cells,
>> + size_cells, bi);
>> + if ( idx < 0 )
>> + {
>> + printk(" failed processing kernel module for domain %s\n",
>
> I think this printout should have XENLOG_ERR in it since it's on the
> error code path.

All of these should have a XENLOG_X so they can be skipped when _INFO
is itself filtered out.

>
>> + name);
>> + return idx;
>> + }
>> +
>> + printk(" kernel: boot module %d\n", idx);
>> + bi->mods[idx].type = BOOTMOD_KERNEL;
>> 
>> + bd->kernel = &bi->mods[idx];
>> 
>> + }
>> + }
>> +
>> + if ( !bd->kernel )
>> 
>> + {
>> + printk(XENLOG_ERR "ERR: no kernel assigned to domain\n");
>
> Drop "ERR" since it is already XENLOG_ERR level?

ERR: is printed though, whereas XENLOG_ERR is not. That's meant to make
it visually clear that's _really_ not meant to happen.

>
>> + return -ENODATA;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> static int __init find_hyperlaunch_node(const void *fdt)
>> {
>> int hv_node = fdt_path_offset(fdt, "/chosen/hypervisor");
>> @@ -217,9 +263,20 @@ int __init walk_hyperlaunch_fdt(struct boot_info *bi)
>> 
>> fdt_for_each_subnode(node, fdt, hv_node)
>> {
>> + if ( bi->nr_domains >= MAX_NR_BOOTDOMS )
>> 
>> + {
>> + printk(XENLOG_WARNING
>> + "WARN: only creating first %u domains\n", MAX_NR_BOOTDOMS);
>
> Drop "WARN" since it is already XENLOG_WARNING level?

Same rationale as above.

>
>> + break;
>> + }
>> +
>> ret = fdt_node_check_compatible(fdt, node, "xen,domain");
>> if ( ret == 0 )
>> + {
>> + if ( (ret = process_domain_node(bi, fdt, node)) < 0 )
>> + break;
>> bi->nr_domains++;
>> 
>> + }
>> }
>> 
>> /* Until multi-domain construction is added, throw an error /
>> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
>> index e5d78bcb48..00e8c8a2a3 100644
>> --- a/xen/arch/x86/setup.c
>> +++ b/xen/arch/x86/setup.c
>> @@ -1284,11 +1284,6 @@ void asmlinkage __init noreturn __start_xen(void)
>> 
>> builder_init(bi);
>> 
>> - / Find first unknown boot module to use as Dom0 kernel */
>> - i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
>> - bi->mods[i].type = BOOTMOD_KERNEL;
>> 
>> - bi->domains[0].kernel = &bi->mods[i];
>> 
>> -
>> if ( pvh_boot )
>> {
>> /* pvh_init() already filled in e820_raw */
>> --
>> 2.43.0

Cheers,
Alejandro

Re: [PATCH v3 09/16] x86/hyperlaunch: locate dom0 kernel with hyperlaunch

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
> 
> Look for a subnode of type `multiboot,kernel` within a domain node. If
> found, locate it using the multiboot module helper to generically ensure
> it lives in the module list. If the bootargs property is present and
> there was not an MB1 string, then use the command line from the device
> tree definition.
> 
> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
> Signed-off-by: Alejandro Vallejo <agarciav@amd.com>
> ---
> v3:
>     * Add const to fdt
>     * Remove idx == NULL checks
>     * Add BUILD_BUG_ON for MAX_NR_BOOTMODS fitting in a uint32_t

At least this one looks to rather belong into patch 08?

> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -155,6 +155,52 @@ int __init fdt_read_multiboot_module(const void *fdt, int node,
>      return idx;
>  }
>  
> +static int __init process_domain_node(
> +    struct boot_info *bi, const void *fdt, int dom_node)
> +{
> +    int node;
> +    struct boot_domain *bd = &bi->domains[bi->nr_domains];
> +    const char *name = fdt_get_name(fdt, dom_node, NULL) ?: "unknown";
> +    int address_cells = fdt_address_cells(fdt, dom_node);
> +    int size_cells = fdt_size_cells(fdt, dom_node);

Oh, okay - regarding my earlier comment elsewhere: If the sizes come from DT,
then of course ASSERT_UNREACHABLE() can't be used at the place where bogus
ones are rejected.

> +    fdt_for_each_subnode(node, fdt, dom_node)
> +    {
> +        if ( fdt_node_check_compatible(fdt, node, "multiboot,kernel") == 0 )
> +        {

When the loop body is merely an if() with a non-negligible body, inverting
the condition and using "continue" is usually better. Much like you do ...

> +            int idx;
> +
> +            if ( bd->kernel )
> +            {
> +                printk(XENLOG_ERR "Duplicate kernel module for domain %s\n",
> +                       name);
> +                continue;

... here already.

Jan

Re: [PATCH v3 09/16] x86/hyperlaunch: locate dom0 kernel with hyperlaunch

[Alejandro Vallejo]

On Thu Apr 10, 2025 at 11:58 AM BST, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>> 
>> Look for a subnode of type `multiboot,kernel` within a domain node. If
>> found, locate it using the multiboot module helper to generically ensure
>> it lives in the module list. If the bootargs property is present and
>> there was not an MB1 string, then use the command line from the device
>> tree definition.
>> 
>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>> Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
>> Signed-off-by: Alejandro Vallejo <agarciav@amd.com>
>> ---
>> v3:
>>     * Add const to fdt
>>     * Remove idx == NULL checks
>>     * Add BUILD_BUG_ON for MAX_NR_BOOTMODS fitting in a uint32_t
>
> At least this one looks to rather belong into patch 08?

Urg, yes. Sorry. There was a lot of code motion when I factored out the
helpers.

>
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -155,6 +155,52 @@ int __init fdt_read_multiboot_module(const void *fdt, int node,
>>      return idx;
>>  }
>>  
>> +static int __init process_domain_node(
>> +    struct boot_info *bi, const void *fdt, int dom_node)
>> +{
>> +    int node;
>> +    struct boot_domain *bd = &bi->domains[bi->nr_domains];
>> +    const char *name = fdt_get_name(fdt, dom_node, NULL) ?: "unknown";
>> +    int address_cells = fdt_address_cells(fdt, dom_node);
>> +    int size_cells = fdt_size_cells(fdt, dom_node);
>
> Oh, okay - regarding my earlier comment elsewhere: If the sizes come from DT,
> then of course ASSERT_UNREACHABLE() can't be used at the place where bogus
> ones are rejected.
>
>> +    fdt_for_each_subnode(node, fdt, dom_node)
>> +    {
>> +        if ( fdt_node_check_compatible(fdt, node, "multiboot,kernel") == 0 )
>> +        {
>
> When the loop body is merely an if() with a non-negligible body, inverting
> the condition and using "continue" is usually better. Much like you do ...

This becomes a chain of if conditions later on, one per property.

>
>> +            int idx;
>> +
>> +            if ( bd->kernel )
>> +            {
>> +                printk(XENLOG_ERR "Duplicate kernel module for domain %s\n",
>> +                       name);
>> +                continue;
>
> ... here already.
>
> Jan

Cheers,
Alejandro

[PATCH v3 10/16] x86/hyperlaunch: obtain cmdline from device tree

[Alejandro Vallejo]

From: "Daniel P. Smith" <dpsmith@apertussolutions.com>

Add support to read the command line from the hyperlauunch device tree.
The device tree command line is located in the "bootargs" property of the
"multiboot,kernel" node.

A boot loader command line, e.g. a grub module string field, takes
precendence ove the device tree one since it is easier to modify.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
---
v3:
    * Rename to fdt_get_prop_offset()
    * Remove size_t cast from builder_get_cmdline_size()
    * fdt_get_prop_offset() use strcmp()
    * fdt_get_prop_offset() return bool
---
 xen/arch/x86/domain-builder/core.c        | 28 +++++++++++++++++++++++
 xen/arch/x86/domain-builder/fdt.c         |  6 +++++
 xen/arch/x86/domain-builder/fdt.h         | 25 ++++++++++++++++++++
 xen/arch/x86/include/asm/bootinfo.h       |  6 +++--
 xen/arch/x86/include/asm/domain-builder.h |  4 ++++
 xen/arch/x86/setup.c                      | 12 +++++++---
 xen/include/xen/libfdt/libfdt-xen.h       | 23 +++++++++++++++++++
 7 files changed, 99 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/domain-builder/core.c b/xen/arch/x86/domain-builder/core.c
index eda7fa7a8f..510a74a675 100644
--- a/xen/arch/x86/domain-builder/core.c
+++ b/xen/arch/x86/domain-builder/core.c
@@ -8,9 +8,37 @@
 #include <xen/lib.h>
 
 #include <asm/bootinfo.h>
+#include <asm/setup.h>
 
 #include "fdt.h"
 
+size_t __init builder_get_cmdline_size(struct boot_info *bi, int offset)
+{
+#ifdef CONFIG_DOMAIN_BUILDER
+    const void *fdt = bootstrap_map_bm(&bi->mods[HYPERLAUNCH_MODULE_IDX]);
+    int size = fdt_cmdline_prop_size(fdt, offset);
+
+    bootstrap_unmap();
+    return size < 0 ? 0 : size;
+#else
+    return 0;
+#endif
+}
+
+int __init builder_get_cmdline(
+    struct boot_info *bi, int offset, char *cmdline, size_t size)
+{
+#ifdef CONFIG_DOMAIN_BUILDER
+    const void *fdt = bootstrap_map_bm(&bi->mods[HYPERLAUNCH_MODULE_IDX]);
+    int ret = fdt_cmdline_prop_copy(fdt, offset, cmdline, size);
+
+    bootstrap_unmap();
+    return ret;
+#else
+    return 0;
+#endif
+}
+
 void __init builder_init(struct boot_info *bi)
 {
     if ( IS_ENABLED(CONFIG_DOMAIN_BUILDER) )
diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
index a037c8b6cb..bc9903a9de 100644
--- a/xen/arch/x86/domain-builder/fdt.c
+++ b/xen/arch/x86/domain-builder/fdt.c
@@ -189,6 +189,12 @@ static int __init process_domain_node(
             printk("  kernel: boot module %d\n", idx);
             bi->mods[idx].type = BOOTMOD_KERNEL;
             bd->kernel = &bi->mods[idx];
+
+            /* If bootloader didn't set cmdline, see if FDT provides one. */
+            if ( bd->kernel->cmdline_pa &&
+                 !((char *)__va(bd->kernel->cmdline_pa))[0] )
+                bd->kernel->fdt_cmdline = fdt_get_prop_offset(
+                    fdt, node, "bootargs", &bd->kernel->cmdline_pa);
         }
     }
 
diff --git a/xen/arch/x86/domain-builder/fdt.h b/xen/arch/x86/domain-builder/fdt.h
index e8769dc51c..91f665c8fd 100644
--- a/xen/arch/x86/domain-builder/fdt.h
+++ b/xen/arch/x86/domain-builder/fdt.h
@@ -12,6 +12,31 @@ struct boot_info;
 #define HYPERLAUNCH_MODULE_IDX 0
 
 #ifdef CONFIG_DOMAIN_BUILDER
+
+static inline int __init fdt_cmdline_prop_size(const void *fdt, int offset)
+{
+    int ret;
+
+    fdt_get_property_by_offset(fdt, offset, &ret);
+
+    return ret;
+}
+
+static inline int __init fdt_cmdline_prop_copy(
+    const void *fdt, int offset, char *cmdline, size_t size)
+{
+    int ret;
+    const struct fdt_property *prop =
+        fdt_get_property_by_offset(fdt, offset, &ret);
+
+    if ( ret < 0 )
+        return ret;
+
+    ASSERT(size > ret);
+
+    return strlcpy(cmdline, prop->data, ret);
+}
+
 int has_hyperlaunch_fdt(const struct boot_info *bi);
 int walk_hyperlaunch_fdt(struct boot_info *bi);
 #else
diff --git a/xen/arch/x86/include/asm/bootinfo.h b/xen/arch/x86/include/asm/bootinfo.h
index 1e3d582e45..5b2c93b1ef 100644
--- a/xen/arch/x86/include/asm/bootinfo.h
+++ b/xen/arch/x86/include/asm/bootinfo.h
@@ -35,11 +35,13 @@ struct boot_module {
 
     /*
      * Module State Flags:
-     *   relocated: indicates module has been relocated in memory.
-     *   released:  indicates module's pages have been freed.
+     *   relocated:   indicates module has been relocated in memory.
+     *   released:    indicates module's pages have been freed.
+     *   fdt_cmdline: indicates module's cmdline is in the FDT.
      */
     bool relocated:1;
     bool released:1;
+    bool fdt_cmdline:1;
 
     /*
      * A boot module may need decompressing by Xen.  Headroom is an estimate of
diff --git a/xen/arch/x86/include/asm/domain-builder.h b/xen/arch/x86/include/asm/domain-builder.h
index b6d9ba94de..7518b6ddf3 100644
--- a/xen/arch/x86/include/asm/domain-builder.h
+++ b/xen/arch/x86/include/asm/domain-builder.h
@@ -3,6 +3,10 @@
 
 struct boot_info;
 
+size_t __init builder_get_cmdline_size(struct boot_info *bi, int offset);
+int __init builder_get_cmdline(
+    struct boot_info *bi, int offset, char *cmdline, size_t size);
+
 void builder_init(struct boot_info *bi);
 
 #endif
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 00e8c8a2a3..ca4415d020 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -984,7 +984,10 @@ static size_t __init domain_cmdline_size(
 {
     size_t s = bi->kextra ? strlen(bi->kextra) : 0;
 
-    s += bd->kernel->cmdline_pa ? strlen(__va(bd->kernel->cmdline_pa)) : 0;
+    if ( bd->kernel->fdt_cmdline )
+        s += builder_get_cmdline_size(bi, bd->kernel->cmdline_pa);
+    else
+        s += strlen(__va(bd->kernel->cmdline_pa));
 
     if ( s == 0 )
         return s;
@@ -1047,9 +1050,12 @@ static struct domain *__init create_dom0(struct boot_info *bi)
         if ( !(cmdline = xzalloc_array(char, cmdline_size)) )
             panic("Error allocating cmdline buffer for %pd\n", d);
 
-        if ( bd->kernel->cmdline_pa )
+        if ( bd->kernel->fdt_cmdline )
+            builder_get_cmdline(
+                bi, bd->kernel->cmdline_pa, cmdline, cmdline_size);
+        else
             strlcpy(cmdline,
-                    cmdline_cook(__va(bd->kernel->cmdline_pa), bi->loader),
+                    cmdline_cook(__va(bd->kernel->cmdline_pa),bi->loader),
                     cmdline_size);
 
         if ( bi->kextra )
diff --git a/xen/include/xen/libfdt/libfdt-xen.h b/xen/include/xen/libfdt/libfdt-xen.h
index 2259c09a6a..e473fbaf0c 100644
--- a/xen/include/xen/libfdt/libfdt-xen.h
+++ b/xen/include/xen/libfdt/libfdt-xen.h
@@ -23,6 +23,29 @@ static inline uint64_t  __init fdt_cell_as_u64(const fdt32_t *cell)
     return ((uint64_t)fdt32_to_cpu(cell[0]) << 32) | fdt32_to_cpu(cell[1]);
 }
 
+static inline bool __init fdt_get_prop_offset(
+    const void *fdt, int node, const char *name, unsigned long *offset)
+{
+    int ret, poffset;
+    const char *pname;
+
+    fdt_for_each_property_offset(poffset, fdt, node)
+    {
+        fdt_getprop_by_offset(fdt, poffset, &pname, &ret);
+
+        if ( ret < 0 )
+            continue;
+
+        if ( strcmp(pname, name) == 0 )
+        {
+            *offset = poffset;
+            return true;
+        }
+    }
+
+    return false;
+}
+
 /*
  * Property: reg
  *
-- 
2.43.0

Re: [PATCH v3 10/16] x86/hyperlaunch: obtain cmdline from device tree

[Denis Mukhin]

On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:

> 
> 
> From: "Daniel P. Smith" dpsmith@apertussolutions.com
> 
> 
> Add support to read the command line from the hyperlauunch device tree.
> The device tree command line is located in the "bootargs" property of the
> "multiboot,kernel" node.
> 
> A boot loader command line, e.g. a grub module string field, takes
> precendence ove the device tree one since it is easier to modify.
> 
> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
> 
> Signed-off-by: Jason Andryuk jason.andryuk@amd.com
> 
> ---
> v3:
> * Rename to fdt_get_prop_offset()
> * Remove size_t cast from builder_get_cmdline_size()
> * fdt_get_prop_offset() use strcmp()
> * fdt_get_prop_offset() return bool
> ---
> xen/arch/x86/domain-builder/core.c | 28 +++++++++++++++++++++++
> xen/arch/x86/domain-builder/fdt.c | 6 +++++
> xen/arch/x86/domain-builder/fdt.h | 25 ++++++++++++++++++++
> xen/arch/x86/include/asm/bootinfo.h | 6 +++--
> xen/arch/x86/include/asm/domain-builder.h | 4 ++++
> xen/arch/x86/setup.c | 12 +++++++---
> xen/include/xen/libfdt/libfdt-xen.h | 23 +++++++++++++++++++
> 7 files changed, 99 insertions(+), 5 deletions(-)
> 
> diff --git a/xen/arch/x86/domain-builder/core.c b/xen/arch/x86/domain-builder/core.c
> index eda7fa7a8f..510a74a675 100644
> --- a/xen/arch/x86/domain-builder/core.c
> +++ b/xen/arch/x86/domain-builder/core.c
> @@ -8,9 +8,37 @@
> #include <xen/lib.h>
> 
> 
> #include <asm/bootinfo.h>
> 
> +#include <asm/setup.h>
> 
> 
> #include "fdt.h"
> 
> +size_t __init builder_get_cmdline_size(struct boot_info *bi, int offset)
> +{
> +#ifdef CONFIG_DOMAIN_BUILDER
> + const void *fdt = bootstrap_map_bm(&bi->mods[HYPERLAUNCH_MODULE_IDX]);
> 
> + int size = fdt_cmdline_prop_size(fdt, offset);
> +
> + bootstrap_unmap();
> + return size < 0 ? 0 : size;
> +#else
> + return 0;
> +#endif
> +}
> +
> +int __init builder_get_cmdline(
> + struct boot_info *bi, int offset, char *cmdline, size_t size)
> +{
> +#ifdef CONFIG_DOMAIN_BUILDER
> + const void *fdt = bootstrap_map_bm(&bi->mods[HYPERLAUNCH_MODULE_IDX]);
> 
> + int ret = fdt_cmdline_prop_copy(fdt, offset, cmdline, size);
> +
> + bootstrap_unmap();
> + return ret;
> +#else
> + return 0;
> +#endif
> +}
> +
> void __init builder_init(struct boot_info *bi)
> {
> if ( IS_ENABLED(CONFIG_DOMAIN_BUILDER) )
> diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
> index a037c8b6cb..bc9903a9de 100644
> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -189,6 +189,12 @@ static int __init process_domain_node(
> printk(" kernel: boot module %d\n", idx);
> bi->mods[idx].type = BOOTMOD_KERNEL;
> 
> bd->kernel = &bi->mods[idx];
> 
> +
> + /* If bootloader didn't set cmdline, see if FDT provides one. */
> + if ( bd->kernel->cmdline_pa &&
> 
> + !((char *)__va(bd->kernel->cmdline_pa))[0] )
> 
> + bd->kernel->fdt_cmdline = fdt_get_prop_offset(
> 
> + fdt, node, "bootargs", &bd->kernel->cmdline_pa);
> 
> }
> }
> 
> diff --git a/xen/arch/x86/domain-builder/fdt.h b/xen/arch/x86/domain-builder/fdt.h
> index e8769dc51c..91f665c8fd 100644
> --- a/xen/arch/x86/domain-builder/fdt.h
> +++ b/xen/arch/x86/domain-builder/fdt.h
> @@ -12,6 +12,31 @@ struct boot_info;
> #define HYPERLAUNCH_MODULE_IDX 0
> 
> #ifdef CONFIG_DOMAIN_BUILDER
> +
> +static inline int __init fdt_cmdline_prop_size(const void *fdt, int offset)
> +{
> + int ret;
> +
> + fdt_get_property_by_offset(fdt, offset, &ret);
> +
> + return ret;
> +}
> +
> +static inline int __init fdt_cmdline_prop_copy(
> + const void *fdt, int offset, char *cmdline, size_t size)
> +{
> + int ret;
> + const struct fdt_property *prop =
> + fdt_get_property_by_offset(fdt, offset, &ret);
> +
> + if ( ret < 0 )
> + return ret;
> +
> + ASSERT(size > ret);
> 
> +
> + return strlcpy(cmdline, prop->data, ret);
> 
> +}
> +
> int has_hyperlaunch_fdt(const struct boot_info *bi);
> int walk_hyperlaunch_fdt(struct boot_info bi);
> #else
> diff --git a/xen/arch/x86/include/asm/bootinfo.h b/xen/arch/x86/include/asm/bootinfo.h
> index 1e3d582e45..5b2c93b1ef 100644
> --- a/xen/arch/x86/include/asm/bootinfo.h
> +++ b/xen/arch/x86/include/asm/bootinfo.h
> @@ -35,11 +35,13 @@ struct boot_module {
> 
> /
> * Module State Flags:
> - * relocated: indicates module has been relocated in memory.
> - * released: indicates module's pages have been freed.
> + * relocated: indicates module has been relocated in memory.
> + * released: indicates module's pages have been freed.
> + * fdt_cmdline: indicates module's cmdline is in the FDT.
> /
> bool relocated:1;
> bool released:1;
> + bool fdt_cmdline:1;
> 
> /
> * A boot module may need decompressing by Xen. Headroom is an estimate of
> diff --git a/xen/arch/x86/include/asm/domain-builder.h b/xen/arch/x86/include/asm/domain-builder.h
> index b6d9ba94de..7518b6ddf3 100644
> --- a/xen/arch/x86/include/asm/domain-builder.h
> +++ b/xen/arch/x86/include/asm/domain-builder.h
> @@ -3,6 +3,10 @@
> 
> struct boot_info;
> 
> +size_t __init builder_get_cmdline_size(struct boot_info *bi, int offset);
> +int __init builder_get_cmdline(
> + struct boot_info *bi, int offset, char *cmdline, size_t size);
> +
> void builder_init(struct boot_info *bi);
> 
> #endif
> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
> index 00e8c8a2a3..ca4415d020 100644
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -984,7 +984,10 @@ static size_t __init domain_cmdline_size(
> {
> size_t s = bi->kextra ? strlen(bi->kextra) : 0;
> 
> 
> - s += bd->kernel->cmdline_pa ? strlen(__va(bd->kernel->cmdline_pa)) : 0;
> 
> + if ( bd->kernel->fdt_cmdline )
> 
> + s += builder_get_cmdline_size(bi, bd->kernel->cmdline_pa);
> 
> + else
> + s += strlen(__va(bd->kernel->cmdline_pa));
> 
> 
> if ( s == 0 )
> return s;
> @@ -1047,9 +1050,12 @@ static struct domain *__init create_dom0(struct boot_info *bi)
> if ( !(cmdline = xzalloc_array(char, cmdline_size)) )
> panic("Error allocating cmdline buffer for %pd\n", d);
> 
> - if ( bd->kernel->cmdline_pa )
> 
> + if ( bd->kernel->fdt_cmdline )
> 
> + builder_get_cmdline(
> + bi, bd->kernel->cmdline_pa, cmdline, cmdline_size);
> 
> + else
> strlcpy(cmdline,
> - cmdline_cook(__va(bd->kernel->cmdline_pa), bi->loader),
> 
> + cmdline_cook(__va(bd->kernel->cmdline_pa),bi->loader),

Add extra space before bi->loader?

> 
> cmdline_size);
> 
> if ( bi->kextra )
> 
> diff --git a/xen/include/xen/libfdt/libfdt-xen.h b/xen/include/xen/libfdt/libfdt-xen.h
> index 2259c09a6a..e473fbaf0c 100644
> --- a/xen/include/xen/libfdt/libfdt-xen.h
> +++ b/xen/include/xen/libfdt/libfdt-xen.h
> @@ -23,6 +23,29 @@ static inline uint64_t __init fdt_cell_as_u64(const fdt32_t *cell)
> return ((uint64_t)fdt32_to_cpu(cell[0]) << 32) | fdt32_to_cpu(cell[1]);
> }
> 
> +static inline bool __init fdt_get_prop_offset(
> + const void *fdt, int node, const char *name, unsigned long *offset)
> +{
> + int ret, poffset;
> + const char *pname;
> +
> + fdt_for_each_property_offset(poffset, fdt, node)
> + {
> + fdt_getprop_by_offset(fdt, poffset, &pname, &ret);

Return value is not checked.

> +
> + if ( ret < 0 )
> + continue;
> +
> + if ( strcmp(pname, name) == 0 )

I got an impression that the preferred form is 
   if ( !strcmp(pname, name) )

> + {
> + offset = poffset;
> + return true;
> + }
> + }
> +
> + return false;
> +}
> +
> /
> * Property: reg
> *
> --
> 2.43.0

Re: [PATCH v3 10/16] x86/hyperlaunch: obtain cmdline from device tree

[Alejandro Vallejo]

On Wed Apr 9, 2025 at 11:04 PM BST, Denis Mukhin wrote:
> On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:
>
>> 
>> 
>> From: "Daniel P. Smith" dpsmith@apertussolutions.com
>> 
>> 
>> Add support to read the command line from the hyperlauunch device tree.
>> The device tree command line is located in the "bootargs" property of the
>> "multiboot,kernel" node.
>> 
>> A boot loader command line, e.g. a grub module string field, takes
>> precendence ove the device tree one since it is easier to modify.
>> 
>> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
>> 
>> Signed-off-by: Jason Andryuk jason.andryuk@amd.com
>> 
>> ---
>> v3:
>> * Rename to fdt_get_prop_offset()
>> * Remove size_t cast from builder_get_cmdline_size()
>> * fdt_get_prop_offset() use strcmp()
>> * fdt_get_prop_offset() return bool
>> ---
>> xen/arch/x86/domain-builder/core.c | 28 +++++++++++++++++++++++
>> xen/arch/x86/domain-builder/fdt.c | 6 +++++
>> xen/arch/x86/domain-builder/fdt.h | 25 ++++++++++++++++++++
>> xen/arch/x86/include/asm/bootinfo.h | 6 +++--
>> xen/arch/x86/include/asm/domain-builder.h | 4 ++++
>> xen/arch/x86/setup.c | 12 +++++++---
>> xen/include/xen/libfdt/libfdt-xen.h | 23 +++++++++++++++++++
>> 7 files changed, 99 insertions(+), 5 deletions(-)
>> 
>> diff --git a/xen/arch/x86/domain-builder/core.c b/xen/arch/x86/domain-builder/core.c
>> index eda7fa7a8f..510a74a675 100644
>> --- a/xen/arch/x86/domain-builder/core.c
>> +++ b/xen/arch/x86/domain-builder/core.c
>> @@ -8,9 +8,37 @@
>> #include <xen/lib.h>
>> 
>> 
>> #include <asm/bootinfo.h>
>> 
>> +#include <asm/setup.h>
>> 
>> 
>> #include "fdt.h"
>> 
>> +size_t __init builder_get_cmdline_size(struct boot_info *bi, int offset)
>> +{
>> +#ifdef CONFIG_DOMAIN_BUILDER
>> + const void *fdt = bootstrap_map_bm(&bi->mods[HYPERLAUNCH_MODULE_IDX]);
>> 
>> + int size = fdt_cmdline_prop_size(fdt, offset);
>> +
>> + bootstrap_unmap();
>> + return size < 0 ? 0 : size;
>> +#else
>> + return 0;
>> +#endif
>> +}
>> +
>> +int __init builder_get_cmdline(
>> + struct boot_info *bi, int offset, char *cmdline, size_t size)
>> +{
>> +#ifdef CONFIG_DOMAIN_BUILDER
>> + const void *fdt = bootstrap_map_bm(&bi->mods[HYPERLAUNCH_MODULE_IDX]);
>> 
>> + int ret = fdt_cmdline_prop_copy(fdt, offset, cmdline, size);
>> +
>> + bootstrap_unmap();
>> + return ret;
>> +#else
>> + return 0;
>> +#endif
>> +}
>> +
>> void __init builder_init(struct boot_info *bi)
>> {
>> if ( IS_ENABLED(CONFIG_DOMAIN_BUILDER) )
>> diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
>> index a037c8b6cb..bc9903a9de 100644
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -189,6 +189,12 @@ static int __init process_domain_node(
>> printk(" kernel: boot module %d\n", idx);
>> bi->mods[idx].type = BOOTMOD_KERNEL;
>> 
>> bd->kernel = &bi->mods[idx];
>> 
>> +
>> + /* If bootloader didn't set cmdline, see if FDT provides one. */
>> + if ( bd->kernel->cmdline_pa &&
>> 
>> + !((char *)__va(bd->kernel->cmdline_pa))[0] )
>> 
>> + bd->kernel->fdt_cmdline = fdt_get_prop_offset(
>> 
>> + fdt, node, "bootargs", &bd->kernel->cmdline_pa);
>> 
>> }
>> }
>> 
>> diff --git a/xen/arch/x86/domain-builder/fdt.h b/xen/arch/x86/domain-builder/fdt.h
>> index e8769dc51c..91f665c8fd 100644
>> --- a/xen/arch/x86/domain-builder/fdt.h
>> +++ b/xen/arch/x86/domain-builder/fdt.h
>> @@ -12,6 +12,31 @@ struct boot_info;
>> #define HYPERLAUNCH_MODULE_IDX 0
>> 
>> #ifdef CONFIG_DOMAIN_BUILDER
>> +
>> +static inline int __init fdt_cmdline_prop_size(const void *fdt, int offset)
>> +{
>> + int ret;
>> +
>> + fdt_get_property_by_offset(fdt, offset, &ret);
>> +
>> + return ret;
>> +}
>> +
>> +static inline int __init fdt_cmdline_prop_copy(
>> + const void *fdt, int offset, char *cmdline, size_t size)
>> +{
>> + int ret;
>> + const struct fdt_property *prop =
>> + fdt_get_property_by_offset(fdt, offset, &ret);
>> +
>> + if ( ret < 0 )
>> + return ret;
>> +
>> + ASSERT(size > ret);
>> 
>> +
>> + return strlcpy(cmdline, prop->data, ret);
>> 
>> +}
>> +
>> int has_hyperlaunch_fdt(const struct boot_info *bi);
>> int walk_hyperlaunch_fdt(struct boot_info bi);
>> #else
>> diff --git a/xen/arch/x86/include/asm/bootinfo.h b/xen/arch/x86/include/asm/bootinfo.h
>> index 1e3d582e45..5b2c93b1ef 100644
>> --- a/xen/arch/x86/include/asm/bootinfo.h
>> +++ b/xen/arch/x86/include/asm/bootinfo.h
>> @@ -35,11 +35,13 @@ struct boot_module {
>> 
>> /
>> * Module State Flags:
>> - * relocated: indicates module has been relocated in memory.
>> - * released: indicates module's pages have been freed.
>> + * relocated: indicates module has been relocated in memory.
>> + * released: indicates module's pages have been freed.
>> + * fdt_cmdline: indicates module's cmdline is in the FDT.
>> /
>> bool relocated:1;
>> bool released:1;
>> + bool fdt_cmdline:1;
>> 
>> /
>> * A boot module may need decompressing by Xen. Headroom is an estimate of
>> diff --git a/xen/arch/x86/include/asm/domain-builder.h b/xen/arch/x86/include/asm/domain-builder.h
>> index b6d9ba94de..7518b6ddf3 100644
>> --- a/xen/arch/x86/include/asm/domain-builder.h
>> +++ b/xen/arch/x86/include/asm/domain-builder.h
>> @@ -3,6 +3,10 @@
>> 
>> struct boot_info;
>> 
>> +size_t __init builder_get_cmdline_size(struct boot_info *bi, int offset);
>> +int __init builder_get_cmdline(
>> + struct boot_info *bi, int offset, char *cmdline, size_t size);
>> +
>> void builder_init(struct boot_info *bi);
>> 
>> #endif
>> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
>> index 00e8c8a2a3..ca4415d020 100644
>> --- a/xen/arch/x86/setup.c
>> +++ b/xen/arch/x86/setup.c
>> @@ -984,7 +984,10 @@ static size_t __init domain_cmdline_size(
>> {
>> size_t s = bi->kextra ? strlen(bi->kextra) : 0;
>> 
>> 
>> - s += bd->kernel->cmdline_pa ? strlen(__va(bd->kernel->cmdline_pa)) : 0;
>> 
>> + if ( bd->kernel->fdt_cmdline )
>> 
>> + s += builder_get_cmdline_size(bi, bd->kernel->cmdline_pa);
>> 
>> + else
>> + s += strlen(__va(bd->kernel->cmdline_pa));
>> 
>> 
>> if ( s == 0 )
>> return s;
>> @@ -1047,9 +1050,12 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>> if ( !(cmdline = xzalloc_array(char, cmdline_size)) )
>> panic("Error allocating cmdline buffer for %pd\n", d);
>> 
>> - if ( bd->kernel->cmdline_pa )
>> 
>> + if ( bd->kernel->fdt_cmdline )
>> 
>> + builder_get_cmdline(
>> + bi, bd->kernel->cmdline_pa, cmdline, cmdline_size);
>> 
>> + else
>> strlcpy(cmdline,
>> - cmdline_cook(__va(bd->kernel->cmdline_pa), bi->loader),
>> 
>> + cmdline_cook(__va(bd->kernel->cmdline_pa),bi->loader),
>
> Add extra space before bi->loader?

Yes, was a spurious diff.

>
>> 
>> cmdline_size);
>> 
>> if ( bi->kextra )
>> 
>> diff --git a/xen/include/xen/libfdt/libfdt-xen.h b/xen/include/xen/libfdt/libfdt-xen.h
>> index 2259c09a6a..e473fbaf0c 100644
>> --- a/xen/include/xen/libfdt/libfdt-xen.h
>> +++ b/xen/include/xen/libfdt/libfdt-xen.h
>> @@ -23,6 +23,29 @@ static inline uint64_t __init fdt_cell_as_u64(const fdt32_t *cell)
>> return ((uint64_t)fdt32_to_cpu(cell[0]) << 32) | fdt32_to_cpu(cell[1]);
>> }
>> 
>> +static inline bool __init fdt_get_prop_offset(
>> + const void *fdt, int node, const char *name, unsigned long *offset)
>> +{
>> + int ret, poffset;
>> + const char *pname;
>> +
>> + fdt_for_each_property_offset(poffset, fdt, node)
>> + {
>> + fdt_getprop_by_offset(fdt, poffset, &pname, &ret);
>
> Return value is not checked.

The pointer itself is ignored, but the error code is placed in "ret"
(when there is an error). Hence the ret < 0 that follows this.

>
>> +
>> + if ( ret < 0 )
>> + continue;
>> +
>> + if ( strcmp(pname, name) == 0 )
>
> I got an impression that the preferred form is 
>    if ( !strcmp(pname, name) )

It varies per file. Doesn't matter much, but sure. Will change.

>
>> + {
>> + offset = poffset;
>> + return true;
>> + }
>> + }
>> +
>> + return false;
>> +}
>> +
>> /
>> * Property: reg
>> *
>> --
>> 2.43.0

Cheers,
Alejandro

Re: [PATCH v3 10/16] x86/hyperlaunch: obtain cmdline from device tree

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> --- a/xen/arch/x86/domain-builder/core.c
> +++ b/xen/arch/x86/domain-builder/core.c
> @@ -8,9 +8,37 @@
>  #include <xen/lib.h>
>  
>  #include <asm/bootinfo.h>
> +#include <asm/setup.h>
>  
>  #include "fdt.h"
>  
> +size_t __init builder_get_cmdline_size(struct boot_info *bi, int offset)
> +{
> +#ifdef CONFIG_DOMAIN_BUILDER
> +    const void *fdt = bootstrap_map_bm(&bi->mods[HYPERLAUNCH_MODULE_IDX]);
> +    int size = fdt_cmdline_prop_size(fdt, offset);
> +
> +    bootstrap_unmap();
> +    return size < 0 ? 0 : size;

Use max() instead of open-coding it?

> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -189,6 +189,12 @@ static int __init process_domain_node(
>              printk("  kernel: boot module %d\n", idx);
>              bi->mods[idx].type = BOOTMOD_KERNEL;
>              bd->kernel = &bi->mods[idx];
> +
> +            /* If bootloader didn't set cmdline, see if FDT provides one. */
> +            if ( bd->kernel->cmdline_pa &&
> +                 !((char *)__va(bd->kernel->cmdline_pa))[0] )
> +                bd->kernel->fdt_cmdline = fdt_get_prop_offset(
> +                    fdt, node, "bootargs", &bd->kernel->cmdline_pa);

Somewhat orthogonal question: Should there perhaps be a way for the boot loader
provided cmdline to go at the tail of the DT provided one?

> --- a/xen/arch/x86/domain-builder/fdt.h
> +++ b/xen/arch/x86/domain-builder/fdt.h
> @@ -12,6 +12,31 @@ struct boot_info;
>  #define HYPERLAUNCH_MODULE_IDX 0
>  
>  #ifdef CONFIG_DOMAIN_BUILDER
> +
> +static inline int __init fdt_cmdline_prop_size(const void *fdt, int offset)
> +{
> +    int ret;
> +
> +    fdt_get_property_by_offset(fdt, offset, &ret);
> +
> +    return ret;
> +}
> +
> +static inline int __init fdt_cmdline_prop_copy(
> +    const void *fdt, int offset, char *cmdline, size_t size)
> +{
> +    int ret;
> +    const struct fdt_property *prop =
> +        fdt_get_property_by_offset(fdt, offset, &ret);
> +
> +    if ( ret < 0 )
> +        return ret;
> +
> +    ASSERT(size > ret);
> +
> +    return strlcpy(cmdline, prop->data, ret);
> +}

What's the rationale for these to be separate functions, rather then the code
being integrated into their sole callers? Especially for the former the extra
layer feels excessive.

> --- a/xen/arch/x86/include/asm/domain-builder.h
> +++ b/xen/arch/x86/include/asm/domain-builder.h
> @@ -3,6 +3,10 @@
>  
>  struct boot_info;
>  
> +size_t __init builder_get_cmdline_size(struct boot_info *bi, int offset);
> +int __init builder_get_cmdline(
> +    struct boot_info *bi, int offset, char *cmdline, size_t size);

No __init on declarations please.

> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -984,7 +984,10 @@ static size_t __init domain_cmdline_size(
>  {
>      size_t s = bi->kextra ? strlen(bi->kextra) : 0;
>  
> -    s += bd->kernel->cmdline_pa ? strlen(__va(bd->kernel->cmdline_pa)) : 0;
> +    if ( bd->kernel->fdt_cmdline )
> +        s += builder_get_cmdline_size(bi, bd->kernel->cmdline_pa);
> +    else
> +        s += strlen(__va(bd->kernel->cmdline_pa));

Why's the check lost for bd->kernel->cmdline_pa being non-zero?

> @@ -1047,9 +1050,12 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>          if ( !(cmdline = xzalloc_array(char, cmdline_size)) )
>              panic("Error allocating cmdline buffer for %pd\n", d);
>  
> -        if ( bd->kernel->cmdline_pa )
> +        if ( bd->kernel->fdt_cmdline )
> +            builder_get_cmdline(
> +                bi, bd->kernel->cmdline_pa, cmdline, cmdline_size);
> +        else

Same here.

>              strlcpy(cmdline,
> -                    cmdline_cook(__va(bd->kernel->cmdline_pa), bi->loader),
> +                    cmdline_cook(__va(bd->kernel->cmdline_pa),bi->loader),

The change to this line is bogus altogether.

> --- a/xen/include/xen/libfdt/libfdt-xen.h
> +++ b/xen/include/xen/libfdt/libfdt-xen.h
> @@ -23,6 +23,29 @@ static inline uint64_t  __init fdt_cell_as_u64(const fdt32_t *cell)
>      return ((uint64_t)fdt32_to_cpu(cell[0]) << 32) | fdt32_to_cpu(cell[1]);
>  }
>  
> +static inline bool __init fdt_get_prop_offset(
> +    const void *fdt, int node, const char *name, unsigned long *offset)
> +{
> +    int ret, poffset;
> +    const char *pname;
> +
> +    fdt_for_each_property_offset(poffset, fdt, node)
> +    {
> +        fdt_getprop_by_offset(fdt, poffset, &pname, &ret);
> +
> +        if ( ret < 0 )
> +            continue;
> +
> +        if ( strcmp(pname, name) == 0 )
> +        {
> +            *offset = poffset;

Variable naming looks backwards here.

Jan

Re: [PATCH v3 10/16] x86/hyperlaunch: obtain cmdline from device tree

[Alejandro Vallejo]

On Thu Apr 10, 2025 at 12:12 PM BST, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> --- a/xen/arch/x86/domain-builder/core.c
>> +++ b/xen/arch/x86/domain-builder/core.c
>> @@ -8,9 +8,37 @@
>>  #include <xen/lib.h>
>>  
>>  #include <asm/bootinfo.h>
>> +#include <asm/setup.h>
>>  
>>  #include "fdt.h"
>>  
>> +size_t __init builder_get_cmdline_size(struct boot_info *bi, int offset)
>> +{
>> +#ifdef CONFIG_DOMAIN_BUILDER
>> +    const void *fdt = bootstrap_map_bm(&bi->mods[HYPERLAUNCH_MODULE_IDX]);
>> +    int size = fdt_cmdline_prop_size(fdt, offset);
>> +
>> +    bootstrap_unmap();
>> +    return size < 0 ? 0 : size;
>
> Use max() instead of open-coding it?

Sure. On an unrelated matter, I've also removed the ifdef and relied on
DCE for v4 for this and the next helper.

>
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -189,6 +189,12 @@ static int __init process_domain_node(
>>              printk("  kernel: boot module %d\n", idx);
>>              bi->mods[idx].type = BOOTMOD_KERNEL;
>>              bd->kernel = &bi->mods[idx];
>> +
>> +            /* If bootloader didn't set cmdline, see if FDT provides one. */
>> +            if ( bd->kernel->cmdline_pa &&
>> +                 !((char *)__va(bd->kernel->cmdline_pa))[0] )
>> +                bd->kernel->fdt_cmdline = fdt_get_prop_offset(
>> +                    fdt, node, "bootargs", &bd->kernel->cmdline_pa);
>
> Somewhat orthogonal question: Should there perhaps be a way for the boot loader
> provided cmdline to go at the tail of the DT provided one?

That would preclude the bootloader fully overriding what's on the DT.
One can always just copy the cmdline in the DT to the bootloader and
adjust whatever is necessary there for testing. Adding append behaviour
sounds more like a hindrance rather than helpful. To me at least.

>
>> --- a/xen/arch/x86/domain-builder/fdt.h
>> +++ b/xen/arch/x86/domain-builder/fdt.h
>> @@ -12,6 +12,31 @@ struct boot_info;
>>  #define HYPERLAUNCH_MODULE_IDX 0
>>  
>>  #ifdef CONFIG_DOMAIN_BUILDER
>> +
>> +static inline int __init fdt_cmdline_prop_size(const void *fdt, int offset)
>> +{
>> +    int ret;
>> +
>> +    fdt_get_property_by_offset(fdt, offset, &ret);
>> +
>> +    return ret;
>> +}
>> +
>> +static inline int __init fdt_cmdline_prop_copy(
>> +    const void *fdt, int offset, char *cmdline, size_t size)
>> +{
>> +    int ret;
>> +    const struct fdt_property *prop =
>> +        fdt_get_property_by_offset(fdt, offset, &ret);
>> +
>> +    if ( ret < 0 )
>> +        return ret;
>> +
>> +    ASSERT(size > ret);
>> +
>> +    return strlcpy(cmdline, prop->data, ret);
>> +}
>
> What's the rationale for these to be separate functions, rather then the code
> being integrated into their sole callers? Especially for the former the extra
> layer feels excessive.

I've moved them onto domain-builder/fdt.c (where I believe they were
originally?) for v4.

>
>> --- a/xen/arch/x86/include/asm/domain-builder.h
>> +++ b/xen/arch/x86/include/asm/domain-builder.h
>> @@ -3,6 +3,10 @@
>>  
>>  struct boot_info;
>>  
>> +size_t __init builder_get_cmdline_size(struct boot_info *bi, int offset);
>> +int __init builder_get_cmdline(
>> +    struct boot_info *bi, int offset, char *cmdline, size_t size);
>
> No __init on declarations please.

Ack.

>
>> --- a/xen/arch/x86/setup.c
>> +++ b/xen/arch/x86/setup.c
>> @@ -984,7 +984,10 @@ static size_t __init domain_cmdline_size(
>>  {
>>      size_t s = bi->kextra ? strlen(bi->kextra) : 0;
>>  
>> -    s += bd->kernel->cmdline_pa ? strlen(__va(bd->kernel->cmdline_pa)) : 0;
>> +    if ( bd->kernel->fdt_cmdline )
>> +        s += builder_get_cmdline_size(bi, bd->kernel->cmdline_pa);
>> +    else
>> +        s += strlen(__va(bd->kernel->cmdline_pa));
>
> Why's the check lost for bd->kernel->cmdline_pa being non-zero?

Shouldn't have been, indeed.
>
>> @@ -1047,9 +1050,12 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>>          if ( !(cmdline = xzalloc_array(char, cmdline_size)) )
>>              panic("Error allocating cmdline buffer for %pd\n", d);
>>  
>> -        if ( bd->kernel->cmdline_pa )
>> +        if ( bd->kernel->fdt_cmdline )
>> +            builder_get_cmdline(
>> +                bi, bd->kernel->cmdline_pa, cmdline, cmdline_size);
>> +        else
>
> Same here.

Ack.

>
>>              strlcpy(cmdline,
>> -                    cmdline_cook(__va(bd->kernel->cmdline_pa), bi->loader),
>> +                    cmdline_cook(__va(bd->kernel->cmdline_pa),bi->loader),
>
> The change to this line is bogus altogether.

Ugh, yes.

>
>> --- a/xen/include/xen/libfdt/libfdt-xen.h
>> +++ b/xen/include/xen/libfdt/libfdt-xen.h
>> @@ -23,6 +23,29 @@ static inline uint64_t  __init fdt_cell_as_u64(const fdt32_t *cell)
>>      return ((uint64_t)fdt32_to_cpu(cell[0]) << 32) | fdt32_to_cpu(cell[1]);
>>  }
>>  
>> +static inline bool __init fdt_get_prop_offset(
>> +    const void *fdt, int node, const char *name, unsigned long *offset)
>> +{
>> +    int ret, poffset;
>> +    const char *pname;
>> +
>> +    fdt_for_each_property_offset(poffset, fdt, node)
>> +    {
>> +        fdt_getprop_by_offset(fdt, poffset, &pname, &ret);
>> +
>> +        if ( ret < 0 )
>> +            continue;
>> +
>> +        if ( strcmp(pname, name) == 0 )
>> +        {
>> +            *offset = poffset;
>
> Variable naming looks backwards here.
>
> Jan

I think the leading p stands for "property" (for the sake of giving it a
name). But I see how that might be confusing when interpreted with a
Hungarian notation lens.

I'll invert it. It doesn't matter which is which, after all.

Cheers,
Alejandro

Re: [PATCH v3 10/16] x86/hyperlaunch: obtain cmdline from device tree

[Jan Beulich]

On 14.04.2025 16:23, Alejandro Vallejo wrote:
> On Thu Apr 10, 2025 at 12:12 PM BST, Jan Beulich wrote:
>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>> --- a/xen/arch/x86/domain-builder/fdt.c
>>> +++ b/xen/arch/x86/domain-builder/fdt.c
>>> @@ -189,6 +189,12 @@ static int __init process_domain_node(
>>>              printk("  kernel: boot module %d\n", idx);
>>>              bi->mods[idx].type = BOOTMOD_KERNEL;
>>>              bd->kernel = &bi->mods[idx];
>>> +
>>> +            /* If bootloader didn't set cmdline, see if FDT provides one. */
>>> +            if ( bd->kernel->cmdline_pa &&
>>> +                 !((char *)__va(bd->kernel->cmdline_pa))[0] )
>>> +                bd->kernel->fdt_cmdline = fdt_get_prop_offset(
>>> +                    fdt, node, "bootargs", &bd->kernel->cmdline_pa);
>>
>> Somewhat orthogonal question: Should there perhaps be a way for the boot loader
>> provided cmdline to go at the tail of the DT provided one?
> 
> That would preclude the bootloader fully overriding what's on the DT.
> One can always just copy the cmdline in the DT to the bootloader and
> adjust whatever is necessary there for testing. Adding append behaviour
> sounds more like a hindrance rather than helpful. To me at least.

Well. This is why I have been pushing for all options to also have a
"negative" form. This way you can override whatever specifically you
need to override, without re-typing the entire (perhaps long) cmdline
from DT.

Also, I didn't mean that to necessarily be the one-and-only behavior.

Jan

[PATCH v3 11/16] x86/hyperlaunch: locate dom0 initrd with hyperlaunch

[Alejandro Vallejo]

From: "Daniel P. Smith" <dpsmith@apertussolutions.com>

Look for a subnode of type `multiboot,ramdisk` within a domain node and
parse via the fdt_read_multiboot_module() helper. After a successful
helper call, the module index is returned and the module is guaranteed
to be in the module list.

Fix unused typo in adjacent comment.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
Signed-off-by: Alejandro Vallejo <agarciav@amd.com>
---
v3:
    * Reworded commit message to state the helper postconditions.
    * Wrapped long line
    * Fix ramdisk -> module rename
    * Move ramdisk parsing from later patch
    * Remove initrdidx indent
---
 xen/arch/x86/domain-builder/fdt.c | 29 +++++++++++++++++++++++++++++
 xen/arch/x86/setup.c              |  4 ++--
 2 files changed, 31 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
index bc9903a9de..0f5fd01557 100644
--- a/xen/arch/x86/domain-builder/fdt.c
+++ b/xen/arch/x86/domain-builder/fdt.c
@@ -195,6 +195,35 @@ static int __init process_domain_node(
                  !((char *)__va(bd->kernel->cmdline_pa))[0] )
                 bd->kernel->fdt_cmdline = fdt_get_prop_offset(
                     fdt, node, "bootargs", &bd->kernel->cmdline_pa);
+
+            continue;
+        }
+        else if ( fdt_node_check_compatible(fdt, node,
+                                            "multiboot,ramdisk") == 0 )
+        {
+            int idx;
+
+            if ( bd->module )
+            {
+                printk(XENLOG_ERR "Duplicate ramdisk module for domain %s)\n",
+                       name);
+                continue;
+            }
+
+            idx = fdt_read_multiboot_module(fdt, node, address_cells,
+                                            size_cells,bi);
+            if ( idx < 0 )
+            {
+                printk("  failed processing ramdisk module for domain %s\n",
+                       name);
+                return -EINVAL;
+            }
+
+            printk("  ramdisk: boot module %d\n", idx);
+            bi->mods[idx].type = BOOTMOD_RAMDISK;
+            bd->module = &bi->mods[idx];
+
+            continue;
         }
     }
 
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index ca4415d020..3dfa81b48c 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -2149,11 +2149,11 @@ void asmlinkage __init noreturn __start_xen(void)
      * At this point all capabilities that consume boot modules should have
      * claimed their boot modules. Find the first unclaimed boot module and
      * claim it as the initrd ramdisk. Do a second search to see if there are
-     * any remaining unclaimed boot modules, and report them as unusued initrd
+     * any remaining unclaimed boot modules, and report them as unused initrd
      * candidates.
      */
     initrdidx = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
-    if ( initrdidx < MAX_NR_BOOTMODS )
+    if ( !bi->hyperlaunch_enabled && initrdidx < MAX_NR_BOOTMODS )
     {
         bi->mods[initrdidx].type = BOOTMOD_RAMDISK;
         bi->domains[0].module = &bi->mods[initrdidx];
-- 
2.43.0

Re: [PATCH v3 11/16] x86/hyperlaunch: locate dom0 initrd with hyperlaunch

[Denis Mukhin]

On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:

> 
> 
> From: "Daniel P. Smith" dpsmith@apertussolutions.com
> 
> 
> Look for a subnode of type `multiboot,ramdisk` within a domain node and
> parse via the fdt_read_multiboot_module() helper. After a successful
> helper call, the module index is returned and the module is guaranteed
> to be in the module list.
> 
> Fix unused typo in adjacent comment.
> 
> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
> 
> Signed-off-by: Jason Andryuk jason.andryuk@amd.com
> 
> Signed-off-by: Alejandro Vallejo agarciav@amd.com
> 
> ---
> v3:
> * Reworded commit message to state the helper postconditions.
> * Wrapped long line
> * Fix ramdisk -> module rename
> 
> * Move ramdisk parsing from later patch
> * Remove initrdidx indent
> ---
> xen/arch/x86/domain-builder/fdt.c | 29 +++++++++++++++++++++++++++++
> xen/arch/x86/setup.c | 4 ++--
> 2 files changed, 31 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
> index bc9903a9de..0f5fd01557 100644
> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -195,6 +195,35 @@ static int __init process_domain_node(
> !((char *)__va(bd->kernel->cmdline_pa))[0] )
> 
> bd->kernel->fdt_cmdline = fdt_get_prop_offset(
> 
> fdt, node, "bootargs", &bd->kernel->cmdline_pa);
> 
> +
> + continue;
> + }
> + else if ( fdt_node_check_compatible(fdt, node,
> + "multiboot,ramdisk") == 0 )
> + {
> + int idx;
> +
> + if ( bd->module )
> 
> + {
> + printk(XENLOG_ERR "Duplicate ramdisk module for domain %s)\n",

I would start the message with lower case so it is consistent with the other one.

> + name);
> + continue;
> + }
> +
> + idx = fdt_read_multiboot_module(fdt, node, address_cells,
> + size_cells,bi);
> + if ( idx < 0 )
> + {
> + printk(" failed processing ramdisk module for domain %s\n",
> + name);

Prepend the log message with XENLOG_ERR ?

> + return -EINVAL;
> + }
> +
> + printk(" ramdisk: boot module %d\n", idx);
> + bi->mods[idx].type = BOOTMOD_RAMDISK;
> 
> + bd->module = &bi->mods[idx];
> 
> +
> + continue;
> }
> }
> 
> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
> index ca4415d020..3dfa81b48c 100644
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -2149,11 +2149,11 @@ void asmlinkage __init noreturn __start_xen(void)
> * At this point all capabilities that consume boot modules should have
> * claimed their boot modules. Find the first unclaimed boot module and
> * claim it as the initrd ramdisk. Do a second search to see if there are
> - * any remaining unclaimed boot modules, and report them as unusued initrd
> + * any remaining unclaimed boot modules, and report them as unused initrd
> * candidates.
> */
> initrdidx = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
> - if ( initrdidx < MAX_NR_BOOTMODS )
> + if ( !bi->hyperlaunch_enabled && initrdidx < MAX_NR_BOOTMODS )
> 
> {
> bi->mods[initrdidx].type = BOOTMOD_RAMDISK;
> 
> bi->domains[0].module = &bi->mods[initrdidx];
> 
> --
> 2.43.0

Re: [PATCH v3 11/16] x86/hyperlaunch: locate dom0 initrd with hyperlaunch

[Alejandro Vallejo]

On Wed Apr 9, 2025 at 11:07 PM BST, Denis Mukhin wrote:
> On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:
>
>> 
>> 
>> From: "Daniel P. Smith" dpsmith@apertussolutions.com
>> 
>> 
>> Look for a subnode of type `multiboot,ramdisk` within a domain node and
>> parse via the fdt_read_multiboot_module() helper. After a successful
>> helper call, the module index is returned and the module is guaranteed
>> to be in the module list.
>> 
>> Fix unused typo in adjacent comment.
>> 
>> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
>> 
>> Signed-off-by: Jason Andryuk jason.andryuk@amd.com
>> 
>> Signed-off-by: Alejandro Vallejo agarciav@amd.com
>> 
>> ---
>> v3:
>> * Reworded commit message to state the helper postconditions.
>> * Wrapped long line
>> * Fix ramdisk -> module rename
>> 
>> * Move ramdisk parsing from later patch
>> * Remove initrdidx indent
>> ---
>> xen/arch/x86/domain-builder/fdt.c | 29 +++++++++++++++++++++++++++++
>> xen/arch/x86/setup.c | 4 ++--
>> 2 files changed, 31 insertions(+), 2 deletions(-)
>> 
>> diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
>> index bc9903a9de..0f5fd01557 100644
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -195,6 +195,35 @@ static int __init process_domain_node(
>> !((char *)__va(bd->kernel->cmdline_pa))[0] )
>> 
>> bd->kernel->fdt_cmdline = fdt_get_prop_offset(
>> 
>> fdt, node, "bootargs", &bd->kernel->cmdline_pa);
>> 
>> +
>> + continue;
>> + }
>> + else if ( fdt_node_check_compatible(fdt, node,
>> + "multiboot,ramdisk") == 0 )
>> + {
>> + int idx;
>> +
>> + if ( bd->module )
>> 
>> + {
>> + printk(XENLOG_ERR "Duplicate ramdisk module for domain %s)\n",
>
> I would start the message with lower case so it is consistent with the other one.

As mentioned before, this is due to how it's meant to be rendered. This
is a standalone message, hence the uppercase (consistent with the
duplicate kernel).

Will change the XENLOG_ERR into XENLOG_WARNING though.

>
>> + name);
>> + continue;
>> + }
>> +
>> + idx = fdt_read_multiboot_module(fdt, node, address_cells,
>> + size_cells,bi);
>> + if ( idx < 0 )
>> + {
>> + printk(" failed processing ramdisk module for domain %s\n",
>> + name);
>
> Prepend the log message with XENLOG_ERR ?

Indeed.

>
>> + return -EINVAL;
>> + }
>> +
>> + printk(" ramdisk: boot module %d\n", idx);
>> + bi->mods[idx].type = BOOTMOD_RAMDISK;
>> 
>> + bd->module = &bi->mods[idx];
>> 
>> +
>> + continue;
>> }
>> }
>> 
>> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
>> index ca4415d020..3dfa81b48c 100644
>> --- a/xen/arch/x86/setup.c
>> +++ b/xen/arch/x86/setup.c
>> @@ -2149,11 +2149,11 @@ void asmlinkage __init noreturn __start_xen(void)
>> * At this point all capabilities that consume boot modules should have
>> * claimed their boot modules. Find the first unclaimed boot module and
>> * claim it as the initrd ramdisk. Do a second search to see if there are
>> - * any remaining unclaimed boot modules, and report them as unusued initrd
>> + * any remaining unclaimed boot modules, and report them as unused initrd
>> * candidates.
>> */
>> initrdidx = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
>> - if ( initrdidx < MAX_NR_BOOTMODS )
>> + if ( !bi->hyperlaunch_enabled && initrdidx < MAX_NR_BOOTMODS )
>> 
>> {
>> bi->mods[initrdidx].type = BOOTMOD_RAMDISK;
>> 
>> bi->domains[0].module = &bi->mods[initrdidx];
>> 
>> --
>> 2.43.0

Cheers,
Alejandro

Re: [PATCH v3 11/16] x86/hyperlaunch: locate dom0 initrd with hyperlaunch

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -195,6 +195,35 @@ static int __init process_domain_node(
>                   !((char *)__va(bd->kernel->cmdline_pa))[0] )
>                  bd->kernel->fdt_cmdline = fdt_get_prop_offset(
>                      fdt, node, "bootargs", &bd->kernel->cmdline_pa);
> +
> +            continue;

With this ...

> +        }
> +        else if ( fdt_node_check_compatible(fdt, node,

... no need for "else" here?

> +                                            "multiboot,ramdisk") == 0 )
> +        {
> +            int idx;
> +
> +            if ( bd->module )
> +            {
> +                printk(XENLOG_ERR "Duplicate ramdisk module for domain %s)\n",

Stray ')' in the string literal.

> +                       name);
> +                continue;
> +            }
> +
> +            idx = fdt_read_multiboot_module(fdt, node, address_cells,
> +                                            size_cells,bi);
> +            if ( idx < 0 )
> +            {
> +                printk("  failed processing ramdisk module for domain %s\n",
> +                       name);
> +                return -EINVAL;
> +            }

Along the lines of what Denis has said - please be consistent about log
messages: XENLOG_* or not, preferably no capital at the start, initial
blank padding. May apply elsewhere in the series as well.

> +            printk("  ramdisk: boot module %d\n", idx);
> +            bi->mods[idx].type = BOOTMOD_RAMDISK;
> +            bd->module = &bi->mods[idx];

The field's named "module" now, but that now ends up inconsistent with
naming used elsewhere, as is pretty noticeable here.

> +            continue;

This isn't strictly needed, is it, ...

>          }
>      }

... considering we're at the bottom of the loop?

Jan

Re: [PATCH v3 11/16] x86/hyperlaunch: locate dom0 initrd with hyperlaunch

[Alejandro Vallejo]

On Thu Apr 10, 2025 at 12:34 PM BST, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -195,6 +195,35 @@ static int __init process_domain_node(
>>                   !((char *)__va(bd->kernel->cmdline_pa))[0] )
>>                  bd->kernel->fdt_cmdline = fdt_get_prop_offset(
>>                      fdt, node, "bootargs", &bd->kernel->cmdline_pa);
>> +
>> +            continue;
>
> With this ...
>
>> +        }
>> +        else if ( fdt_node_check_compatible(fdt, node,
>
> ... no need for "else" here?

Sure

>
>> +                                            "multiboot,ramdisk") == 0 )
>> +        {
>> +            int idx;
>> +
>> +            if ( bd->module )
>> +            {
>> +                printk(XENLOG_ERR "Duplicate ramdisk module for domain %s)\n",
>
> Stray ')' in the string literal.

Ack.

>
>> +                       name);
>> +                continue;
>> +            }
>> +
>> +            idx = fdt_read_multiboot_module(fdt, node, address_cells,
>> +                                            size_cells,bi);
>> +            if ( idx < 0 )
>> +            {
>> +                printk("  failed processing ramdisk module for domain %s\n",
>> +                       name);
>> +                return -EINVAL;
>> +            }
>
> Along the lines of what Denis has said - please be consistent about log
> messages: XENLOG_* or not, preferably no capital at the start, initial
> blank padding. May apply elsewhere in the series as well.

I don't mind dropping that and making everything flat (uppercase + no
padding), but there is some consistency. Albeit, it is true the
rationale is somewhat obscure.

ATM the consistency is: "padding spaces + lowercase" when giving extra
information on hyperlaunch. It ends up creating a hyperlaunch block in
`dmesg` with a "Hyperlaunch detected" line on top so it's easier to
know what lines are hyperlaunch related and which ones aren't.

Do you have a preference for a specific reporting style?

>
>> +            printk("  ramdisk: boot module %d\n", idx);
>> +            bi->mods[idx].type = BOOTMOD_RAMDISK;
>> +            bd->module = &bi->mods[idx];
>
> The field's named "module" now, but that now ends up inconsistent with
> naming used elsewhere, as is pretty noticeable here.

Well, yes. It is confusing. Also, the DTB is called multiboot,ramdisk,
because multiboot,module is already used to detect what nodes are
expressed as multiboot,modules. I'm considering going back and calling
them ramdisk again. If anything, to avoid the ambiguity between
domain modules and multiboot modules. e.g: a kernel is a multiboot
module, but not a domain module.

>
>> +            continue;
>
> This isn't strictly needed, is it, ...
>
>>          }
>>      }
>
> ... considering we're at the bottom of the loop?

Indeed

>
> Jan

Cheers,
Alejandro

Re: [PATCH v3 11/16] x86/hyperlaunch: locate dom0 initrd with hyperlaunch

[Alejandro Vallejo]

On Mon Apr 14, 2025 at 6:06 PM BST, Alejandro Vallejo wrote:
> On Thu Apr 10, 2025 at 12:34 PM BST, Jan Beulich wrote:
>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>
>>> +            printk("  ramdisk: boot module %d\n", idx);
>>> +            bi->mods[idx].type = BOOTMOD_RAMDISK;
>>> +            bd->module = &bi->mods[idx];
>>
>> The field's named "module" now, but that now ends up inconsistent with
>> naming used elsewhere, as is pretty noticeable here.
>
> Well, yes. It is confusing. Also, the DTB is called multiboot,ramdisk,
> because multiboot,module is already used to detect what nodes are
> expressed as multiboot,modules. I'm considering going back and calling
> them ramdisk again. If anything, to avoid the ambiguity between
> domain modules and multiboot modules. e.g: a kernel is a multiboot
> module, but not a domain module.

Particularly when misc/arm/device-tree/booting.txt already states that
the initrd for dom0 ought to be provided with the "multiboot,ramdisk"
string in the "compatible" prop.  Deviating from that is just going to
make it far more annoying to unify arm and x86 in the future.  And
calling those ramdisks anything but ramdisk internally is just plain
confusing (as evidenced in the current series).

So... how frontally opposed would you be to restoring the ramdisk
nomenclature? Also, for ease of rebasing future patches it'd be far
nicer to go back to ramdisk rather than reinventing some new name.

I'm for the time being leaving things as they are (because it is a pain
to change these things) until we settle on something.

Cheers,
Alejandro

Re: [PATCH v3 11/16] x86/hyperlaunch: locate dom0 initrd with hyperlaunch

[Jan Beulich]

On 14.04.2025 19:27, Alejandro Vallejo wrote:
> On Mon Apr 14, 2025 at 6:06 PM BST, Alejandro Vallejo wrote:
>> On Thu Apr 10, 2025 at 12:34 PM BST, Jan Beulich wrote:
>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>
>>>> +            printk("  ramdisk: boot module %d\n", idx);
>>>> +            bi->mods[idx].type = BOOTMOD_RAMDISK;
>>>> +            bd->module = &bi->mods[idx];
>>>
>>> The field's named "module" now, but that now ends up inconsistent with
>>> naming used elsewhere, as is pretty noticeable here.
>>
>> Well, yes. It is confusing. Also, the DTB is called multiboot,ramdisk,
>> because multiboot,module is already used to detect what nodes are
>> expressed as multiboot,modules. I'm considering going back and calling
>> them ramdisk again. If anything, to avoid the ambiguity between
>> domain modules and multiboot modules. e.g: a kernel is a multiboot
>> module, but not a domain module.
> 
> Particularly when misc/arm/device-tree/booting.txt already states that
> the initrd for dom0 ought to be provided with the "multiboot,ramdisk"
> string in the "compatible" prop.  Deviating from that is just going to
> make it far more annoying to unify arm and x86 in the future.  And
> calling those ramdisks anything but ramdisk internally is just plain
> confusing (as evidenced in the current series).

Yet the limitation of this is quite obvious: How would you express
multiple such items? Have many "ramdisk"s? Even if some of them serve
an entirely different purpose? See how Linux has gone to tuck together
multiple CPIOs, as they can have only one thing called "ramdisk"
(which, aiui, now no longer truly is).

> So... how frontally opposed would you be to restoring the ramdisk
> nomenclature? Also, for ease of rebasing future patches it'd be far
> nicer to go back to ramdisk rather than reinventing some new name.

Well, I fear I wouldn't ack such a patch. If everyone else agrees
that "ramdisk" is the best of all names (or at least getting close),
I'd perhaps mumble over, but let it go in.

(Only partly as a joke: If we dislike "module", how about "blob" or
some such?)

Jan

Re: [PATCH v3 11/16] x86/hyperlaunch: locate dom0 initrd with hyperlaunch

[Alejandro Vallejo]

On Tue Apr 15, 2025 at 7:17 AM BST, Jan Beulich wrote:
> On 14.04.2025 19:27, Alejandro Vallejo wrote:
>> On Mon Apr 14, 2025 at 6:06 PM BST, Alejandro Vallejo wrote:
>>> On Thu Apr 10, 2025 at 12:34 PM BST, Jan Beulich wrote:
>>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>>
>>>>> +            printk("  ramdisk: boot module %d\n", idx);
>>>>> +            bi->mods[idx].type = BOOTMOD_RAMDISK;
>>>>> +            bd->module = &bi->mods[idx];
>>>>
>>>> The field's named "module" now, but that now ends up inconsistent with
>>>> naming used elsewhere, as is pretty noticeable here.
>>>
>>> Well, yes. It is confusing. Also, the DTB is called multiboot,ramdisk,
>>> because multiboot,module is already used to detect what nodes are
>>> expressed as multiboot,modules. I'm considering going back and calling
>>> them ramdisk again. If anything, to avoid the ambiguity between
>>> domain modules and multiboot modules. e.g: a kernel is a multiboot
>>> module, but not a domain module.
>> 
>> Particularly when misc/arm/device-tree/booting.txt already states that
>> the initrd for dom0 ought to be provided with the "multiboot,ramdisk"
>> string in the "compatible" prop.  Deviating from that is just going to
>> make it far more annoying to unify arm and x86 in the future.  And
>> calling those ramdisks anything but ramdisk internally is just plain
>> confusing (as evidenced in the current series).
>
> Yet the limitation of this is quite obvious: How would you express
> multiple such items?

With multiple such nodes.

  initrd1 {
      compatible = "multiboot,ramdisk", "multiboot,module"
      module-idx = <2>
  };
  initrd2 {
      compatible = "multiboot,ramdisk", "multiboot,module"
      module-idx = <3>
  };

as is done in dom0less. This is not a hypothetical, it's already
comitted.

> Have many "ramdisk"s?

As many as I require. If I was booting Xen(dom0+initrd) on
Xen(hyperlaunch), that'd be 2 (with Xen passed as the kernel).

> Even if some of them serve an entirely different purpose?

The purpose of a ramdisk/initrd is of no concern to the hypervisor. They are
specially crafted blobs consumed by kernels for purposes of bootstrap.

> See how Linux has gone to tuck together multiple CPIOs, as they can
> have only one thing called "ramdisk" (which, aiui, now no longer truly
> is).

That's an internal Linux matter. If they need 1, we'll pass 1. We do
need several such blobs, because some OSs do need them. Namely, Xen. And
AFAICS we call them modules mostly because (a) Xen does not have the
module infrastructure that Linux has, which would causes ambiguities and
(b) is typically loaded via multiboot, which does call each blob a
module.

>
>> So... how frontally opposed would you be to restoring the ramdisk
>> nomenclature? Also, for ease of rebasing future patches it'd be far
>> nicer to go back to ramdisk rather than reinventing some new name.
>
> Well, I fear I wouldn't ack such a patch. If everyone else agrees
> that "ramdisk" is the best of all names (or at least getting close),
> I'd perhaps mumble over, but let it go in.

Ok... When I send v4 I'll do so keeping the "module" rename. Meanwhile,
I'll try to think of some options. Calling Xen's modules and the booted
kernel modules the same way is just way too confusing.

I take it you have the same dislike for initrd as you do for ramdisk?

>
> (Only partly as a joke: If we dislike "module", how about "blob" or
> some such?)

Better not. Blob makes "kernel" sound like an adjective: Kernel blob.

There might be some other suitable word. I'll think about it.

Cheers,
Alejandro

Re: [PATCH v3 11/16] x86/hyperlaunch: locate dom0 initrd with hyperlaunch

[Jan Beulich]

On 15.04.2025 13:59, Alejandro Vallejo wrote:
> On Tue Apr 15, 2025 at 7:17 AM BST, Jan Beulich wrote:
>> On 14.04.2025 19:27, Alejandro Vallejo wrote:
>>> So... how frontally opposed would you be to restoring the ramdisk
>>> nomenclature? Also, for ease of rebasing future patches it'd be far
>>> nicer to go back to ramdisk rather than reinventing some new name.
>>
>> Well, I fear I wouldn't ack such a patch. If everyone else agrees
>> that "ramdisk" is the best of all names (or at least getting close),
>> I'd perhaps mumble over, but let it go in.
> 
> Ok... When I send v4 I'll do so keeping the "module" rename. Meanwhile,
> I'll try to think of some options. Calling Xen's modules and the booted
> kernel modules the same way is just way too confusing.
> 
> I take it you have the same dislike for initrd as you do for ramdisk?

Indeed.

Jan

Re: [PATCH v3 11/16] x86/hyperlaunch: locate dom0 initrd with hyperlaunch

[Daniel P. Smith]

On 4/15/25 07:59, Alejandro Vallejo wrote:
> On Tue Apr 15, 2025 at 7:17 AM BST, Jan Beulich wrote:
>> On 14.04.2025 19:27, Alejandro Vallejo wrote:
>>> On Mon Apr 14, 2025 at 6:06 PM BST, Alejandro Vallejo wrote:
>>>> On Thu Apr 10, 2025 at 12:34 PM BST, Jan Beulich wrote:
>>>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>>>
>>>>>> +            printk("  ramdisk: boot module %d\n", idx);
>>>>>> +            bi->mods[idx].type = BOOTMOD_RAMDISK;
>>>>>> +            bd->module = &bi->mods[idx];
>>>>>
>>>>> The field's named "module" now, but that now ends up inconsistent with
>>>>> naming used elsewhere, as is pretty noticeable here.
>>>>
>>>> Well, yes. It is confusing. Also, the DTB is called multiboot,ramdisk,
>>>> because multiboot,module is already used to detect what nodes are
>>>> expressed as multiboot,modules. I'm considering going back and calling
>>>> them ramdisk again. If anything, to avoid the ambiguity between
>>>> domain modules and multiboot modules. e.g: a kernel is a multiboot
>>>> module, but not a domain module.
>>>
>>> Particularly when misc/arm/device-tree/booting.txt already states that
>>> the initrd for dom0 ought to be provided with the "multiboot,ramdisk"
>>> string in the "compatible" prop.  Deviating from that is just going to
>>> make it far more annoying to unify arm and x86 in the future.  And
>>> calling those ramdisks anything but ramdisk internally is just plain
>>> confusing (as evidenced in the current series).
>>
>> Yet the limitation of this is quite obvious: How would you express
>> multiple such items?
> 
> With multiple such nodes.
> 
>    initrd1 {
>        compatible = "multiboot,ramdisk", "multiboot,module"
>        module-idx = <2>
>    };
>    initrd2 {
>        compatible = "multiboot,ramdisk", "multiboot,module"
>        module-idx = <3>
>    };
> 
> as is done in dom0less. This is not a hypothetical, it's already
> comitted.
> 
>> Have many "ramdisk"s?
> 
> As many as I require. If I was booting Xen(dom0+initrd) on
> Xen(hyperlaunch), that'd be 2 (with Xen passed as the kernel).
> 
>> Even if some of them serve an entirely different purpose?
> 
> The purpose of a ramdisk/initrd is of no concern to the hypervisor. They are
> specially crafted blobs consumed by kernels for purposes of bootstrap.
> 
>> See how Linux has gone to tuck together multiple CPIOs, as they can
>> have only one thing called "ramdisk" (which, aiui, now no longer truly
>> is).
> 
> That's an internal Linux matter. If they need 1, we'll pass 1. We do
> need several such blobs, because some OSs do need them. Namely, Xen. And
> AFAICS we call them modules mostly because (a) Xen does not have the
> module infrastructure that Linux has, which would causes ambiguities and
> (b) is typically loaded via multiboot, which does call each blob a
> module.
> 
>>
>>> So... how frontally opposed would you be to restoring the ramdisk
>>> nomenclature? Also, for ease of rebasing future patches it'd be far
>>> nicer to go back to ramdisk rather than reinventing some new name.
>>
>> Well, I fear I wouldn't ack such a patch. If everyone else agrees
>> that "ramdisk" is the best of all names (or at least getting close),
>> I'd perhaps mumble over, but let it go in.
> 
> Ok... When I send v4 I'll do so keeping the "module" rename. Meanwhile,
> I'll try to think of some options. Calling Xen's modules and the booted
> kernel modules the same way is just way too confusing.
> 
> I take it you have the same dislike for initrd as you do for ramdisk?
> 
>>
>> (Only partly as a joke: If we dislike "module", how about "blob" or
>> some such?)
> 
> Better not. Blob makes "kernel" sound like an adjective: Kernel blob.
> 
> There might be some other suitable word. I'll think about it.

And all this confusion and mess would have been avoided if left as the 
abstract ramdisk name.

v/r,
dps

Re: [PATCH v3 11/16] x86/hyperlaunch: locate dom0 initrd with hyperlaunch

[Jan Beulich]

On 14.04.2025 19:06, Alejandro Vallejo wrote:
> On Thu Apr 10, 2025 at 12:34 PM BST, Jan Beulich wrote:
>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>> --- a/xen/arch/x86/domain-builder/fdt.c
>>> +++ b/xen/arch/x86/domain-builder/fdt.c
>>> @@ -195,6 +195,35 @@ static int __init process_domain_node(
>>>                   !((char *)__va(bd->kernel->cmdline_pa))[0] )
>>>                  bd->kernel->fdt_cmdline = fdt_get_prop_offset(
>>>                      fdt, node, "bootargs", &bd->kernel->cmdline_pa);
>>> +
>>> +            continue;
>>
>> With this ...
>>
>>> +        }
>>> +        else if ( fdt_node_check_compatible(fdt, node,
>>
>> ... no need for "else" here?
> 
> Sure
> 
>>
>>> +                                            "multiboot,ramdisk") == 0 )
>>> +        {
>>> +            int idx;
>>> +
>>> +            if ( bd->module )
>>> +            {
>>> +                printk(XENLOG_ERR "Duplicate ramdisk module for domain %s)\n",

For context below, note this message.

>>> +                       name);
>>> +                continue;
>>> +            }
>>> +
>>> +            idx = fdt_read_multiboot_module(fdt, node, address_cells,
>>> +                                            size_cells,bi);
>>> +            if ( idx < 0 )
>>> +            {
>>> +                printk("  failed processing ramdisk module for domain %s\n",
>>> +                       name);
>>> +                return -EINVAL;
>>> +            }
>>
>> Along the lines of what Denis has said - please be consistent about log
>> messages: XENLOG_* or not, preferably no capital at the start, initial
>> blank padding. May apply elsewhere in the series as well.
> 
> I don't mind dropping that and making everything flat (uppercase + no
> padding), but there is some consistency. Albeit, it is true the
> rationale is somewhat obscure.

Obscure or not - it might be fine if stated that way sufficiently
prominently, such that future additions here then will adhere to that
model.

> ATM the consistency is: "padding spaces + lowercase" when giving extra
> information on hyperlaunch. It ends up creating a hyperlaunch block in
> `dmesg` with a "Hyperlaunch detected" line on top so it's easier to
> know what lines are hyperlaunch related and which ones aren't.
> 
> Do you have a preference for a specific reporting style?

XENLOG_* or not want to be consistent, no matter what. Generally I think
that log messages shouldn't start with capitals, unless it's e.g. an
acronym. The padding to help grouping would be fine with me if, as said,
properly spelled as the scheme to use.

Jan

[PATCH v3 12/16] x86/hyperlaunch: add domain id parsing to domain config

[Alejandro Vallejo]

From: "Daniel P. Smith" <dpsmith@apertussolutions.com>

Introduce the ability to specify the desired domain id for the domain
definition. The domain id will be populated in the domid property of the
domain
node in the device tree configuration.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
---
v3:
    * Remove ramdisk parsing
    * Add missing xen/errno.h include
---
 xen/arch/x86/domain-builder/fdt.c   | 39 ++++++++++++++++++++++++++++-
 xen/arch/x86/setup.c                |  5 ++--
 xen/include/xen/libfdt/libfdt-xen.h | 11 ++++++++
 3 files changed, 52 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
index 0f5fd01557..4c6aafe195 100644
--- a/xen/arch/x86/domain-builder/fdt.c
+++ b/xen/arch/x86/domain-builder/fdt.c
@@ -8,6 +8,7 @@
 #include <xen/libfdt/libfdt.h>
 
 #include <asm/bootinfo.h>
+#include <asm/guest.h>
 #include <asm/page.h>
 #include <asm/setup.h>
 
@@ -158,12 +159,42 @@ int __init fdt_read_multiboot_module(const void *fdt, int node,
 static int __init process_domain_node(
     struct boot_info *bi, const void *fdt, int dom_node)
 {
-    int node;
+    int node, property;
     struct boot_domain *bd = &bi->domains[bi->nr_domains];
     const char *name = fdt_get_name(fdt, dom_node, NULL) ?: "unknown";
     int address_cells = fdt_address_cells(fdt, dom_node);
     int size_cells = fdt_size_cells(fdt, dom_node);
 
+    fdt_for_each_property_offset(property, fdt, dom_node)
+    {
+        const struct fdt_property *prop;
+        const char *prop_name;
+        int name_len;
+
+        prop = fdt_get_property_by_offset(fdt, property, NULL);
+        if ( !prop )
+            continue; /* silently skip */
+
+        prop_name = fdt_get_string(fdt, fdt32_to_cpu(prop->nameoff), &name_len);
+
+        if ( strncmp(prop_name, "domid", name_len) == 0 )
+        {
+            uint32_t val = DOMID_INVALID;
+            if ( fdt_prop_as_u32(prop, &val) != 0 )
+            {
+                printk("  failed processing domain id for domain %s\n", name);
+                return -EINVAL;
+            }
+            if ( val >= DOMID_FIRST_RESERVED )
+            {
+                printk("  invalid domain id for domain %s\n", name);
+                return -EINVAL;
+            }
+            bd->domid = (domid_t)val;
+            printk("  domid: %d\n", bd->domid);
+        }
+    }
+
     fdt_for_each_subnode(node, fdt, dom_node)
     {
         if ( fdt_node_check_compatible(fdt, node, "multiboot,kernel") == 0 )
@@ -233,6 +264,12 @@ static int __init process_domain_node(
         return -ENODATA;
     }
 
+    if ( bd->domid == DOMID_INVALID )
+        bd->domid = get_initial_domain_id();
+    else if ( bd->domid != get_initial_domain_id() )
+        printk(XENLOG_WARNING
+               "WARN: Booting without initial domid not supported.\n");
+
     return 0;
 }
 
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 3dfa81b48c..db7280225e 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1033,8 +1033,9 @@ static struct domain *__init create_dom0(struct boot_info *bi)
     if ( iommu_enabled )
         dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
 
-    /* Create initial domain.  Not d0 for pvshim. */
-    bd->domid = get_initial_domain_id();
+    if ( bd->domid == DOMID_INVALID )
+        /* Create initial domain.  Not d0 for pvshim. */
+        bd->domid = get_initial_domain_id();
     d = domain_create(bd->domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
     if ( IS_ERR(d) )
         panic("Error creating d%u: %ld\n", bd->domid, PTR_ERR(d));
diff --git a/xen/include/xen/libfdt/libfdt-xen.h b/xen/include/xen/libfdt/libfdt-xen.h
index e473fbaf0c..3031bec90e 100644
--- a/xen/include/xen/libfdt/libfdt-xen.h
+++ b/xen/include/xen/libfdt/libfdt-xen.h
@@ -12,6 +12,7 @@
 #define LIBFDT_XEN_H
 
 #include <xen/libfdt/libfdt.h>
+#include <xen/errno.h>
 
 static inline int __init fdt_cell_as_u32(const fdt32_t *cell)
 {
@@ -23,6 +24,16 @@ static inline uint64_t  __init fdt_cell_as_u64(const fdt32_t *cell)
     return ((uint64_t)fdt32_to_cpu(cell[0]) << 32) | fdt32_to_cpu(cell[1]);
 }
 
+static inline int __init fdt_prop_as_u32(
+    const struct fdt_property *prop, uint32_t *val)
+{
+    if ( !prop || fdt32_to_cpu(prop->len) < sizeof(u32) )
+        return -EINVAL;
+
+    *val = fdt_cell_as_u32((fdt32_t *)prop->data);
+    return 0;
+}
+
 static inline bool __init fdt_get_prop_offset(
     const void *fdt, int node, const char *name, unsigned long *offset)
 {
-- 
2.43.0

Re: [PATCH v3 12/16] x86/hyperlaunch: add domain id parsing to domain config

[Denis Mukhin]

On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:

> 
> 
> From: "Daniel P. Smith" dpsmith@apertussolutions.com
> 
> 
> Introduce the ability to specify the desired domain id for the domain
> definition. The domain id will be populated in the domid property of the
> domain
> node in the device tree configuration.
> 
> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
> 
> ---
> v3:
> * Remove ramdisk parsing
> * Add missing xen/errno.h include
> ---
> xen/arch/x86/domain-builder/fdt.c | 39 ++++++++++++++++++++++++++++-
> xen/arch/x86/setup.c | 5 ++--
> xen/include/xen/libfdt/libfdt-xen.h | 11 ++++++++
> 3 files changed, 52 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
> index 0f5fd01557..4c6aafe195 100644
> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -8,6 +8,7 @@
> #include <xen/libfdt/libfdt.h>
> 
> 
> #include <asm/bootinfo.h>
> 
> +#include <asm/guest.h>
> 
> #include <asm/page.h>
> 
> #include <asm/setup.h>
> 
> 
> @@ -158,12 +159,42 @@ int __init fdt_read_multiboot_module(const void *fdt, int node,
> static int __init process_domain_node(
> struct boot_info *bi, const void *fdt, int dom_node)
> {
> - int node;
> + int node, property;
> struct boot_domain *bd = &bi->domains[bi->nr_domains];
> 
> const char *name = fdt_get_name(fdt, dom_node, NULL) ?: "unknown";
> int address_cells = fdt_address_cells(fdt, dom_node);
> int size_cells = fdt_size_cells(fdt, dom_node);
> 
> + fdt_for_each_property_offset(property, fdt, dom_node)
> + {
> + const struct fdt_property *prop;
> + const char prop_name;
> + int name_len;
> +
> + prop = fdt_get_property_by_offset(fdt, property, NULL);
> + if ( !prop )
> + continue; / silently skip */
> +
> + prop_name = fdt_get_string(fdt, fdt32_to_cpu(prop->nameoff), &name_len);
> 
> +
> + if ( strncmp(prop_name, "domid", name_len) == 0 )
> + {
> + uint32_t val = DOMID_INVALID;
> + if ( fdt_prop_as_u32(prop, &val) != 0 )
> + {
> + printk(" failed processing domain id for domain %s\n", name);

Add XENLOG_ERR ?

> + return -EINVAL;
> + }
> + if ( val >= DOMID_FIRST_RESERVED )
> 
> + {
> + printk(" invalid domain id for domain %s\n", name);

Add XENLOG_ERR ?

> + return -EINVAL;
> + }
> + bd->domid = (domid_t)val;
> 
> + printk(" domid: %d\n", bd->domid);
> 
> + }
> + }
> +
> fdt_for_each_subnode(node, fdt, dom_node)
> {
> if ( fdt_node_check_compatible(fdt, node, "multiboot,kernel") == 0 )
> @@ -233,6 +264,12 @@ static int __init process_domain_node(
> return -ENODATA;
> }
> 
> + if ( bd->domid == DOMID_INVALID )
> 
> + bd->domid = get_initial_domain_id();
> 
> + else if ( bd->domid != get_initial_domain_id() )
> 
> + printk(XENLOG_WARNING
> + "WARN: Booting without initial domid not supported.\n");

Drop WARN since the log message is XENLOG_WARNING level already?

> +
> return 0;
> }
> 
> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
> index 3dfa81b48c..db7280225e 100644
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -1033,8 +1033,9 @@ static struct domain *__init create_dom0(struct boot_info bi)
> if ( iommu_enabled )
> dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
> 
> - / Create initial domain. Not d0 for pvshim. */
> - bd->domid = get_initial_domain_id();
> 
> + if ( bd->domid == DOMID_INVALID )
> 
> + /* Create initial domain. Not d0 for pvshim. */
> + bd->domid = get_initial_domain_id();
> 
> d = domain_create(bd->domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
> 
> if ( IS_ERR(d) )
> panic("Error creating d%u: %ld\n", bd->domid, PTR_ERR(d));
> 
> diff --git a/xen/include/xen/libfdt/libfdt-xen.h b/xen/include/xen/libfdt/libfdt-xen.h
> index e473fbaf0c..3031bec90e 100644
> --- a/xen/include/xen/libfdt/libfdt-xen.h
> +++ b/xen/include/xen/libfdt/libfdt-xen.h
> @@ -12,6 +12,7 @@
> #define LIBFDT_XEN_H
> 
> #include <xen/libfdt/libfdt.h>
> 
> +#include <xen/errno.h>
> 
> 
> static inline int __init fdt_cell_as_u32(const fdt32_t *cell)
> {
> @@ -23,6 +24,16 @@ static inline uint64_t __init fdt_cell_as_u64(const fdt32_t *cell)
> return ((uint64_t)fdt32_to_cpu(cell[0]) << 32) | fdt32_to_cpu(cell[1]);
> }
> 
> +static inline int __init fdt_prop_as_u32(
> + const struct fdt_property *prop, uint32_t *val)
> +{
> + if ( !prop || fdt32_to_cpu(prop->len) < sizeof(u32) )
> 
> + return -EINVAL;
> +
> + *val = fdt_cell_as_u32((fdt32_t *)prop->data);
> 
> + return 0;
> +}
> +
> static inline bool __init fdt_get_prop_offset(
> const void *fdt, int node, const char *name, unsigned long *offset)
> {
> --
> 2.43.0

Re: [PATCH v3 12/16] x86/hyperlaunch: add domain id parsing to domain config

[Alejandro Vallejo]

On Wed Apr 9, 2025 at 11:15 PM BST, Denis Mukhin wrote:
> On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:
>
>> 
>> 
>> From: "Daniel P. Smith" dpsmith@apertussolutions.com
>> 
>> 
>> Introduce the ability to specify the desired domain id for the domain
>> definition. The domain id will be populated in the domid property of the
>> domain
>> node in the device tree configuration.
>> 
>> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
>> 
>> ---
>> v3:
>> * Remove ramdisk parsing
>> * Add missing xen/errno.h include
>> ---
>> xen/arch/x86/domain-builder/fdt.c | 39 ++++++++++++++++++++++++++++-
>> xen/arch/x86/setup.c | 5 ++--
>> xen/include/xen/libfdt/libfdt-xen.h | 11 ++++++++
>> 3 files changed, 52 insertions(+), 3 deletions(-)
>> 
>> diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
>> index 0f5fd01557..4c6aafe195 100644
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -8,6 +8,7 @@
>> #include <xen/libfdt/libfdt.h>
>> 
>> 
>> #include <asm/bootinfo.h>
>> 
>> +#include <asm/guest.h>
>> 
>> #include <asm/page.h>
>> 
>> #include <asm/setup.h>
>> 
>> 
>> @@ -158,12 +159,42 @@ int __init fdt_read_multiboot_module(const void *fdt, int node,
>> static int __init process_domain_node(
>> struct boot_info *bi, const void *fdt, int dom_node)
>> {
>> - int node;
>> + int node, property;
>> struct boot_domain *bd = &bi->domains[bi->nr_domains];
>> 
>> const char *name = fdt_get_name(fdt, dom_node, NULL) ?: "unknown";
>> int address_cells = fdt_address_cells(fdt, dom_node);
>> int size_cells = fdt_size_cells(fdt, dom_node);
>> 
>> + fdt_for_each_property_offset(property, fdt, dom_node)
>> + {
>> + const struct fdt_property *prop;
>> + const char prop_name;
>> + int name_len;
>> +
>> + prop = fdt_get_property_by_offset(fdt, property, NULL);
>> + if ( !prop )
>> + continue; / silently skip */
>> +
>> + prop_name = fdt_get_string(fdt, fdt32_to_cpu(prop->nameoff), &name_len);
>> 
>> +
>> + if ( strncmp(prop_name, "domid", name_len) == 0 )
>> + {
>> + uint32_t val = DOMID_INVALID;
>> + if ( fdt_prop_as_u32(prop, &val) != 0 )
>> + {
>> + printk(" failed processing domain id for domain %s\n", name);
>
> Add XENLOG_ERR ?

Yes, and...

>
>> + return -EINVAL;
>> + }
>> + if ( val >= DOMID_FIRST_RESERVED )
>> 
>> + {
>> + printk(" invalid domain id for domain %s\n", name);
>
> Add XENLOG_ERR ?

... yes.

>
>> + return -EINVAL;
>> + }
>> + bd->domid = (domid_t)val;
>> 
>> + printk(" domid: %d\n", bd->domid);
>> 
>> + }
>> + }
>> +
>> fdt_for_each_subnode(node, fdt, dom_node)
>> {
>> if ( fdt_node_check_compatible(fdt, node, "multiboot,kernel") == 0 )
>> @@ -233,6 +264,12 @@ static int __init process_domain_node(
>> return -ENODATA;
>> }
>> 
>> + if ( bd->domid == DOMID_INVALID )
>> 
>> + bd->domid = get_initial_domain_id();
>> 
>> + else if ( bd->domid != get_initial_domain_id() )
>> 
>> + printk(XENLOG_WARNING
>> + "WARN: Booting without initial domid not supported.\n");
>
> Drop WARN since the log message is XENLOG_WARNING level already?

As mentioned elsewhere, the point of those prefixes are to be readable.

Though I'm starting to get urges to rewrite many of this error handlers
as asserts, on the basis that "why do we think it's ok to boot with
malformed DTBs"? A safe system that doesn't boot is more helpful than an
unsafe one that boots everything except a critical component for you to
find later on.

Cheers,
Alejandro

Re: [PATCH v3 12/16] x86/hyperlaunch: add domain id parsing to domain config

[Stefano Stabellini]

On Mon, 14 Apr 2025, Alejandro Vallejo wrote:
> Though I'm starting to get urges to rewrite many of this error handlers
> as asserts, on the basis that "why do we think it's ok to boot with
> malformed DTBs"? A safe system that doesn't boot is more helpful than an
> unsafe one that boots everything except a critical component for you to
> find later on.

It is totally OK to panic on boot if a malformed DTB was passed.  See
the number of panics in xen/arch/arm/dom0less-build.c.

Re: [PATCH v3 12/16] x86/hyperlaunch: add domain id parsing to domain config

[Jan Beulich]

On 14.04.2025 20:07, Alejandro Vallejo wrote:
> On Wed Apr 9, 2025 at 11:15 PM BST, Denis Mukhin wrote:
>> On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:
>>> @@ -233,6 +264,12 @@ static int __init process_domain_node(
>>> return -ENODATA;
>>> }
>>>
>>> + if ( bd->domid == DOMID_INVALID )
>>>
>>> + bd->domid = get_initial_domain_id();
>>>
>>> + else if ( bd->domid != get_initial_domain_id() )
>>>
>>> + printk(XENLOG_WARNING
>>> + "WARN: Booting without initial domid not supported.\n");
>>
>> Drop WARN since the log message is XENLOG_WARNING level already?
> 
> As mentioned elsewhere, the point of those prefixes are to be readable.

This, however, imo is a matter of consistency across the codebase, not just
within hyperlaunch. Plus (again imo) if anything, prefixes that are part of
the log output should contain proper words ("Warning:" or "Error:"), and
they shouldn't needlessly "shout" (i.e. "FATAL:" is okay-ish to be all caps,
but the others aren't).

Jan

Re: [PATCH v3 12/16] x86/hyperlaunch: add domain id parsing to domain config

[Alejandro Vallejo]

On Tue Apr 15, 2025 at 7:21 AM BST, Jan Beulich wrote:
> On 14.04.2025 20:07, Alejandro Vallejo wrote:
>> On Wed Apr 9, 2025 at 11:15 PM BST, Denis Mukhin wrote:
>>> On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:
>>>> @@ -233,6 +264,12 @@ static int __init process_domain_node(
>>>> return -ENODATA;
>>>> }
>>>>
>>>> + if ( bd->domid == DOMID_INVALID )
>>>>
>>>> + bd->domid = get_initial_domain_id();
>>>>
>>>> + else if ( bd->domid != get_initial_domain_id() )
>>>>
>>>> + printk(XENLOG_WARNING
>>>> + "WARN: Booting without initial domid not supported.\n");
>>>
>>> Drop WARN since the log message is XENLOG_WARNING level already?
>> 
>> As mentioned elsewhere, the point of those prefixes are to be readable.
>
> This, however, imo is a matter of consistency across the codebase, not just
> within hyperlaunch.

I agree. There is precedent though for certain printks to have a
collective pattern for ease of reading (e.g: spec_ctrl.c when printing
mitigations). With I'm merely justifying the 2 spaces followed by
lowercase.

I did try to remove them and it was strictly harder to know what they
referred to.

> Plus (again imo) if anything, prefixes that are part of
> the log output should contain proper words ("Warning:" or "Error:"), and
> they shouldn't needlessly "shout" (i.e. "FATAL:" is okay-ish to be all caps,
> but the others aren't).
>
> Jan

Right. I'm happy to rewrite them as "Warning: ..." and "Error: ..."

Cheers,
Alejandro

Re: [PATCH v3 12/16] x86/hyperlaunch: add domain id parsing to domain config

[Jan Beulich]

On 15.04.2025 13:37, Alejandro Vallejo wrote:
> On Tue Apr 15, 2025 at 7:21 AM BST, Jan Beulich wrote:
>> On 14.04.2025 20:07, Alejandro Vallejo wrote:
>>> On Wed Apr 9, 2025 at 11:15 PM BST, Denis Mukhin wrote:
>>>> On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:
>>>>> @@ -233,6 +264,12 @@ static int __init process_domain_node(
>>>>> return -ENODATA;
>>>>> }
>>>>>
>>>>> + if ( bd->domid == DOMID_INVALID )
>>>>>
>>>>> + bd->domid = get_initial_domain_id();
>>>>>
>>>>> + else if ( bd->domid != get_initial_domain_id() )
>>>>>
>>>>> + printk(XENLOG_WARNING
>>>>> + "WARN: Booting without initial domid not supported.\n");
>>>>
>>>> Drop WARN since the log message is XENLOG_WARNING level already?
>>>
>>> As mentioned elsewhere, the point of those prefixes are to be readable.
>>
>> This, however, imo is a matter of consistency across the codebase, not just
>> within hyperlaunch.
> 
> I agree. There is precedent though for certain printks to have a
> collective pattern for ease of reading (e.g: spec_ctrl.c when printing
> mitigations). With I'm merely justifying the 2 spaces followed by
> lowercase.
> 
> I did try to remove them and it was strictly harder to know what they
> referred to.
> 
>> Plus (again imo) if anything, prefixes that are part of
>> the log output should contain proper words ("Warning:" or "Error:"), and
>> they shouldn't needlessly "shout" (i.e. "FATAL:" is okay-ish to be all caps,
>> but the others aren't).
> 
> Right. I'm happy to rewrite them as "Warning: ..." and "Error: ..."

FTAOD - in the common case I'd prefer such prefixes to be omitted. Which
still means that there may be special cases where having them is warranted.

Jan

Re: [PATCH v3 12/16] x86/hyperlaunch: add domain id parsing to domain config

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
> 
> Introduce the ability to specify the desired domain id for the domain
> definition. The domain id will be populated in the domid property of the
> domain
> node in the device tree configuration.

Nit: Odd splitting of lines.

> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -8,6 +8,7 @@
>  #include <xen/libfdt/libfdt.h>
>  
>  #include <asm/bootinfo.h>
> +#include <asm/guest.h>

What is this needed for?

> @@ -158,12 +159,42 @@ int __init fdt_read_multiboot_module(const void *fdt, int node,
>  static int __init process_domain_node(
>      struct boot_info *bi, const void *fdt, int dom_node)
>  {
> -    int node;
> +    int node, property;
>      struct boot_domain *bd = &bi->domains[bi->nr_domains];
>      const char *name = fdt_get_name(fdt, dom_node, NULL) ?: "unknown";
>      int address_cells = fdt_address_cells(fdt, dom_node);
>      int size_cells = fdt_size_cells(fdt, dom_node);
>  
> +    fdt_for_each_property_offset(property, fdt, dom_node)
> +    {
> +        const struct fdt_property *prop;
> +        const char *prop_name;
> +        int name_len;
> +
> +        prop = fdt_get_property_by_offset(fdt, property, NULL);
> +        if ( !prop )
> +            continue; /* silently skip */
> +
> +        prop_name = fdt_get_string(fdt, fdt32_to_cpu(prop->nameoff), &name_len);
> +
> +        if ( strncmp(prop_name, "domid", name_len) == 0 )
> +        {
> +            uint32_t val = DOMID_INVALID;
> +            if ( fdt_prop_as_u32(prop, &val) != 0 )

Nit: Blank line please between declaration(s) and statement(s).

> +            {
> +                printk("  failed processing domain id for domain %s\n", name);
> +                return -EINVAL;
> +            }
> +            if ( val >= DOMID_FIRST_RESERVED )
> +            {
> +                printk("  invalid domain id for domain %s\n", name);
> +                return -EINVAL;
> +            }
> +            bd->domid = (domid_t)val;

And a conflict with other domains' IDs will not be complained about?

> +            printk("  domid: %d\n", bd->domid);

If the error messages log "name" for (I suppose) disambiguation, why would
the success message here not also do so?

> @@ -233,6 +264,12 @@ static int __init process_domain_node(
>          return -ENODATA;
>      }
>  
> +    if ( bd->domid == DOMID_INVALID )
> +        bd->domid = get_initial_domain_id();
> +    else if ( bd->domid != get_initial_domain_id() )
> +        printk(XENLOG_WARNING
> +               "WARN: Booting without initial domid not supported.\n");

I'm not a native speaker, but (or perhaps because of that) "without" feels
wrong here.

Also nit: No full stops please at the end of log messages, at least in the
common case.

Is the resolving of DOMID_INVALID invalid really needed both here and ...

> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -1033,8 +1033,9 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>      if ( iommu_enabled )
>          dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
>  
> -    /* Create initial domain.  Not d0 for pvshim. */
> -    bd->domid = get_initial_domain_id();
> +    if ( bd->domid == DOMID_INVALID )
> +        /* Create initial domain.  Not d0 for pvshim. */
> +        bd->domid = get_initial_domain_id();

... here?

> @@ -23,6 +24,16 @@ static inline uint64_t  __init fdt_cell_as_u64(const fdt32_t *cell)
>      return ((uint64_t)fdt32_to_cpu(cell[0]) << 32) | fdt32_to_cpu(cell[1]);
>  }
>  
> +static inline int __init fdt_prop_as_u32(
> +    const struct fdt_property *prop, uint32_t *val)
> +{
> +    if ( !prop || fdt32_to_cpu(prop->len) < sizeof(u32) )
> +        return -EINVAL;
> +
> +    *val = fdt_cell_as_u32((fdt32_t *)prop->data);
> +    return 0;
> +}

Path 08 looks to (partly) open-code this. Perhaps better to introduce already
there?

Jan

Re: [PATCH v3 12/16] x86/hyperlaunch: add domain id parsing to domain config

[Alejandro Vallejo]

On Thu Apr 10, 2025 at 12:49 PM BST, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>> 
>> Introduce the ability to specify the desired domain id for the domain
>> definition. The domain id will be populated in the domid property of the
>> domain
>> node in the device tree configuration.
>
> Nit: Odd splitting of lines.

Fixed

>
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -8,6 +8,7 @@
>>  #include <xen/libfdt/libfdt.h>
>>  
>>  #include <asm/bootinfo.h>
>> +#include <asm/guest.h>
>
> What is this needed for?

get_initial_domain_id(), but that ought to come from xen/domain.h instead.

Fixed.

>
>> @@ -158,12 +159,42 @@ int __init fdt_read_multiboot_module(const void *fdt, int node,
>>  static int __init process_domain_node(
>>      struct boot_info *bi, const void *fdt, int dom_node)
>>  {
>> -    int node;
>> +    int node, property;
>>      struct boot_domain *bd = &bi->domains[bi->nr_domains];
>>      const char *name = fdt_get_name(fdt, dom_node, NULL) ?: "unknown";
>>      int address_cells = fdt_address_cells(fdt, dom_node);
>>      int size_cells = fdt_size_cells(fdt, dom_node);
>>  
>> +    fdt_for_each_property_offset(property, fdt, dom_node)
>> +    {
>> +        const struct fdt_property *prop;
>> +        const char *prop_name;
>> +        int name_len;
>> +
>> +        prop = fdt_get_property_by_offset(fdt, property, NULL);
>> +        if ( !prop )
>> +            continue; /* silently skip */
>> +
>> +        prop_name = fdt_get_string(fdt, fdt32_to_cpu(prop->nameoff), &name_len);
>> +
>> +        if ( strncmp(prop_name, "domid", name_len) == 0 )
>> +        {
>> +            uint32_t val = DOMID_INVALID;
>> +            if ( fdt_prop_as_u32(prop, &val) != 0 )
>
> Nit: Blank line please between declaration(s) and statement(s).

Ack.

>
>> +            {
>> +                printk("  failed processing domain id for domain %s\n", name);
>> +                return -EINVAL;
>> +            }
>> +            if ( val >= DOMID_FIRST_RESERVED )
>> +            {
>> +                printk("  invalid domain id for domain %s\n", name);
>> +                return -EINVAL;
>> +            }
>> +            bd->domid = (domid_t)val;
>
> And a conflict with other domains' IDs will not be complained about?

Hmmm... sure, I can iterate the domlist and check.

>
>> +            printk("  domid: %d\n", bd->domid);
>
> If the error messages log "name" for (I suppose) disambiguation, why would
> the success message here not also do so?
>
>> @@ -233,6 +264,12 @@ static int __init process_domain_node(
>>          return -ENODATA;
>>      }
>>  
>> +    if ( bd->domid == DOMID_INVALID )
>> +        bd->domid = get_initial_domain_id();
>> +    else if ( bd->domid != get_initial_domain_id() )
>> +        printk(XENLOG_WARNING
>> +               "WARN: Booting without initial domid not supported.\n");
>
> I'm not a native speaker, but (or perhaps because of that) "without" feels
> wrong here.

It's probably the compound effect of without and "not supported". The
statement is correct, but it's arguably a bit obtuse.

I'll replace it with "WARN: Unsupported boot with missing initial domid.".

As for the first branch of that clause... I'm not sure we want to
support running with DTs that are missing the domid property.

>
> Also nit: No full stops please at the end of log messages, at least in the
> common case.

Ack

>
> Is the resolving of DOMID_INVALID invalid really needed both here and ...
>
>> --- a/xen/arch/x86/setup.c
>> +++ b/xen/arch/x86/setup.c
>> @@ -1033,8 +1033,9 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>>      if ( iommu_enabled )
>>          dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
>>  
>> -    /* Create initial domain.  Not d0 for pvshim. */
>> -    bd->domid = get_initial_domain_id();
>> +    if ( bd->domid == DOMID_INVALID )
>> +        /* Create initial domain.  Not d0 for pvshim. */
>> +        bd->domid = get_initial_domain_id();
>
> ... here?

I'll rationatise all that on v4.

>
>> @@ -23,6 +24,16 @@ static inline uint64_t  __init fdt_cell_as_u64(const fdt32_t *cell)
>>      return ((uint64_t)fdt32_to_cpu(cell[0]) << 32) | fdt32_to_cpu(cell[1]);
>>  }
>>  
>> +static inline int __init fdt_prop_as_u32(
>> +    const struct fdt_property *prop, uint32_t *val)
>> +{
>> +    if ( !prop || fdt32_to_cpu(prop->len) < sizeof(u32) )
>> +        return -EINVAL;
>> +
>> +    *val = fdt_cell_as_u32((fdt32_t *)prop->data);
>> +    return 0;
>> +}
>
> Path 08 looks to (partly) open-code this. Perhaps better to introduce already
> there?

Already done.

Cheers,
Alejandro

Re: [PATCH v3 12/16] x86/hyperlaunch: add domain id parsing to domain config

[Jan Beulich]

On 14.04.2025 20:35, Alejandro Vallejo wrote:
> On Thu Apr 10, 2025 at 12:49 PM BST, Jan Beulich wrote:
>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>> @@ -158,12 +159,42 @@ int __init fdt_read_multiboot_module(const void *fdt, int node,
>>>  static int __init process_domain_node(
>>>      struct boot_info *bi, const void *fdt, int dom_node)
>>>  {
>>> -    int node;
>>> +    int node, property;
>>>      struct boot_domain *bd = &bi->domains[bi->nr_domains];
>>>      const char *name = fdt_get_name(fdt, dom_node, NULL) ?: "unknown";
>>>      int address_cells = fdt_address_cells(fdt, dom_node);
>>>      int size_cells = fdt_size_cells(fdt, dom_node);
>>>  
>>> +    fdt_for_each_property_offset(property, fdt, dom_node)
>>> +    {
>>> +        const struct fdt_property *prop;
>>> +        const char *prop_name;
>>> +        int name_len;
>>> +
>>> +        prop = fdt_get_property_by_offset(fdt, property, NULL);
>>> +        if ( !prop )
>>> +            continue; /* silently skip */
>>> +
>>> +        prop_name = fdt_get_string(fdt, fdt32_to_cpu(prop->nameoff), &name_len);
>>> +
>>> +        if ( strncmp(prop_name, "domid", name_len) == 0 )
>>> +        {
>>> +            uint32_t val = DOMID_INVALID;
>>> +            if ( fdt_prop_as_u32(prop, &val) != 0 )
>>> +            {
>>> +                printk("  failed processing domain id for domain %s\n", name);
>>> +                return -EINVAL;
>>> +            }
>>> +            if ( val >= DOMID_FIRST_RESERVED )
>>> +            {
>>> +                printk("  invalid domain id for domain %s\n", name);
>>> +                return -EINVAL;
>>> +            }
>>> +            bd->domid = (domid_t)val;
>>
>> And a conflict with other domains' IDs will not be complained about?
> 
> Hmmm... sure, I can iterate the domlist and check.

Well, just to clarify: The checking doesn't necessarily need to happen here
and now. It may also happen as domains are actually created. Yet then I
think a pointer there (in a code comment) would be helpful here.

>>> @@ -233,6 +264,12 @@ static int __init process_domain_node(
>>>          return -ENODATA;
>>>      }
>>>  
>>> +    if ( bd->domid == DOMID_INVALID )
>>> +        bd->domid = get_initial_domain_id();
>>> +    else if ( bd->domid != get_initial_domain_id() )
>>> +        printk(XENLOG_WARNING
>>> +               "WARN: Booting without initial domid not supported.\n");
>>
>> I'm not a native speaker, but (or perhaps because of that) "without" feels
>> wrong here.
> 
> It's probably the compound effect of without and "not supported". The
> statement is correct, but it's arguably a bit obtuse.
> 
> I'll replace it with "WARN: Unsupported boot with missing initial domid.".

But that still doesn't fit the check, which compares the given ID (i.e.
there's nothing "missing" here) with the expected one. The "no ID given"
is handled by the plain if() that's first.

> As for the first branch of that clause... I'm not sure we want to
> support running with DTs that are missing the domid property.

This I can't really judge on.

Jan

Re: [PATCH v3 12/16] x86/hyperlaunch: add domain id parsing to domain config

[Alejandro Vallejo]

On Tue Apr 15, 2025 at 7:27 AM BST, Jan Beulich wrote:
> On 14.04.2025 20:35, Alejandro Vallejo wrote:
>> On Thu Apr 10, 2025 at 12:49 PM BST, Jan Beulich wrote:
>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>>> @@ -158,12 +159,42 @@ int __init fdt_read_multiboot_module(const void *fdt, int node,
>>>>  static int __init process_domain_node(
>>>>      struct boot_info *bi, const void *fdt, int dom_node)
>>>>  {
>>>> -    int node;
>>>> +    int node, property;
>>>>      struct boot_domain *bd = &bi->domains[bi->nr_domains];
>>>>      const char *name = fdt_get_name(fdt, dom_node, NULL) ?: "unknown";
>>>>      int address_cells = fdt_address_cells(fdt, dom_node);
>>>>      int size_cells = fdt_size_cells(fdt, dom_node);
>>>>  
>>>> +    fdt_for_each_property_offset(property, fdt, dom_node)
>>>> +    {
>>>> +        const struct fdt_property *prop;
>>>> +        const char *prop_name;
>>>> +        int name_len;
>>>> +
>>>> +        prop = fdt_get_property_by_offset(fdt, property, NULL);
>>>> +        if ( !prop )
>>>> +            continue; /* silently skip */
>>>> +
>>>> +        prop_name = fdt_get_string(fdt, fdt32_to_cpu(prop->nameoff), &name_len);
>>>> +
>>>> +        if ( strncmp(prop_name, "domid", name_len) == 0 )
>>>> +        {
>>>> +            uint32_t val = DOMID_INVALID;
>>>> +            if ( fdt_prop_as_u32(prop, &val) != 0 )
>>>> +            {
>>>> +                printk("  failed processing domain id for domain %s\n", name);
>>>> +                return -EINVAL;
>>>> +            }
>>>> +            if ( val >= DOMID_FIRST_RESERVED )
>>>> +            {
>>>> +                printk("  invalid domain id for domain %s\n", name);
>>>> +                return -EINVAL;
>>>> +            }
>>>> +            bd->domid = (domid_t)val;
>>>
>>> And a conflict with other domains' IDs will not be complained about?
>> 
>> Hmmm... sure, I can iterate the domlist and check.
>
> Well, just to clarify: The checking doesn't necessarily need to happen here
> and now. It may also happen as domains are actually created. Yet then I
> think a pointer there (in a code comment) would be helpful here.

That'd be fairly unsafe. In the case of parallel boot it'd be
indeterminate which VMs end up running if they happen to have a domid
clash. It's better to detect the error earlier and crash before any get
to start up.

>
>>>> @@ -233,6 +264,12 @@ static int __init process_domain_node(
>>>>          return -ENODATA;
>>>>      }
>>>>  
>>>> +    if ( bd->domid == DOMID_INVALID )
>>>> +        bd->domid = get_initial_domain_id();
>>>> +    else if ( bd->domid != get_initial_domain_id() )
>>>> +        printk(XENLOG_WARNING
>>>> +               "WARN: Booting without initial domid not supported.\n");
>>>
>>> I'm not a native speaker, but (or perhaps because of that) "without" feels
>>> wrong here.
>> 
>> It's probably the compound effect of without and "not supported". The
>> statement is correct, but it's arguably a bit obtuse.
>> 
>> I'll replace it with "WARN: Unsupported boot with missing initial domid.".
>
> But that still doesn't fit the check, which compares the given ID (i.e.
> there's nothing "missing" here) with the expected one. The "no ID given"
> is handled by the plain if() that's first.

It's not that the domid is missing from the node, but that the domid in
the node doesn't match the initial domid. Maybe s/domid/domain, then?

  "Warning: Unsupported boot with missing initial domain"

Cheers,
Alejandro

Re: [PATCH v3 12/16] x86/hyperlaunch: add domain id parsing to domain config

[Jan Beulich]

On 15.04.2025 14:05, Alejandro Vallejo wrote:
> On Tue Apr 15, 2025 at 7:27 AM BST, Jan Beulich wrote:
>> On 14.04.2025 20:35, Alejandro Vallejo wrote:
>>> On Thu Apr 10, 2025 at 12:49 PM BST, Jan Beulich wrote:
>>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>>>> @@ -158,12 +159,42 @@ int __init fdt_read_multiboot_module(const void *fdt, int node,
>>>>>  static int __init process_domain_node(
>>>>>      struct boot_info *bi, const void *fdt, int dom_node)
>>>>>  {
>>>>> -    int node;
>>>>> +    int node, property;
>>>>>      struct boot_domain *bd = &bi->domains[bi->nr_domains];
>>>>>      const char *name = fdt_get_name(fdt, dom_node, NULL) ?: "unknown";
>>>>>      int address_cells = fdt_address_cells(fdt, dom_node);
>>>>>      int size_cells = fdt_size_cells(fdt, dom_node);
>>>>>  
>>>>> +    fdt_for_each_property_offset(property, fdt, dom_node)
>>>>> +    {
>>>>> +        const struct fdt_property *prop;
>>>>> +        const char *prop_name;
>>>>> +        int name_len;
>>>>> +
>>>>> +        prop = fdt_get_property_by_offset(fdt, property, NULL);
>>>>> +        if ( !prop )
>>>>> +            continue; /* silently skip */
>>>>> +
>>>>> +        prop_name = fdt_get_string(fdt, fdt32_to_cpu(prop->nameoff), &name_len);
>>>>> +
>>>>> +        if ( strncmp(prop_name, "domid", name_len) == 0 )
>>>>> +        {
>>>>> +            uint32_t val = DOMID_INVALID;
>>>>> +            if ( fdt_prop_as_u32(prop, &val) != 0 )
>>>>> +            {
>>>>> +                printk("  failed processing domain id for domain %s\n", name);
>>>>> +                return -EINVAL;
>>>>> +            }
>>>>> +            if ( val >= DOMID_FIRST_RESERVED )
>>>>> +            {
>>>>> +                printk("  invalid domain id for domain %s\n", name);
>>>>> +                return -EINVAL;
>>>>> +            }
>>>>> +            bd->domid = (domid_t)val;
>>>>
>>>> And a conflict with other domains' IDs will not be complained about?
>>>
>>> Hmmm... sure, I can iterate the domlist and check.
>>
>> Well, just to clarify: The checking doesn't necessarily need to happen here
>> and now. It may also happen as domains are actually created. Yet then I
>> think a pointer there (in a code comment) would be helpful here.
> 
> That'd be fairly unsafe. In the case of parallel boot it'd be
> indeterminate which VMs end up running if they happen to have a domid
> clash. It's better to detect the error earlier and crash before any get
> to start up.

What's the unsafe aspect here? We'd crash either way; the domain(s) that
may be successfully launched wouldn't make it very far.

Yet irrespective - my request is _that_ collisions are checked for. I
don't mind much _where_ that checking lives.

>>>>> @@ -233,6 +264,12 @@ static int __init process_domain_node(
>>>>>          return -ENODATA;
>>>>>      }
>>>>>  
>>>>> +    if ( bd->domid == DOMID_INVALID )
>>>>> +        bd->domid = get_initial_domain_id();
>>>>> +    else if ( bd->domid != get_initial_domain_id() )
>>>>> +        printk(XENLOG_WARNING
>>>>> +               "WARN: Booting without initial domid not supported.\n");
>>>>
>>>> I'm not a native speaker, but (or perhaps because of that) "without" feels
>>>> wrong here.
>>>
>>> It's probably the compound effect of without and "not supported". The
>>> statement is correct, but it's arguably a bit obtuse.
>>>
>>> I'll replace it with "WARN: Unsupported boot with missing initial domid.".
>>
>> But that still doesn't fit the check, which compares the given ID (i.e.
>> there's nothing "missing" here) with the expected one. The "no ID given"
>> is handled by the plain if() that's first.
> 
> It's not that the domid is missing from the node, but that the domid in
> the node doesn't match the initial domid. Maybe s/domid/domain, then?
> 
>   "Warning: Unsupported boot with missing initial domain"

I must be missing something: When it's "don't match" why would the message
say "missing"?

Jan

[PATCH v3 13/16] x86/hyperlaunch: specify dom0 mode with device tree

[Alejandro Vallejo]

From: "Daniel P. Smith" <dpsmith@apertussolutions.com>

Enable selecting the mode in which the domain will be built and ran. This
includes:

- whether it will be either a 32/64 bit domain
- if it will be run as a PV or HVM domain
- and if it will require a device model (not applicable for dom0)

In the device tree, this will be represented as a bit map that will be carried
through into struct boot_domain.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
---
 xen/arch/x86/domain-builder/fdt.c      | 19 +++++++++++++++++++
 xen/arch/x86/include/asm/boot-domain.h |  5 +++++
 xen/arch/x86/setup.c                   |  3 ++-
 3 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
index 4c6aafe195..da65f6a5a0 100644
--- a/xen/arch/x86/domain-builder/fdt.c
+++ b/xen/arch/x86/domain-builder/fdt.c
@@ -193,6 +193,25 @@ static int __init process_domain_node(
             bd->domid = (domid_t)val;
             printk("  domid: %d\n", bd->domid);
         }
+        else if ( strncmp(prop_name, "mode", name_len) == 0 )
+        {
+            if ( fdt_prop_as_u32(prop, &bd->mode) != 0 )
+            {
+                printk("  failed processing mode for domain %s\n", name);
+                return -EINVAL;
+            }
+
+            printk("  mode: ");
+            if ( !(bd->mode & BUILD_MODE_PARAVIRT) )
+            {
+                if ( bd->mode & BUILD_MODE_ENABLE_DM )
+                    printk("HVM\n");
+                else
+                    printk("PVH\n");
+            }
+            else
+                printk("PV\n");
+        }
     }
 
     fdt_for_each_subnode(node, fdt, dom_node)
diff --git a/xen/arch/x86/include/asm/boot-domain.h b/xen/arch/x86/include/asm/boot-domain.h
index d7c6042e25..e316d4bcde 100644
--- a/xen/arch/x86/include/asm/boot-domain.h
+++ b/xen/arch/x86/include/asm/boot-domain.h
@@ -13,6 +13,11 @@
 struct boot_domain {
     domid_t domid;
 
+                                          /* On     | Off    */
+#define BUILD_MODE_PARAVIRT      (1 << 0) /* PV     | PVH/HVM */
+#define BUILD_MODE_ENABLE_DM     (1 << 1) /* HVM    | PVH     */
+    uint32_t mode;
+
     struct boot_module *kernel;
     struct boot_module *module;
     const char *cmdline;
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index db7280225e..4127a0105d 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1020,7 +1020,8 @@ static struct domain *__init create_dom0(struct boot_info *bi)
     struct boot_domain *bd = &bi->domains[0];
     struct domain *d;
 
-    if ( opt_dom0_pvh )
+    if ( opt_dom0_pvh ||
+         (bi->hyperlaunch_enabled && !(bd->mode & BUILD_MODE_PARAVIRT)) )
     {
         dom0_cfg.flags |= (XEN_DOMCTL_CDF_hvm |
                            ((hvm_hap_supported() && !opt_dom0_shadow) ?
-- 
2.43.0

Re: [PATCH v3 13/16] x86/hyperlaunch: specify dom0 mode with device tree

[Denis Mukhin]

On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:

> 
> 
> From: "Daniel P. Smith" dpsmith@apertussolutions.com
> 
> 
> Enable selecting the mode in which the domain will be built and ran. This
> includes:
> 
> - whether it will be either a 32/64 bit domain
> - if it will be run as a PV or HVM domain
> - and if it will require a device model (not applicable for dom0)
> 
> In the device tree, this will be represented as a bit map that will be carried
> through into struct boot_domain.
> 
> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
> 
> Reviewed-by: Jason Andryuk jason.andryuk@amd.com
> 
> ---
> xen/arch/x86/domain-builder/fdt.c | 19 +++++++++++++++++++
> xen/arch/x86/include/asm/boot-domain.h | 5 +++++
> xen/arch/x86/setup.c | 3 ++-
> 3 files changed, 26 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
> index 4c6aafe195..da65f6a5a0 100644
> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -193,6 +193,25 @@ static int __init process_domain_node(
> bd->domid = (domid_t)val;
> 
> printk(" domid: %d\n", bd->domid);
> 
> }
> + else if ( strncmp(prop_name, "mode", name_len) == 0 )
> + {
> + if ( fdt_prop_as_u32(prop, &bd->mode) != 0 )
> 
> + {
> + printk(" failed processing mode for domain %s\n", name);
> + return -EINVAL;
> + }
> +
> + printk(" mode: ");
> + if ( !(bd->mode & BUILD_MODE_PARAVIRT) )
> 
> + {
> + if ( bd->mode & BUILD_MODE_ENABLE_DM )
> 
> + printk("HVM\n");
> + else
> + printk("PVH\n");
> + }
> + else
> + printk("PV\n");
> + }
> }

I would re-write so the positive condition is processed first, e.g.:

    if ( bd->mode & BUILD_MODE_PARAVIRT )
        printk("PV\n");
    else if ( bd->mode & BUILD_MODE_ENABLE_DM )
        printk("HVM\n");
    else
        printk("PVH\n");

I think it will reduce indentation and make code block a bit easier to read.

Also, if there are more uses for printing string representation of a
boot module mode in the future, perhaps move it to a separate function?

What do you think?

> 
> fdt_for_each_subnode(node, fdt, dom_node)
> diff --git a/xen/arch/x86/include/asm/boot-domain.h b/xen/arch/x86/include/asm/boot-domain.h
> index d7c6042e25..e316d4bcde 100644
> --- a/xen/arch/x86/include/asm/boot-domain.h
> +++ b/xen/arch/x86/include/asm/boot-domain.h
> @@ -13,6 +13,11 @@
> struct boot_domain {
> domid_t domid;
> 
> + /* On | Off /
> +#define BUILD_MODE_PARAVIRT (1 << 0) / PV | PVH/HVM /
> +#define BUILD_MODE_ENABLE_DM (1 << 1) / HVM | PVH */
> + uint32_t mode;
> +
> struct boot_module *kernel;
> struct boot_module *module;
> const char *cmdline;
> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
> index db7280225e..4127a0105d 100644
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -1020,7 +1020,8 @@ static struct domain *__init create_dom0(struct boot_info *bi)
> struct boot_domain *bd = &bi->domains[0];
> 
> struct domain *d;
> 
> - if ( opt_dom0_pvh )
> + if ( opt_dom0_pvh ||
> + (bi->hyperlaunch_enabled && !(bd->mode & BUILD_MODE_PARAVIRT)) )
> 
> {
> dom0_cfg.flags |= (XEN_DOMCTL_CDF_hvm |
> ((hvm_hap_supported() && !opt_dom0_shadow) ?
> --
> 2.43.0

Re: [PATCH v3 13/16] x86/hyperlaunch: specify dom0 mode with device tree

[Jan Beulich]

On 10.04.2025 00:24, Denis Mukhin wrote:
> On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -193,6 +193,25 @@ static int __init process_domain_node(
>> bd->domid = (domid_t)val;
>>
>> printk(" domid: %d\n", bd->domid);
>>
>> }
>> + else if ( strncmp(prop_name, "mode", name_len) == 0 )
>> + {
>> + if ( fdt_prop_as_u32(prop, &bd->mode) != 0 )
>>
>> + {
>> + printk(" failed processing mode for domain %s\n", name);
>> + return -EINVAL;
>> + }
>> +
>> + printk(" mode: ");
>> + if ( !(bd->mode & BUILD_MODE_PARAVIRT) )
>>
>> + {
>> + if ( bd->mode & BUILD_MODE_ENABLE_DM )
>>
>> + printk("HVM\n");
>> + else
>> + printk("PVH\n");
>> + }
>> + else
>> + printk("PV\n");
>> + }
>> }
> 
> I would re-write so the positive condition is processed first, e.g.:
> 
>     if ( bd->mode & BUILD_MODE_PARAVIRT )
>         printk("PV\n");
>     else if ( bd->mode & BUILD_MODE_ENABLE_DM )
>         printk("HVM\n");
>     else
>         printk("PVH\n");
> 
> I think it will reduce indentation and make code block a bit easier to read.

I agree, except it's not so much the "positive condition" but "can be written
as if/else-if sequence".

Jan

Re: [PATCH v3 13/16] x86/hyperlaunch: specify dom0 mode with device tree

[Alejandro Vallejo]

On Wed Apr 9, 2025 at 11:24 PM BST, Denis Mukhin wrote:
> On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:
>
>> 
>> 
>> From: "Daniel P. Smith" dpsmith@apertussolutions.com
>> 
>> 
>> Enable selecting the mode in which the domain will be built and ran. This
>> includes:
>> 
>> - whether it will be either a 32/64 bit domain
>> - if it will be run as a PV or HVM domain
>> - and if it will require a device model (not applicable for dom0)
>> 
>> In the device tree, this will be represented as a bit map that will be carried
>> through into struct boot_domain.
>> 
>> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
>> 
>> Reviewed-by: Jason Andryuk jason.andryuk@amd.com
>> 
>> ---
>> xen/arch/x86/domain-builder/fdt.c | 19 +++++++++++++++++++
>> xen/arch/x86/include/asm/boot-domain.h | 5 +++++
>> xen/arch/x86/setup.c | 3 ++-
>> 3 files changed, 26 insertions(+), 1 deletion(-)
>> 
>> diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
>> index 4c6aafe195..da65f6a5a0 100644
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -193,6 +193,25 @@ static int __init process_domain_node(
>> bd->domid = (domid_t)val;
>> 
>> printk(" domid: %d\n", bd->domid);
>> 
>> }
>> + else if ( strncmp(prop_name, "mode", name_len) == 0 )
>> + {
>> + if ( fdt_prop_as_u32(prop, &bd->mode) != 0 )
>> 
>> + {
>> + printk(" failed processing mode for domain %s\n", name);
>> + return -EINVAL;
>> + }
>> +
>> + printk(" mode: ");
>> + if ( !(bd->mode & BUILD_MODE_PARAVIRT) )
>> 
>> + {
>> + if ( bd->mode & BUILD_MODE_ENABLE_DM )
>> 
>> + printk("HVM\n");
>> + else
>> + printk("PVH\n");
>> + }
>> + else
>> + printk("PV\n");
>> + }
>> }
>
> I would re-write so the positive condition is processed first, e.g.:
>
>     if ( bd->mode & BUILD_MODE_PARAVIRT )
>         printk("PV\n");
>     else if ( bd->mode & BUILD_MODE_ENABLE_DM )
>         printk("HVM\n");
>     else
>         printk("PVH\n");
>
> I think it will reduce indentation and make code block a bit easier to read.
>

For sure. You're absolutely right.

> Also, if there are more uses for printing string representation of a
> boot module mode in the future, perhaps move it to a separate function?
>
> What do you think?

If there's more existing cases I'm happy to unify them, but otherwise
I'd rather keep the code inlined to avoid too much indirection.

Cheers,
Alejandro

Re: [PATCH v3 13/16] x86/hyperlaunch: specify dom0 mode with device tree

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -193,6 +193,25 @@ static int __init process_domain_node(
>              bd->domid = (domid_t)val;
>              printk("  domid: %d\n", bd->domid);
>          }
> +        else if ( strncmp(prop_name, "mode", name_len) == 0 )
> +        {
> +            if ( fdt_prop_as_u32(prop, &bd->mode) != 0 )
> +            {
> +                printk("  failed processing mode for domain %s\n", name);
> +                return -EINVAL;
> +            }
> +
> +            printk("  mode: ");
> +            if ( !(bd->mode & BUILD_MODE_PARAVIRT) )
> +            {
> +                if ( bd->mode & BUILD_MODE_ENABLE_DM )
> +                    printk("HVM\n");
> +                else
> +                    printk("PVH\n");
> +            }
> +            else
> +                printk("PV\n");
> +        }

My prior questions here remain: What's the significance of
BUILD_MODE_ENABLE_DM when set alongside BUILD_MODE_PARAVIRT? What about
any of the other bits being set?

Jan

Re: [PATCH v3 13/16] x86/hyperlaunch: specify dom0 mode with device tree

[Daniel P. Smith]

On 4/10/25 07:57, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -193,6 +193,25 @@ static int __init process_domain_node(
>>               bd->domid = (domid_t)val;
>>               printk("  domid: %d\n", bd->domid);
>>           }
>> +        else if ( strncmp(prop_name, "mode", name_len) == 0 )
>> +        {
>> +            if ( fdt_prop_as_u32(prop, &bd->mode) != 0 )
>> +            {
>> +                printk("  failed processing mode for domain %s\n", name);
>> +                return -EINVAL;
>> +            }
>> +
>> +            printk("  mode: ");
>> +            if ( !(bd->mode & BUILD_MODE_PARAVIRT) )
>> +            {
>> +                if ( bd->mode & BUILD_MODE_ENABLE_DM )
>> +                    printk("HVM\n");
>> +                else
>> +                    printk("PVH\n");
>> +            }
>> +            else
>> +                printk("PV\n");
>> +        }
> 
> My prior questions here remain: What's the significance of
> BUILD_MODE_ENABLE_DM when set alongside BUILD_MODE_PARAVIRT? What about
> any of the other bits being set?

 From boot-domain.h:
                                           /* On     | Off    */
#define BUILD_MODE_PARAVIRT      (1 << 0) /* PV     | PVH/HVM */
#define BUILD_MODE_ENABLE_DM     (1 << 1) /* HVM    | PVH     */

The logic says, if BUILD_MODE_PARAVIRT bit is not set, thus an HVM 
domain, check if BUILD_MODE_ENABLE_DM has been set. This is determin if 
the domain is what the toolstack differentiates as either an HVM or PVH 
domain. As you should know, there is no case of a PV domain requiring a 
backing device mode (DM) domain. IOW, BUILD_MODE_ENABLE_DM is only 
relevant to an HVM domain.

v/r,
dps

Re: [PATCH v3 13/16] x86/hyperlaunch: specify dom0 mode with device tree

[Jan Beulich]

On 16.04.2025 15:32, Daniel P. Smith wrote:
> On 4/10/25 07:57, Jan Beulich wrote:
>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>> --- a/xen/arch/x86/domain-builder/fdt.c
>>> +++ b/xen/arch/x86/domain-builder/fdt.c
>>> @@ -193,6 +193,25 @@ static int __init process_domain_node(
>>>               bd->domid = (domid_t)val;
>>>               printk("  domid: %d\n", bd->domid);
>>>           }
>>> +        else if ( strncmp(prop_name, "mode", name_len) == 0 )
>>> +        {
>>> +            if ( fdt_prop_as_u32(prop, &bd->mode) != 0 )
>>> +            {
>>> +                printk("  failed processing mode for domain %s\n", name);
>>> +                return -EINVAL;
>>> +            }
>>> +
>>> +            printk("  mode: ");
>>> +            if ( !(bd->mode & BUILD_MODE_PARAVIRT) )
>>> +            {
>>> +                if ( bd->mode & BUILD_MODE_ENABLE_DM )
>>> +                    printk("HVM\n");
>>> +                else
>>> +                    printk("PVH\n");
>>> +            }
>>> +            else
>>> +                printk("PV\n");
>>> +        }
>>
>> My prior questions here remain: What's the significance of
>> BUILD_MODE_ENABLE_DM when set alongside BUILD_MODE_PARAVIRT? What about
>> any of the other bits being set?
> 
>  From boot-domain.h:
>                                            /* On     | Off    */
> #define BUILD_MODE_PARAVIRT      (1 << 0) /* PV     | PVH/HVM */
> #define BUILD_MODE_ENABLE_DM     (1 << 1) /* HVM    | PVH     */
> 
> The logic says, if BUILD_MODE_PARAVIRT bit is not set, thus an HVM 
> domain, check if BUILD_MODE_ENABLE_DM has been set. This is determin if 
> the domain is what the toolstack differentiates as either an HVM or PVH 
> domain. As you should know, there is no case of a PV domain requiring a 
> backing device mode (DM) domain. IOW, BUILD_MODE_ENABLE_DM is only 
> relevant to an HVM domain.

And hence should (my conclusion) never be set for a PV one.

Except - how wide or narrow do you mean "DM"? There are certainly cases
where a PV guest requires a qemu to serve as backend for one or more
devices. That's not what "DM" originally meant, but it goes in that
direction. Hence just to avoid such an ambiguity I think it's better to
properly reject any flags / flag combinations that we can't make sense
of.

Jan

Re: [PATCH v3 13/16] x86/hyperlaunch: specify dom0 mode with device tree

[Daniel P. Smith]

On 4/16/25 09:38, Jan Beulich wrote:
> On 16.04.2025 15:32, Daniel P. Smith wrote:
>> On 4/10/25 07:57, Jan Beulich wrote:
>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>> --- a/xen/arch/x86/domain-builder/fdt.c
>>>> +++ b/xen/arch/x86/domain-builder/fdt.c
>>>> @@ -193,6 +193,25 @@ static int __init process_domain_node(
>>>>                bd->domid = (domid_t)val;
>>>>                printk("  domid: %d\n", bd->domid);
>>>>            }
>>>> +        else if ( strncmp(prop_name, "mode", name_len) == 0 )
>>>> +        {
>>>> +            if ( fdt_prop_as_u32(prop, &bd->mode) != 0 )
>>>> +            {
>>>> +                printk("  failed processing mode for domain %s\n", name);
>>>> +                return -EINVAL;
>>>> +            }
>>>> +
>>>> +            printk("  mode: ");
>>>> +            if ( !(bd->mode & BUILD_MODE_PARAVIRT) )
>>>> +            {
>>>> +                if ( bd->mode & BUILD_MODE_ENABLE_DM )
>>>> +                    printk("HVM\n");
>>>> +                else
>>>> +                    printk("PVH\n");
>>>> +            }
>>>> +            else
>>>> +                printk("PV\n");
>>>> +        }
>>>
>>> My prior questions here remain: What's the significance of
>>> BUILD_MODE_ENABLE_DM when set alongside BUILD_MODE_PARAVIRT? What about
>>> any of the other bits being set?
>>
>>   From boot-domain.h:
>>                                             /* On     | Off    */
>> #define BUILD_MODE_PARAVIRT      (1 << 0) /* PV     | PVH/HVM */
>> #define BUILD_MODE_ENABLE_DM     (1 << 1) /* HVM    | PVH     */
>>
>> The logic says, if BUILD_MODE_PARAVIRT bit is not set, thus an HVM
>> domain, check if BUILD_MODE_ENABLE_DM has been set. This is determin if
>> the domain is what the toolstack differentiates as either an HVM or PVH
>> domain. As you should know, there is no case of a PV domain requiring a
>> backing device mode (DM) domain. IOW, BUILD_MODE_ENABLE_DM is only
>> relevant to an HVM domain.
> 
> And hence should (my conclusion) never be set for a PV one.

Yes and?

> Except - how wide or narrow do you mean "DM"? There are certainly cases
> where a PV guest requires a qemu to serve as backend for one or more
> devices. That's not what "DM" originally meant, but it goes in that
> direction. Hence just to avoid such an ambiguity I think it's better to
> properly reject any flags / flag combinations that we can't make sense
> of.

OpenXT has done this for since it was XenClientXT, and those are driver 
domains. We do distinguish between the two because a device model domain 
is specific to provide the larger core device plane to a domain. While a 
driver domain is a much narrower scope of providing a specific emulated 
hardware device to one or more domains. As a result, they have totally 
different security policies applied to them.

v/r,
dps

Re: [PATCH v3 13/16] x86/hyperlaunch: specify dom0 mode with device tree

[Jan Beulich]

On 16.04.2025 16:09, Daniel P. Smith wrote:
> On 4/16/25 09:38, Jan Beulich wrote:
>> On 16.04.2025 15:32, Daniel P. Smith wrote:
>>> On 4/10/25 07:57, Jan Beulich wrote:
>>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>>> --- a/xen/arch/x86/domain-builder/fdt.c
>>>>> +++ b/xen/arch/x86/domain-builder/fdt.c
>>>>> @@ -193,6 +193,25 @@ static int __init process_domain_node(
>>>>>                bd->domid = (domid_t)val;
>>>>>                printk("  domid: %d\n", bd->domid);
>>>>>            }
>>>>> +        else if ( strncmp(prop_name, "mode", name_len) == 0 )
>>>>> +        {
>>>>> +            if ( fdt_prop_as_u32(prop, &bd->mode) != 0 )
>>>>> +            {
>>>>> +                printk("  failed processing mode for domain %s\n", name);
>>>>> +                return -EINVAL;
>>>>> +            }
>>>>> +
>>>>> +            printk("  mode: ");
>>>>> +            if ( !(bd->mode & BUILD_MODE_PARAVIRT) )
>>>>> +            {
>>>>> +                if ( bd->mode & BUILD_MODE_ENABLE_DM )
>>>>> +                    printk("HVM\n");
>>>>> +                else
>>>>> +                    printk("PVH\n");
>>>>> +            }
>>>>> +            else
>>>>> +                printk("PV\n");
>>>>> +        }
>>>>
>>>> My prior questions here remain: What's the significance of
>>>> BUILD_MODE_ENABLE_DM when set alongside BUILD_MODE_PARAVIRT? What about
>>>> any of the other bits being set?
>>>
>>>   From boot-domain.h:
>>>                                             /* On     | Off    */
>>> #define BUILD_MODE_PARAVIRT      (1 << 0) /* PV     | PVH/HVM */
>>> #define BUILD_MODE_ENABLE_DM     (1 << 1) /* HVM    | PVH     */
>>>
>>> The logic says, if BUILD_MODE_PARAVIRT bit is not set, thus an HVM
>>> domain, check if BUILD_MODE_ENABLE_DM has been set. This is determin if
>>> the domain is what the toolstack differentiates as either an HVM or PVH
>>> domain. As you should know, there is no case of a PV domain requiring a
>>> backing device mode (DM) domain. IOW, BUILD_MODE_ENABLE_DM is only
>>> relevant to an HVM domain.
>>
>> And hence should (my conclusion) never be set for a PV one.
> 
> Yes and?

And it being wrongly set should be rejected, rather than silently ignored.

Jan

[PATCH v3 14/16] x86/hyperlaunch: add memory parsing to domain config

[Alejandro Vallejo]

From: "Daniel P. Smith" <dpsmith@apertussolutions.com>

Add three properties, memory, mem-min, and mem-max, to the domain node device
tree parsing to define the memory allocation for a domain. All three fields are
expressed in kb and written as a u64 in the device tree entries.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
---
 xen/arch/x86/dom0_build.c              |  8 ++++++
 xen/arch/x86/domain-builder/fdt.c      | 34 ++++++++++++++++++++++++++
 xen/arch/x86/include/asm/boot-domain.h |  4 +++
 xen/include/xen/libfdt/libfdt-xen.h    | 10 ++++++++
 4 files changed, 56 insertions(+)

diff --git a/xen/arch/x86/dom0_build.c b/xen/arch/x86/dom0_build.c
index 0b467fd4a4..36fb090643 100644
--- a/xen/arch/x86/dom0_build.c
+++ b/xen/arch/x86/dom0_build.c
@@ -627,6 +627,14 @@ int __init construct_dom0(const struct boot_domain *bd)
 
     process_pending_softirqs();
 
+    /* If param dom0_size was not set and HL config provided memory size */
+    if ( !get_memsize(&dom0_size, LONG_MAX) && bd->mem_pages )
+        dom0_size.nr_pages = bd->mem_pages;
+    if ( !get_memsize(&dom0_min_size, LONG_MAX) && bd->min_pages )
+        dom0_size.nr_pages = bd->min_pages;
+    if ( !get_memsize(&dom0_max_size, LONG_MAX) && bd->max_pages )
+        dom0_size.nr_pages = bd->max_pages;
+
     if ( is_hvm_domain(d) )
         rc = dom0_construct_pvh(bd);
     else if ( is_pv_domain(d) )
diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
index da65f6a5a0..338b4838c2 100644
--- a/xen/arch/x86/domain-builder/fdt.c
+++ b/xen/arch/x86/domain-builder/fdt.c
@@ -6,6 +6,7 @@
 #include <xen/init.h>
 #include <xen/lib.h>
 #include <xen/libfdt/libfdt.h>
+#include <xen/sizes.h>
 
 #include <asm/bootinfo.h>
 #include <asm/guest.h>
@@ -212,6 +213,39 @@ static int __init process_domain_node(
             else
                 printk("PV\n");
         }
+        else if ( strncmp(prop_name, "memory", name_len) == 0 )
+        {
+            uint64_t kb;
+            if ( fdt_prop_as_u64(prop, &kb) != 0 )
+            {
+                printk("  failed processing memory for domain %s\n", name);
+                return -EINVAL;
+            }
+            bd->mem_pages = PFN_DOWN(kb * SZ_1K);
+            printk("  memory: %ld kb\n", kb);
+        }
+        else if ( strncmp(prop_name, "mem-min", name_len) == 0 )
+        {
+            uint64_t kb;
+            if ( fdt_prop_as_u64(prop, &kb) != 0 )
+            {
+                printk("  failed processing memory for domain %s\n", name);
+                return -EINVAL;
+            }
+            bd->min_pages = PFN_DOWN(kb * SZ_1K);
+            printk("  min memory: %ld kb\n", kb);
+        }
+        else if ( strncmp(prop_name, "mem-max", name_len) == 0 )
+        {
+            uint64_t kb;
+            if ( fdt_prop_as_u64(prop, &kb) != 0 )
+            {
+                printk("  failed processing memory for domain %s\n", name);
+                return -EINVAL;
+            }
+            bd->max_pages = PFN_DOWN(kb * SZ_1K);
+            printk("  max memory: %ld kb\n", kb);
+        }
     }
 
     fdt_for_each_subnode(node, fdt, dom_node)
diff --git a/xen/arch/x86/include/asm/boot-domain.h b/xen/arch/x86/include/asm/boot-domain.h
index e316d4bcde..fa8ea1cc66 100644
--- a/xen/arch/x86/include/asm/boot-domain.h
+++ b/xen/arch/x86/include/asm/boot-domain.h
@@ -18,6 +18,10 @@ struct boot_domain {
 #define BUILD_MODE_ENABLE_DM     (1 << 1) /* HVM    | PVH     */
     uint32_t mode;
 
+    unsigned long mem_pages;
+    unsigned long min_pages;
+    unsigned long max_pages;
+
     struct boot_module *kernel;
     struct boot_module *module;
     const char *cmdline;
diff --git a/xen/include/xen/libfdt/libfdt-xen.h b/xen/include/xen/libfdt/libfdt-xen.h
index 3031bec90e..da43e12e38 100644
--- a/xen/include/xen/libfdt/libfdt-xen.h
+++ b/xen/include/xen/libfdt/libfdt-xen.h
@@ -34,6 +34,16 @@ static inline int __init fdt_prop_as_u32(
     return 0;
 }
 
+static inline int __init fdt_prop_as_u64(
+    const struct fdt_property *prop, uint64_t *val)
+{
+    if ( !prop || fdt32_to_cpu(prop->len) < sizeof(u64) )
+        return -EINVAL;
+
+    *val = fdt_cell_as_u64((fdt32_t *)prop->data);
+    return 0;
+}
+
 static inline bool __init fdt_get_prop_offset(
     const void *fdt, int node, const char *name, unsigned long *offset)
 {
-- 
2.43.0

Re: [PATCH v3 14/16] x86/hyperlaunch: add memory parsing to domain config

[Denis Mukhin]

On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:

> 
> 
> From: "Daniel P. Smith" dpsmith@apertussolutions.com
> 
> 
> Add three properties, memory, mem-min, and mem-max, to the domain node device
> tree parsing to define the memory allocation for a domain. All three fields are
> expressed in kb and written as a u64 in the device tree entries.
> 
> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
> 
> Reviewed-by: Jason Andryuk jason.andryuk@amd.com
> 
> ---
> xen/arch/x86/dom0_build.c | 8 ++++++
> xen/arch/x86/domain-builder/fdt.c | 34 ++++++++++++++++++++++++++
> xen/arch/x86/include/asm/boot-domain.h | 4 +++
> xen/include/xen/libfdt/libfdt-xen.h | 10 ++++++++
> 4 files changed, 56 insertions(+)
> 
> diff --git a/xen/arch/x86/dom0_build.c b/xen/arch/x86/dom0_build.c
> index 0b467fd4a4..36fb090643 100644
> --- a/xen/arch/x86/dom0_build.c
> +++ b/xen/arch/x86/dom0_build.c
> @@ -627,6 +627,14 @@ int __init construct_dom0(const struct boot_domain bd)
> 
> process_pending_softirqs();
> 
> + / If param dom0_size was not set and HL config provided memory size */
> + if ( !get_memsize(&dom0_size, LONG_MAX) && bd->mem_pages )
> 
> + dom0_size.nr_pages = bd->mem_pages;
> 
> + if ( !get_memsize(&dom0_min_size, LONG_MAX) && bd->min_pages )
> 
> + dom0_size.nr_pages = bd->min_pages;
> 
> + if ( !get_memsize(&dom0_max_size, LONG_MAX) && bd->max_pages )
> 
> + dom0_size.nr_pages = bd->max_pages;
> 
> +
> if ( is_hvm_domain(d) )
> rc = dom0_construct_pvh(bd);
> else if ( is_pv_domain(d) )
> diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
> index da65f6a5a0..338b4838c2 100644
> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -6,6 +6,7 @@
> #include <xen/init.h>
> 
> #include <xen/lib.h>
> 
> #include <xen/libfdt/libfdt.h>
> 
> +#include <xen/sizes.h>
> 
> 
> #include <asm/bootinfo.h>
> 
> #include <asm/guest.h>
> 
> @@ -212,6 +213,39 @@ static int __init process_domain_node(
> else
> printk("PV\n");
> }
> + else if ( strncmp(prop_name, "memory", name_len) == 0 )
> + {
> + uint64_t kb;
> + if ( fdt_prop_as_u64(prop, &kb) != 0 )
> + {
> + printk(" failed processing memory for domain %s\n", name);
> + return -EINVAL;
> + }
> + bd->mem_pages = PFN_DOWN(kb * SZ_1K);

Perhaps use shorter form of KB(kb) (KB() from include/xen/config.h)?

What do you think?

> 
> + printk(" memory: %ld kb\n", kb);
> + }
> + else if ( strncmp(prop_name, "mem-min", name_len) == 0 )
> + {
> + uint64_t kb;
> + if ( fdt_prop_as_u64(prop, &kb) != 0 )
> + {
> + printk(" failed processing memory for domain %s\n", name);
> + return -EINVAL;
> + }
> + bd->min_pages = PFN_DOWN(kb * SZ_1K);
> 
> + printk(" min memory: %ld kb\n", kb);
> + }
> + else if ( strncmp(prop_name, "mem-max", name_len) == 0 )
> + {
> + uint64_t kb;
> + if ( fdt_prop_as_u64(prop, &kb) != 0 )
> + {
> + printk(" failed processing memory for domain %s\n", name);
> + return -EINVAL;
> + }
> + bd->max_pages = PFN_DOWN(kb * SZ_1K);
> 
> + printk(" max memory: %ld kb\n", kb);
> + }
> }
> 
> fdt_for_each_subnode(node, fdt, dom_node)
> diff --git a/xen/arch/x86/include/asm/boot-domain.h b/xen/arch/x86/include/asm/boot-domain.h
> index e316d4bcde..fa8ea1cc66 100644
> --- a/xen/arch/x86/include/asm/boot-domain.h
> +++ b/xen/arch/x86/include/asm/boot-domain.h
> @@ -18,6 +18,10 @@ struct boot_domain {
> #define BUILD_MODE_ENABLE_DM (1 << 1) /* HVM | PVH */
> uint32_t mode;
> 
> + unsigned long mem_pages;
> + unsigned long min_pages;
> + unsigned long max_pages;
> +
> struct boot_module *kernel;
> struct boot_module *module;
> const char *cmdline;
> diff --git a/xen/include/xen/libfdt/libfdt-xen.h b/xen/include/xen/libfdt/libfdt-xen.h
> index 3031bec90e..da43e12e38 100644
> --- a/xen/include/xen/libfdt/libfdt-xen.h
> +++ b/xen/include/xen/libfdt/libfdt-xen.h
> @@ -34,6 +34,16 @@ static inline int __init fdt_prop_as_u32(
> return 0;
> }
> 
> +static inline int __init fdt_prop_as_u64(
> + const struct fdt_property *prop, uint64_t *val)
> +{
> + if ( !prop || fdt32_to_cpu(prop->len) < sizeof(u64) )
> 
> + return -EINVAL;
> +
> + *val = fdt_cell_as_u64((fdt32_t *)prop->data);
> 
> + return 0;
> +}
> +
> static inline bool __init fdt_get_prop_offset(
> const void *fdt, int node, const char *name, unsigned long *offset)
> {
> --
> 2.43.0

Re: [PATCH v3 14/16] x86/hyperlaunch: add memory parsing to domain config

[Alejandro Vallejo]

On Wed Apr 9, 2025 at 11:29 PM BST, Denis Mukhin wrote:
> On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:
>
>> 
>> 
>> From: "Daniel P. Smith" dpsmith@apertussolutions.com
>> 
>> 
>> Add three properties, memory, mem-min, and mem-max, to the domain node device
>> tree parsing to define the memory allocation for a domain. All three fields are
>> expressed in kb and written as a u64 in the device tree entries.
>> 
>> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
>> 
>> Reviewed-by: Jason Andryuk jason.andryuk@amd.com
>> 
>> ---
>> xen/arch/x86/dom0_build.c | 8 ++++++
>> xen/arch/x86/domain-builder/fdt.c | 34 ++++++++++++++++++++++++++
>> xen/arch/x86/include/asm/boot-domain.h | 4 +++
>> xen/include/xen/libfdt/libfdt-xen.h | 10 ++++++++
>> 4 files changed, 56 insertions(+)
>> 
>> diff --git a/xen/arch/x86/dom0_build.c b/xen/arch/x86/dom0_build.c
>> index 0b467fd4a4..36fb090643 100644
>> --- a/xen/arch/x86/dom0_build.c
>> +++ b/xen/arch/x86/dom0_build.c
>> @@ -627,6 +627,14 @@ int __init construct_dom0(const struct boot_domain bd)
>> 
>> process_pending_softirqs();
>> 
>> + / If param dom0_size was not set and HL config provided memory size */
>> + if ( !get_memsize(&dom0_size, LONG_MAX) && bd->mem_pages )
>> 
>> + dom0_size.nr_pages = bd->mem_pages;
>> 
>> + if ( !get_memsize(&dom0_min_size, LONG_MAX) && bd->min_pages )
>> 
>> + dom0_size.nr_pages = bd->min_pages;
>> 
>> + if ( !get_memsize(&dom0_max_size, LONG_MAX) && bd->max_pages )
>> 
>> + dom0_size.nr_pages = bd->max_pages;
>> 
>> +
>> if ( is_hvm_domain(d) )
>> rc = dom0_construct_pvh(bd);
>> else if ( is_pv_domain(d) )
>> diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
>> index da65f6a5a0..338b4838c2 100644
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -6,6 +6,7 @@
>> #include <xen/init.h>
>> 
>> #include <xen/lib.h>
>> 
>> #include <xen/libfdt/libfdt.h>
>> 
>> +#include <xen/sizes.h>
>> 
>> 
>> #include <asm/bootinfo.h>
>> 
>> #include <asm/guest.h>
>> 
>> @@ -212,6 +213,39 @@ static int __init process_domain_node(
>> else
>> printk("PV\n");
>> }
>> + else if ( strncmp(prop_name, "memory", name_len) == 0 )
>> + {
>> + uint64_t kb;
>> + if ( fdt_prop_as_u64(prop, &kb) != 0 )
>> + {
>> + printk(" failed processing memory for domain %s\n", name);
>> + return -EINVAL;
>> + }
>> + bd->mem_pages = PFN_DOWN(kb * SZ_1K);
>
> Perhaps use shorter form of KB(kb) (KB() from include/xen/config.h)?
>
> What do you think?

Sure.

Cheers,
Alejandro

Re: [PATCH v3 14/16] x86/hyperlaunch: add memory parsing to domain config

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> @@ -212,6 +213,39 @@ static int __init process_domain_node(
>              else
>                  printk("PV\n");
>          }
> +        else if ( strncmp(prop_name, "memory", name_len) == 0 )
> +        {
> +            uint64_t kb;
> +            if ( fdt_prop_as_u64(prop, &kb) != 0 )

Nit (you know what I have to say here, and again below.)

> +            {
> +                printk("  failed processing memory for domain %s\n", name);
> +                return -EINVAL;

Any reason to override fdt_prop_as_u64()'s return value here?

> +            }
> +            bd->mem_pages = PFN_DOWN(kb * SZ_1K);
> +            printk("  memory: %ld kb\n", kb);
> +        }
> +        else if ( strncmp(prop_name, "mem-min", name_len) == 0 )
> +        {
> +            uint64_t kb;
> +            if ( fdt_prop_as_u64(prop, &kb) != 0 )
> +            {
> +                printk("  failed processing memory for domain %s\n", name);
> +                return -EINVAL;
> +            }
> +            bd->min_pages = PFN_DOWN(kb * SZ_1K);
> +            printk("  min memory: %ld kb\n", kb);
> +        }
> +        else if ( strncmp(prop_name, "mem-max", name_len) == 0 )
> +        {
> +            uint64_t kb;
> +            if ( fdt_prop_as_u64(prop, &kb) != 0 )
> +            {
> +                printk("  failed processing memory for domain %s\n", name);

All three error messages being identical doesn't help diagnosing issues.

> --- a/xen/include/xen/libfdt/libfdt-xen.h
> +++ b/xen/include/xen/libfdt/libfdt-xen.h
> @@ -34,6 +34,16 @@ static inline int __init fdt_prop_as_u32(
>      return 0;
>  }
>  
> +static inline int __init fdt_prop_as_u64(
> +    const struct fdt_property *prop, uint64_t *val)
> +{
> +    if ( !prop || fdt32_to_cpu(prop->len) < sizeof(u64) )
> +        return -EINVAL;
> +
> +    *val = fdt_cell_as_u64((fdt32_t *)prop->data);

Please avoid casting away const. Looks like I overlooked this in
fdt_prop_as_u32() that was introduced by an earlier patch.

Jan

Re: [PATCH v3 14/16] x86/hyperlaunch: add memory parsing to domain config

[Alejandro Vallejo]

On Thu Apr 10, 2025 at 1:03 PM BST, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> @@ -212,6 +213,39 @@ static int __init process_domain_node(
>>              else
>>                  printk("PV\n");
>>          }
>> +        else if ( strncmp(prop_name, "memory", name_len) == 0 )
>> +        {
>> +            uint64_t kb;
>> +            if ( fdt_prop_as_u64(prop, &kb) != 0 )
>
> Nit (you know what I have to say here, and again below.)

Ack

>
>> +            {
>> +                printk("  failed processing memory for domain %s\n", name);
>> +                return -EINVAL;
>
> Any reason to override fdt_prop_as_u64()'s return value here?

Mostly to avoid needing to recover the error code. I'll just do it.

>
>> +            }
>> +            bd->mem_pages = PFN_DOWN(kb * SZ_1K);
>> +            printk("  memory: %ld kb\n", kb);
>> +        }
>> +        else if ( strncmp(prop_name, "mem-min", name_len) == 0 )
>> +        {
>> +            uint64_t kb;
>> +            if ( fdt_prop_as_u64(prop, &kb) != 0 )
>> +            {
>> +                printk("  failed processing memory for domain %s\n", name);
>> +                return -EINVAL;
>> +            }
>> +            bd->min_pages = PFN_DOWN(kb * SZ_1K);
>> +            printk("  min memory: %ld kb\n", kb);
>> +        }
>> +        else if ( strncmp(prop_name, "mem-max", name_len) == 0 )
>> +        {
>> +            uint64_t kb;
>> +            if ( fdt_prop_as_u64(prop, &kb) != 0 )
>> +            {
>> +                printk("  failed processing memory for domain %s\n", name);
>
> All three error messages being identical doesn't help diagnosing issues.

Indeed. Will add the prop that trigger each.

>
>> --- a/xen/include/xen/libfdt/libfdt-xen.h
>> +++ b/xen/include/xen/libfdt/libfdt-xen.h
>> @@ -34,6 +34,16 @@ static inline int __init fdt_prop_as_u32(
>>      return 0;
>>  }
>>  
>> +static inline int __init fdt_prop_as_u64(
>> +    const struct fdt_property *prop, uint64_t *val)
>> +{
>> +    if ( !prop || fdt32_to_cpu(prop->len) < sizeof(u64) )
>> +        return -EINVAL;
>> +
>> +    *val = fdt_cell_as_u64((fdt32_t *)prop->data);
>
> Please avoid casting away const. Looks like I overlooked this in
> fdt_prop_as_u32() that was introduced by an earlier patch.

As part of v4 I moved this and fdt_prop_as_u32() earlier to patch8 and
already adjusted accordingly.

Cheers,
Alejandro

Re: [PATCH v3 14/16] x86/hyperlaunch: add memory parsing to domain config

[Daniel P. Smith]

On 4/10/25 08:03, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> @@ -212,6 +213,39 @@ static int __init process_domain_node(
>>               else
>>                   printk("PV\n");
>>           }
>> +        else if ( strncmp(prop_name, "memory", name_len) == 0 )
>> +        {
>> +            uint64_t kb;
>> +            if ( fdt_prop_as_u64(prop, &kb) != 0 )
> 
> Nit (you know what I have to say here, and again below.)
> 
>> +            {
>> +                printk("  failed processing memory for domain %s\n", name);
>> +                return -EINVAL;
> 
> Any reason to override fdt_prop_as_u64()'s return value here?
> 

IMHO this should be a function that libfdt should provide, but altering 
libftd directly would make uprev'ing it challenging. The least I could 
do is make the function behave like the rest of libfdt's helper functions.

v/r,
dps

Re: [PATCH v3 14/16] x86/hyperlaunch: add memory parsing to domain config

[Jan Beulich]

On 16.04.2025 15:37, Daniel P. Smith wrote:
> On 4/10/25 08:03, Jan Beulich wrote:
>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>> @@ -212,6 +213,39 @@ static int __init process_domain_node(
>>>               else
>>>                   printk("PV\n");
>>>           }
>>> +        else if ( strncmp(prop_name, "memory", name_len) == 0 )
>>> +        {
>>> +            uint64_t kb;
>>> +            if ( fdt_prop_as_u64(prop, &kb) != 0 )
>>
>> Nit (you know what I have to say here, and again below.)
>>
>>> +            {
>>> +                printk("  failed processing memory for domain %s\n", name);
>>> +                return -EINVAL;
>>
>> Any reason to override fdt_prop_as_u64()'s return value here?
> 
> IMHO this should be a function that libfdt should provide, but altering 
> libftd directly would make uprev'ing it challenging. The least I could 
> do is make the function behave like the rest of libfdt's helper functions.

How's this related to the question that I raised? I didn't question the
function, but a particular aspect of the specific use that is being made
of it here.

Jan

Re: [PATCH v3 14/16] x86/hyperlaunch: add memory parsing to domain config

[Daniel P. Smith]

V/r,
Daniel P. Smith
Apertus Solutions, LLC

On 4/16/25 09:41, Jan Beulich wrote:
> On 16.04.2025 15:37, Daniel P. Smith wrote:
>> On 4/10/25 08:03, Jan Beulich wrote:
>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>> @@ -212,6 +213,39 @@ static int __init process_domain_node(
>>>>                else
>>>>                    printk("PV\n");
>>>>            }
>>>> +        else if ( strncmp(prop_name, "memory", name_len) == 0 )
>>>> +        {
>>>> +            uint64_t kb;
>>>> +            if ( fdt_prop_as_u64(prop, &kb) != 0 )
>>>
>>> Nit (you know what I have to say here, and again below.)
>>>
>>>> +            {
>>>> +                printk("  failed processing memory for domain %s\n", name);
>>>> +                return -EINVAL;
>>>
>>> Any reason to override fdt_prop_as_u64()'s return value here?
>>
>> IMHO this should be a function that libfdt should provide, but altering
>> libftd directly would make uprev'ing it challenging. The least I could
>> do is make the function behave like the rest of libfdt's helper functions.
> 
> How's this related to the question that I raised? I didn't question the
> function, but a particular aspect of the specific use that is being made
> of it here.

Your question was, "Any reason to override fdt_prop_as_u64()'s return 
value here?"

And my answer was, I copied libfdt's behavior for its helper functions. 
IOW, to have the helper behave like libfdt's other helper functions.

v/r,
dps

Re: [PATCH v3 14/16] x86/hyperlaunch: add memory parsing to domain config

[Jan Beulich]

On 16.04.2025 16:12, Daniel P. Smith wrote:
> On 4/16/25 09:41, Jan Beulich wrote:
>> On 16.04.2025 15:37, Daniel P. Smith wrote:
>>> On 4/10/25 08:03, Jan Beulich wrote:
>>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>>> @@ -212,6 +213,39 @@ static int __init process_domain_node(
>>>>>                else
>>>>>                    printk("PV\n");
>>>>>            }
>>>>> +        else if ( strncmp(prop_name, "memory", name_len) == 0 )
>>>>> +        {
>>>>> +            uint64_t kb;
>>>>> +            if ( fdt_prop_as_u64(prop, &kb) != 0 )
>>>>
>>>> Nit (you know what I have to say here, and again below.)
>>>>
>>>>> +            {
>>>>> +                printk("  failed processing memory for domain %s\n", name);
>>>>> +                return -EINVAL;
>>>>
>>>> Any reason to override fdt_prop_as_u64()'s return value here?
>>>
>>> IMHO this should be a function that libfdt should provide, but altering
>>> libftd directly would make uprev'ing it challenging. The least I could
>>> do is make the function behave like the rest of libfdt's helper functions.
>>
>> How's this related to the question that I raised? I didn't question the
>> function, but a particular aspect of the specific use that is being made
>> of it here.
> 
> Your question was, "Any reason to override fdt_prop_as_u64()'s return 
> value here?"
> 
> And my answer was, I copied libfdt's behavior for its helper functions. 
> IOW, to have the helper behave like libfdt's other helper functions.

I'm sorry, but no. It meanwhile feels like you're intentionally
misunderstanding. We're in process_domain_node() here. That's not a libfdt-
like helper function? And the question was why this function throws away
the return value it got from fdt_prop_as_u64().

Jan

[PATCH v3 15/16] x86/hyperlaunch: add max vcpu parsing of hyperlaunch device tree

[Alejandro Vallejo]

From: "Daniel P. Smith" <dpsmith@apertussolutions.com>

Introduce the `cpus` property, named as such for dom0less compatibility, that
represents the maximum number of vpcus to allocate for a domain. In the device
tree, it will be encoded as a u32 value.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
---
 xen/arch/x86/dom0_build.c              |  3 +++
 xen/arch/x86/domain-builder/fdt.c      | 11 +++++++++++
 xen/arch/x86/include/asm/boot-domain.h |  2 ++
 3 files changed, 16 insertions(+)

diff --git a/xen/arch/x86/dom0_build.c b/xen/arch/x86/dom0_build.c
index 36fb090643..7b3e31a08f 100644
--- a/xen/arch/x86/dom0_build.c
+++ b/xen/arch/x86/dom0_build.c
@@ -635,6 +635,9 @@ int __init construct_dom0(const struct boot_domain *bd)
     if ( !get_memsize(&dom0_max_size, LONG_MAX) && bd->max_pages )
         dom0_size.nr_pages = bd->max_pages;
 
+    if ( opt_dom0_max_vcpus_max == UINT_MAX && bd->max_vcpus )
+        opt_dom0_max_vcpus_max = bd->max_vcpus;
+
     if ( is_hvm_domain(d) )
         rc = dom0_construct_pvh(bd);
     else if ( is_pv_domain(d) )
diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
index 338b4838c2..5fcb767bdd 100644
--- a/xen/arch/x86/domain-builder/fdt.c
+++ b/xen/arch/x86/domain-builder/fdt.c
@@ -246,6 +246,17 @@ static int __init process_domain_node(
             bd->max_pages = PFN_DOWN(kb * SZ_1K);
             printk("  max memory: %ld kb\n", kb);
         }
+        else if ( strncmp(prop_name, "cpus", name_len) == 0 )
+        {
+            uint32_t val = UINT_MAX;
+            if ( fdt_prop_as_u32(prop, &val) != 0 )
+            {
+                printk("  failed processing max_vcpus for domain %s\n", name);
+                return -EINVAL;
+            }
+            bd->max_vcpus = val;
+            printk("  max vcpus: %d\n", bd->max_vcpus);
+        }
     }
 
     fdt_for_each_subnode(node, fdt, dom_node)
diff --git a/xen/arch/x86/include/asm/boot-domain.h b/xen/arch/x86/include/asm/boot-domain.h
index fa8ea1cc66..969c02a6ea 100644
--- a/xen/arch/x86/include/asm/boot-domain.h
+++ b/xen/arch/x86/include/asm/boot-domain.h
@@ -22,6 +22,8 @@ struct boot_domain {
     unsigned long min_pages;
     unsigned long max_pages;
 
+    unsigned int max_vcpus;
+
     struct boot_module *kernel;
     struct boot_module *module;
     const char *cmdline;
-- 
2.43.0

Re: [PATCH v3 15/16] x86/hyperlaunch: add max vcpu parsing of hyperlaunch device tree

[Denis Mukhin]

On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:

> 
> 
> From: "Daniel P. Smith" dpsmith@apertussolutions.com
> 
> 
> Introduce the `cpus` property, named as such for dom0less compatibility, that
> represents the maximum number of vpcus to allocate for a domain. In the device
> tree, it will be encoded as a u32 value.
> 
> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
> 
> Reviewed-by: Jason Andryuk jason.andryuk@amd.com
> 
> ---
> xen/arch/x86/dom0_build.c | 3 +++
> xen/arch/x86/domain-builder/fdt.c | 11 +++++++++++
> xen/arch/x86/include/asm/boot-domain.h | 2 ++
> 3 files changed, 16 insertions(+)
> 
> diff --git a/xen/arch/x86/dom0_build.c b/xen/arch/x86/dom0_build.c
> index 36fb090643..7b3e31a08f 100644
> --- a/xen/arch/x86/dom0_build.c
> +++ b/xen/arch/x86/dom0_build.c
> @@ -635,6 +635,9 @@ int __init construct_dom0(const struct boot_domain *bd)
> if ( !get_memsize(&dom0_max_size, LONG_MAX) && bd->max_pages )
> 
> dom0_size.nr_pages = bd->max_pages;
> 
> 
> + if ( opt_dom0_max_vcpus_max == UINT_MAX && bd->max_vcpus )
> 
> + opt_dom0_max_vcpus_max = bd->max_vcpus;
> 
> +
> if ( is_hvm_domain(d) )
> rc = dom0_construct_pvh(bd);
> else if ( is_pv_domain(d) )
> diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
> index 338b4838c2..5fcb767bdd 100644
> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -246,6 +246,17 @@ static int __init process_domain_node(
> bd->max_pages = PFN_DOWN(kb * SZ_1K);
> 
> printk(" max memory: %ld kb\n", kb);
> }
> + else if ( strncmp(prop_name, "cpus", name_len) == 0 )
> + {
> + uint32_t val = UINT_MAX;
> + if ( fdt_prop_as_u32(prop, &val) != 0 )
> + {
> + printk(" failed processing max_vcpus for domain %s\n", name);

Suggest adding XENLOG_ERR to the error message.

> + return -EINVAL;
> + }
> + bd->max_vcpus = val;
> 
> + printk(" max vcpus: %d\n", bd->max_vcpus);
> 
> + }
> }
> 
> fdt_for_each_subnode(node, fdt, dom_node)
> diff --git a/xen/arch/x86/include/asm/boot-domain.h b/xen/arch/x86/include/asm/boot-domain.h
> index fa8ea1cc66..969c02a6ea 100644
> --- a/xen/arch/x86/include/asm/boot-domain.h
> +++ b/xen/arch/x86/include/asm/boot-domain.h
> @@ -22,6 +22,8 @@ struct boot_domain {
> unsigned long min_pages;
> unsigned long max_pages;
> 
> + unsigned int max_vcpus;
> +
> struct boot_module *kernel;
> struct boot_module *module;
> const char *cmdline;
> --
> 2.43.0

Re: [PATCH v3 15/16] x86/hyperlaunch: add max vcpu parsing of hyperlaunch device tree

[Alejandro Vallejo]

On Wed Apr 9, 2025 at 11:33 PM BST, Denis Mukhin wrote:
> On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:
>
>> 
>> 
>> From: "Daniel P. Smith" dpsmith@apertussolutions.com
>> 
>> 
>> Introduce the `cpus` property, named as such for dom0less compatibility, that
>> represents the maximum number of vpcus to allocate for a domain. In the device
>> tree, it will be encoded as a u32 value.
>> 
>> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
>> 
>> Reviewed-by: Jason Andryuk jason.andryuk@amd.com
>> 
>> ---
>> xen/arch/x86/dom0_build.c | 3 +++
>> xen/arch/x86/domain-builder/fdt.c | 11 +++++++++++
>> xen/arch/x86/include/asm/boot-domain.h | 2 ++
>> 3 files changed, 16 insertions(+)
>> 
>> diff --git a/xen/arch/x86/dom0_build.c b/xen/arch/x86/dom0_build.c
>> index 36fb090643..7b3e31a08f 100644
>> --- a/xen/arch/x86/dom0_build.c
>> +++ b/xen/arch/x86/dom0_build.c
>> @@ -635,6 +635,9 @@ int __init construct_dom0(const struct boot_domain *bd)
>> if ( !get_memsize(&dom0_max_size, LONG_MAX) && bd->max_pages )
>> 
>> dom0_size.nr_pages = bd->max_pages;
>> 
>> 
>> + if ( opt_dom0_max_vcpus_max == UINT_MAX && bd->max_vcpus )
>> 
>> + opt_dom0_max_vcpus_max = bd->max_vcpus;
>> 
>> +
>> if ( is_hvm_domain(d) )
>> rc = dom0_construct_pvh(bd);
>> else if ( is_pv_domain(d) )
>> diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
>> index 338b4838c2..5fcb767bdd 100644
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -246,6 +246,17 @@ static int __init process_domain_node(
>> bd->max_pages = PFN_DOWN(kb * SZ_1K);
>> 
>> printk(" max memory: %ld kb\n", kb);
>> }
>> + else if ( strncmp(prop_name, "cpus", name_len) == 0 )
>> + {
>> + uint32_t val = UINT_MAX;
>> + if ( fdt_prop_as_u32(prop, &val) != 0 )
>> + {
>> + printk(" failed processing max_vcpus for domain %s\n", name);
>
> Suggest adding XENLOG_ERR to the error message.

And XENLOG_INFO to the one below.

Ack.

Cheers,
Alejandro

Re: [PATCH v3 15/16] x86/hyperlaunch: add max vcpu parsing of hyperlaunch device tree

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
> 
> Introduce the `cpus` property, named as such for dom0less compatibility, that
> represents the maximum number of vpcus to allocate for a domain. In the device

Nit: vcpus

> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -246,6 +246,17 @@ static int __init process_domain_node(
>              bd->max_pages = PFN_DOWN(kb * SZ_1K);
>              printk("  max memory: %ld kb\n", kb);
>          }
> +        else if ( strncmp(prop_name, "cpus", name_len) == 0 )
> +        {
> +            uint32_t val = UINT_MAX;
> +            if ( fdt_prop_as_u32(prop, &val) != 0 )

And again the same nit.

> +            {
> +                printk("  failed processing max_vcpus for domain %s\n", name);

There's no "max_vcpus" being processed here; that purely ...

> +                return -EINVAL;
> +            }
> +            bd->max_vcpus = val;

... the internal name we use for the struct field etc. The user observing the
message ought to be able to easily associate it back with the DT item.

Jan

Re: [PATCH v3 15/16] x86/hyperlaunch: add max vcpu parsing of hyperlaunch device tree

[Alejandro Vallejo]

On Thu Apr 10, 2025 at 1:08 PM BST, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>> 
>> Introduce the `cpus` property, named as such for dom0less compatibility, that
>> represents the maximum number of vpcus to allocate for a domain. In the device
>
> Nit: vcpus

Ack, and same below

>
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -246,6 +246,17 @@ static int __init process_domain_node(
>>              bd->max_pages = PFN_DOWN(kb * SZ_1K);
>>              printk("  max memory: %ld kb\n", kb);
>>          }
>> +        else if ( strncmp(prop_name, "cpus", name_len) == 0 )
>> +        {
>> +            uint32_t val = UINT_MAX;
>> +            if ( fdt_prop_as_u32(prop, &val) != 0 )
>
> And again the same nit.
>
>> +            {
>> +                printk("  failed processing max_vcpus for domain %s\n", name);
>
> There's no "max_vcpus" being processed here; that purely ...
>
>> +                return -EINVAL;
>> +            }
>> +            bd->max_vcpus = val;
>
> ... the internal name we use for the struct field etc. The user observing the
> message ought to be able to easily associate it back with the DT item.
>
> Jan

Very true. I agree, and will change accordingly.

Cheers,
Alejandro

Re: [PATCH v3 15/16] x86/hyperlaunch: add max vcpu parsing of hyperlaunch device tree

[Daniel P. Smith]

On 4/10/25 08:08, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>
>> Introduce the `cpus` property, named as such for dom0less compatibility, that
>> represents the maximum number of vpcus to allocate for a domain. In the device
> 
> Nit: vcpus

I agree with you here, the issue is that it was requested that we keep 
this field in line with Arm's DT, and they unfortunately used `cpus` to 
specify the vcpu allocation.

>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -246,6 +246,17 @@ static int __init process_domain_node(
>>               bd->max_pages = PFN_DOWN(kb * SZ_1K);
>>               printk("  max memory: %ld kb\n", kb);
>>           }
>> +        else if ( strncmp(prop_name, "cpus", name_len) == 0 )
>> +        {
>> +            uint32_t val = UINT_MAX;
>> +            if ( fdt_prop_as_u32(prop, &val) != 0 )
> 
> And again the same nit.
> 
>> +            {
>> +                printk("  failed processing max_vcpus for domain %s\n", name);
> 
> There's no "max_vcpus" being processed here; that purely ...
> 
>> +                return -EINVAL;
>> +            }
>> +            bd->max_vcpus = val;
> 
> ... the internal name we use for the struct field etc. The user observing the
> message ought to be able to easily associate it back with the DT item.
> 
> Jan

Re: [PATCH v3 15/16] x86/hyperlaunch: add max vcpu parsing of hyperlaunch device tree

[Alejandro Vallejo]

On Wed Apr 16, 2025 at 2:42 PM BST, Daniel P. Smith wrote:
>
> On 4/10/25 08:08, Jan Beulich wrote:
>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>>
>>> Introduce the `cpus` property, named as such for dom0less compatibility, that
>>> represents the maximum number of vpcus to allocate for a domain. In the device
                                       ^
                                       |
				     "vcpus"

>> 
>> Nit: vcpus
>
> I agree with you here, the issue is that it was requested that we keep 
> this field in line with Arm's DT, and they unfortunately used `cpus` to 
> specify the vcpu allocation.

He meant in the commit message. There's a typo.

Cheers,
Alejandro

Re: [PATCH v3 15/16] x86/hyperlaunch: add max vcpu parsing of hyperlaunch device tree

[Daniel P. Smith]

On 4/16/25 09:54, Alejandro Vallejo wrote:
> On Wed Apr 16, 2025 at 2:42 PM BST, Daniel P. Smith wrote:
>>
>> On 4/10/25 08:08, Jan Beulich wrote:
>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>>>
>>>> Introduce the `cpus` property, named as such for dom0less compatibility, that
>>>> represents the maximum number of vpcus to allocate for a domain. In the device
>                                         ^
>                                         |
> 				     "vcpus"
> 
>>>
>>> Nit: vcpus
>>
>> I agree with you here, the issue is that it was requested that we keep
>> this field in line with Arm's DT, and they unfortunately used `cpus` to
>> specify the vcpu allocation.
> 
> He meant in the commit message. There's a typo.

Right, so I guess my response should have been further down. He points 
it out as a nit the second time, and then his final comment led me to 
believe he was raising an issue with vcpu vs cpu and not just the typo.

v/r,
dps

Re: [PATCH v3 15/16] x86/hyperlaunch: add max vcpu parsing of hyperlaunch device tree

[Jan Beulich]

On 16.04.2025 15:42, Daniel P. Smith wrote:
> 
> On 4/10/25 08:08, Jan Beulich wrote:
>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>>
>>> Introduce the `cpus` property, named as such for dom0less compatibility, that
>>> represents the maximum number of vpcus to allocate for a domain. In the device
>>
>> Nit: vcpus
> 
> I agree with you here, the issue is that it was requested that we keep 
> this field in line with Arm's DT, and they unfortunately used `cpus` to 
> specify the vcpu allocation.

You misunderstood, I think. The comment was on the mis-spelling in the latter
of the quoted lines.

Jan

Re: [PATCH v3 15/16] x86/hyperlaunch: add max vcpu parsing of hyperlaunch device tree

[Daniel P. Smith]

On 4/16/25 09:54, Jan Beulich wrote:
> On 16.04.2025 15:42, Daniel P. Smith wrote:
>>
>> On 4/10/25 08:08, Jan Beulich wrote:
>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>>>
>>>> Introduce the `cpus` property, named as such for dom0less compatibility, that
>>>> represents the maximum number of vpcus to allocate for a domain. In the device
>>>
>>> Nit: vcpus
>>
>> I agree with you here, the issue is that it was requested that we keep
>> this field in line with Arm's DT, and they unfortunately used `cpus` to
>> specify the vcpu allocation.
> 
> You misunderstood, I think. The comment was on the mis-spelling in the latter
> of the quoted lines.
> 

Then your latter comment is that you want the internal field to be 
renamed to cpu? Wouldn't that create further confusion of a physical cpu 
assignment vs virtual cpu allocation?

v/r,
dps

Re: [PATCH v3 15/16] x86/hyperlaunch: add max vcpu parsing of hyperlaunch device tree

[Jan Beulich]

On 16.04.2025 16:19, Daniel P. Smith wrote:
> On 4/16/25 09:54, Jan Beulich wrote:
>> On 16.04.2025 15:42, Daniel P. Smith wrote:
>>>
>>> On 4/10/25 08:08, Jan Beulich wrote:
>>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>>>>>
>>>>> Introduce the `cpus` property, named as such for dom0less compatibility, that
>>>>> represents the maximum number of vpcus to allocate for a domain. In the device
>>>>
>>>> Nit: vcpus
>>>
>>> I agree with you here, the issue is that it was requested that we keep
>>> this field in line with Arm's DT, and they unfortunately used `cpus` to
>>> specify the vcpu allocation.
>>
>> You misunderstood, I think. The comment was on the mis-spelling in the latter
>> of the quoted lines.
> 
> Then your latter comment is that you want the internal field to be 
> renamed to cpu?

No? Where are you taking that from? My comment was that a log message refers
to "max_vcpus", when no field / property of that name is being processed.
Going back to my reply (and seeing that Alejandro understood what I meant,
afaict) I see nothing ambiguous there at all.

In any event, ftaod, there were three entirely independent comments in my
original reply to that patch.

Jan

> Wouldn't that create further confusion of a physical cpu 
> assignment vs virtual cpu allocation?
> 
> v/r,
> dps
> 

[PATCH v3 16/16] x86/hyperlaunch: add capabilities to boot domain

[Alejandro Vallejo]

From: "Daniel P. Smith" <dpsmith@apertussolutions.com>

Introduce the ability to assign capabilities to a domain via its definition in
device tree. The first capability enabled to select is the control domain
capability. The capability property is a bitfield in both the device tree and
`struct boot_domain`.

Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
---
 xen/arch/x86/domain-builder/core.c     |  1 +
 xen/arch/x86/domain-builder/fdt.c      | 12 ++++++++++++
 xen/arch/x86/include/asm/boot-domain.h |  4 ++++
 xen/arch/x86/setup.c                   |  6 +++++-
 4 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/domain-builder/core.c b/xen/arch/x86/domain-builder/core.c
index 510a74a675..6ab4e6fe53 100644
--- a/xen/arch/x86/domain-builder/core.c
+++ b/xen/arch/x86/domain-builder/core.c
@@ -96,6 +96,7 @@ void __init builder_init(struct boot_info *bi)
         i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
         bi->mods[i].type = BOOTMOD_KERNEL;
         bi->domains[0].kernel = &bi->mods[i];
+        bi->domains[0].capabilities |= BUILD_CAPS_CONTROL;
         bi->nr_domains = 1;
     }
 }
diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
index 5fcb767bdd..dbfbcffb0a 100644
--- a/xen/arch/x86/domain-builder/fdt.c
+++ b/xen/arch/x86/domain-builder/fdt.c
@@ -257,6 +257,18 @@ static int __init process_domain_node(
             bd->max_vcpus = val;
             printk("  max vcpus: %d\n", bd->max_vcpus);
         }
+        else if ( strncmp(prop_name, "capabilities", name_len) == 0 )
+        {
+            if ( fdt_prop_as_u32(prop, &bd->capabilities) != 0 )
+            {
+                printk("  failed processing domain id for domain %s\n", name);
+                return -EINVAL;
+            }
+            printk("  caps: ");
+            if ( bd->capabilities & BUILD_CAPS_CONTROL )
+                printk("c");
+            printk("\n");
+        }
     }
 
     fdt_for_each_subnode(node, fdt, dom_node)
diff --git a/xen/arch/x86/include/asm/boot-domain.h b/xen/arch/x86/include/asm/boot-domain.h
index 969c02a6ea..29a7d806de 100644
--- a/xen/arch/x86/include/asm/boot-domain.h
+++ b/xen/arch/x86/include/asm/boot-domain.h
@@ -13,6 +13,10 @@
 struct boot_domain {
     domid_t domid;
 
+#define BUILD_CAPS_NONE          (0)
+#define BUILD_CAPS_CONTROL       (1 << 0)
+    uint32_t capabilities;
+
                                           /* On     | Off    */
 #define BUILD_MODE_PARAVIRT      (1 << 0) /* PV     | PVH/HVM */
 #define BUILD_MODE_ENABLE_DM     (1 << 1) /* HVM    | PVH     */
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 4127a0105d..7e1a26b4d2 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1006,6 +1006,7 @@ static struct domain *__init create_dom0(struct boot_info *bi)
 {
     char *cmdline = NULL;
     size_t cmdline_size;
+    unsigned int create_flags = 0;
     struct xen_domctl_createdomain dom0_cfg = {
         .flags = IS_ENABLED(CONFIG_TBOOT) ? XEN_DOMCTL_CDF_s3_integrity : 0,
         .max_evtchn_port = -1,
@@ -1037,7 +1038,10 @@ static struct domain *__init create_dom0(struct boot_info *bi)
     if ( bd->domid == DOMID_INVALID )
         /* Create initial domain.  Not d0 for pvshim. */
         bd->domid = get_initial_domain_id();
-    d = domain_create(bd->domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
+    if ( bd->capabilities & BUILD_CAPS_CONTROL )
+        create_flags |= CDF_privileged;
+    d = domain_create(bd->domid, &dom0_cfg,
+                      pv_shim ? 0 : create_flags);
     if ( IS_ERR(d) )
         panic("Error creating d%u: %ld\n", bd->domid, PTR_ERR(d));
 
-- 
2.43.0

Re: [PATCH v3 16/16] x86/hyperlaunch: add capabilities to boot domain

[Denis Mukhin]

On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:

> 
> 
> From: "Daniel P. Smith" dpsmith@apertussolutions.com
> 
> 
> Introduce the ability to assign capabilities to a domain via its definition in
> device tree. The first capability enabled to select is the control domain
> capability. The capability property is a bitfield in both the device tree and
> `struct boot_domain`.
> 
> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
> 
> Reviewed-by: Jason Andryuk jason.andryuk@amd.com
> 
> Signed-off-by: Jason Andryuk jason.andryuk@amd.com
> 
> ---
> xen/arch/x86/domain-builder/core.c | 1 +
> xen/arch/x86/domain-builder/fdt.c | 12 ++++++++++++
> xen/arch/x86/include/asm/boot-domain.h | 4 ++++
> xen/arch/x86/setup.c | 6 +++++-
> 4 files changed, 22 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/x86/domain-builder/core.c b/xen/arch/x86/domain-builder/core.c
> index 510a74a675..6ab4e6fe53 100644
> --- a/xen/arch/x86/domain-builder/core.c
> +++ b/xen/arch/x86/domain-builder/core.c
> @@ -96,6 +96,7 @@ void __init builder_init(struct boot_info *bi)
> i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
> bi->mods[i].type = BOOTMOD_KERNEL;
> 
> bi->domains[0].kernel = &bi->mods[i];
> 
> + bi->domains[0].capabilities |= BUILD_CAPS_CONTROL;
> 
> bi->nr_domains = 1;
> 
> }
> }
> diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
> index 5fcb767bdd..dbfbcffb0a 100644
> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -257,6 +257,18 @@ static int __init process_domain_node(
> bd->max_vcpus = val;
> 
> printk(" max vcpus: %d\n", bd->max_vcpus);
> 
> }
> + else if ( strncmp(prop_name, "capabilities", name_len) == 0 )
> + {
> + if ( fdt_prop_as_u32(prop, &bd->capabilities) != 0 )
> 
> + {
> + printk(" failed processing domain id for domain %s\n", name);

Suggest adding XENLOG_ERR to the error message.

> + return -EINVAL;
> + }
> + printk(" caps: ");
> + if ( bd->capabilities & BUILD_CAPS_CONTROL )
> 
> + printk("c");

Perhaps wrap string generation into a separate function?

That will help if the number of capabilities will grow over time
and if there will be a need to use string representation somewhere else
in the code.

Thoughts?

> + printk("\n");
> + }
> }
> 
> fdt_for_each_subnode(node, fdt, dom_node)
> diff --git a/xen/arch/x86/include/asm/boot-domain.h b/xen/arch/x86/include/asm/boot-domain.h
> index 969c02a6ea..29a7d806de 100644
> --- a/xen/arch/x86/include/asm/boot-domain.h
> +++ b/xen/arch/x86/include/asm/boot-domain.h
> @@ -13,6 +13,10 @@
> struct boot_domain {
> domid_t domid;
> 
> +#define BUILD_CAPS_NONE (0)
> +#define BUILD_CAPS_CONTROL (1 << 0)
> + uint32_t capabilities;
> +
> /* On | Off /
> #define BUILD_MODE_PARAVIRT (1 << 0) / PV | PVH/HVM /
> #define BUILD_MODE_ENABLE_DM (1 << 1) / HVM | PVH */
> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
> index 4127a0105d..7e1a26b4d2 100644
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -1006,6 +1006,7 @@ static struct domain *__init create_dom0(struct boot_info *bi)
> {
> char *cmdline = NULL;
> size_t cmdline_size;
> + unsigned int create_flags = 0;
> struct xen_domctl_createdomain dom0_cfg = {
> .flags = IS_ENABLED(CONFIG_TBOOT) ? XEN_DOMCTL_CDF_s3_integrity : 0,
> .max_evtchn_port = -1,
> @@ -1037,7 +1038,10 @@ static struct domain *__init create_dom0(struct boot_info *bi)
> if ( bd->domid == DOMID_INVALID )
> 
> /* Create initial domain. Not d0 for pvshim. */
> bd->domid = get_initial_domain_id();
> 
> - d = domain_create(bd->domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
> 
> + if ( bd->capabilities & BUILD_CAPS_CONTROL )
> 
> + create_flags |= CDF_privileged;
> + d = domain_create(bd->domid, &dom0_cfg,
> 
> + pv_shim ? 0 : create_flags);
> if ( IS_ERR(d) )
> panic("Error creating d%u: %ld\n", bd->domid, PTR_ERR(d));
> 
> 
> --
> 2.43.0

Re: [PATCH v3 16/16] x86/hyperlaunch: add capabilities to boot domain

[Alejandro Vallejo]

On Wed Apr 9, 2025 at 11:39 PM BST, Denis Mukhin wrote:
> On Tuesday, April 8th, 2025 at 9:07 AM, Alejandro Vallejo <agarciav@amd.com> wrote:
>
>> 
>> 
>> From: "Daniel P. Smith" dpsmith@apertussolutions.com
>> 
>> 
>> Introduce the ability to assign capabilities to a domain via its definition in
>> device tree. The first capability enabled to select is the control domain
>> capability. The capability property is a bitfield in both the device tree and
>> `struct boot_domain`.
>> 
>> Signed-off-by: Daniel P. Smith dpsmith@apertussolutions.com
>> 
>> Reviewed-by: Jason Andryuk jason.andryuk@amd.com
>> 
>> Signed-off-by: Jason Andryuk jason.andryuk@amd.com
>> 
>> ---
>> xen/arch/x86/domain-builder/core.c | 1 +
>> xen/arch/x86/domain-builder/fdt.c | 12 ++++++++++++
>> xen/arch/x86/include/asm/boot-domain.h | 4 ++++
>> xen/arch/x86/setup.c | 6 +++++-
>> 4 files changed, 22 insertions(+), 1 deletion(-)
>> 
>> diff --git a/xen/arch/x86/domain-builder/core.c b/xen/arch/x86/domain-builder/core.c
>> index 510a74a675..6ab4e6fe53 100644
>> --- a/xen/arch/x86/domain-builder/core.c
>> +++ b/xen/arch/x86/domain-builder/core.c
>> @@ -96,6 +96,7 @@ void __init builder_init(struct boot_info *bi)
>> i = first_boot_module_index(bi, BOOTMOD_UNKNOWN);
>> bi->mods[i].type = BOOTMOD_KERNEL;
>> 
>> bi->domains[0].kernel = &bi->mods[i];
>> 
>> + bi->domains[0].capabilities |= BUILD_CAPS_CONTROL;
>> 
>> bi->nr_domains = 1;
>> 
>> }
>> }
>> diff --git a/xen/arch/x86/domain-builder/fdt.c b/xen/arch/x86/domain-builder/fdt.c
>> index 5fcb767bdd..dbfbcffb0a 100644
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -257,6 +257,18 @@ static int __init process_domain_node(
>> bd->max_vcpus = val;
>> 
>> printk(" max vcpus: %d\n", bd->max_vcpus);
>> 
>> }
>> + else if ( strncmp(prop_name, "capabilities", name_len) == 0 )
>> + {
>> + if ( fdt_prop_as_u32(prop, &bd->capabilities) != 0 )
>> 
>> + {
>> + printk(" failed processing domain id for domain %s\n", name);
>
> Suggest adding XENLOG_ERR to the error message.

Yes, and the message itself seems bogus. The dangers of copy-paste...

Will fix both.

>
>> + return -EINVAL;
>> + }
>> + printk(" caps: ");
>> + if ( bd->capabilities & BUILD_CAPS_CONTROL )
>> 
>> + printk("c");
>
> Perhaps wrap string generation into a separate function?
>
> That will help if the number of capabilities will grow over time
> and if there will be a need to use string representation somewhere else
> in the code.
>
> Thoughts?

If/when such other code appears I'm happy to unify them, but until then
I'd rather reduce indirection if possible and keep it inlined.

Cheers,
Alejandro

Re: [PATCH v3 16/16] x86/hyperlaunch: add capabilities to boot domain

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
> 
> Introduce the ability to assign capabilities to a domain via its definition in
> device tree. The first capability enabled to select is the control domain
> capability. The capability property is a bitfield in both the device tree and
> `struct boot_domain`.
> 
> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
> Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>

The R-b feels kind of redundant with the subsequent S-o-b.

> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -257,6 +257,18 @@ static int __init process_domain_node(
>              bd->max_vcpus = val;
>              printk("  max vcpus: %d\n", bd->max_vcpus);
>          }
> +        else if ( strncmp(prop_name, "capabilities", name_len) == 0 )
> +        {
> +            if ( fdt_prop_as_u32(prop, &bd->capabilities) != 0 )
> +            {
> +                printk("  failed processing domain id for domain %s\n", name);
> +                return -EINVAL;
> +            }
> +            printk("  caps: ");
> +            if ( bd->capabilities & BUILD_CAPS_CONTROL )
> +                printk("c");
> +            printk("\n");
> +        }

Like for the other patch: What about other bits being set in the value read?

> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -1006,6 +1006,7 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>  {
>      char *cmdline = NULL;
>      size_t cmdline_size;
> +    unsigned int create_flags = 0;
>      struct xen_domctl_createdomain dom0_cfg = {
>          .flags = IS_ENABLED(CONFIG_TBOOT) ? XEN_DOMCTL_CDF_s3_integrity : 0,
>          .max_evtchn_port = -1,
> @@ -1037,7 +1038,10 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>      if ( bd->domid == DOMID_INVALID )
>          /* Create initial domain.  Not d0 for pvshim. */
>          bd->domid = get_initial_domain_id();
> -    d = domain_create(bd->domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
> +    if ( bd->capabilities & BUILD_CAPS_CONTROL )
> +        create_flags |= CDF_privileged;

Seeing that builder_init() in the non-DT case sets the new bit unconditionally,
isn't the shim's only domain suddenly getting CDF_privileged set this way? Oh,
no, you then ...

> +    d = domain_create(bd->domid, &dom0_cfg,
> +                      pv_shim ? 0 : create_flags);

... hide the flag here. Any reason to have the intermediate variable in the
first place (can't resist: when there's already a wall of local variables here)?

Jan

Re: [PATCH v3 16/16] x86/hyperlaunch: add capabilities to boot domain

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
> 
> Introduce the ability to assign capabilities to a domain via its definition in
> device tree. The first capability enabled to select is the control domain
> capability. The capability property is a bitfield in both the device tree and
> `struct boot_domain`.
> 
> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
> Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>

The R-b feels kind of redundant with the subsequent S-o-b.

> --- a/xen/arch/x86/domain-builder/fdt.c
> +++ b/xen/arch/x86/domain-builder/fdt.c
> @@ -257,6 +257,18 @@ static int __init process_domain_node(
>              bd->max_vcpus = val;
>              printk("  max vcpus: %d\n", bd->max_vcpus);
>          }
> +        else if ( strncmp(prop_name, "capabilities", name_len) == 0 )
> +        {
> +            if ( fdt_prop_as_u32(prop, &bd->capabilities) != 0 )
> +            {
> +                printk("  failed processing domain id for domain %s\n", name);
> +                return -EINVAL;
> +            }
> +            printk("  caps: ");
> +            if ( bd->capabilities & BUILD_CAPS_CONTROL )
> +                printk("c");
> +            printk("\n");
> +        }

Like for the other patch: What about other bits being set in the value read?

> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -1006,6 +1006,7 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>  {
>      char *cmdline = NULL;
>      size_t cmdline_size;
> +    unsigned int create_flags = 0;
>      struct xen_domctl_createdomain dom0_cfg = {
>          .flags = IS_ENABLED(CONFIG_TBOOT) ? XEN_DOMCTL_CDF_s3_integrity : 0,
>          .max_evtchn_port = -1,
> @@ -1037,7 +1038,10 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>      if ( bd->domid == DOMID_INVALID )
>          /* Create initial domain.  Not d0 for pvshim. */
>          bd->domid = get_initial_domain_id();
> -    d = domain_create(bd->domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
> +    if ( bd->capabilities & BUILD_CAPS_CONTROL )
> +        create_flags |= CDF_privileged;

Seeing that builder_init() in the non-DT case sets the new bit unconditionally,
isn't the shim's only domain suddenly getting CDF_privileged set this way? Oh,
no, you then ...

> +    d = domain_create(bd->domid, &dom0_cfg,
> +                      pv_shim ? 0 : create_flags);

... hide the flag here. Any reason to have the intermediate variable in the
first place (can't resist: when there's already a wall of local variables here)?

Jan

Re: [PATCH v3 16/16] x86/hyperlaunch: add capabilities to boot domain

[Alejandro Vallejo]

On Thu Apr 10, 2025 at 1:18 PM BST, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> From: "Daniel P. Smith" <dpsmith@apertussolutions.com>
>> 
>> Introduce the ability to assign capabilities to a domain via its definition in
>> device tree. The first capability enabled to select is the control domain
>> capability. The capability property is a bitfield in both the device tree and
>> `struct boot_domain`.
>> 
>> Signed-off-by: Daniel P. Smith <dpsmith@apertussolutions.com>
>> Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
>> Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
>
> The R-b feels kind of redundant with the subsequent S-o-b.

I'll drop it.

>
>> --- a/xen/arch/x86/domain-builder/fdt.c
>> +++ b/xen/arch/x86/domain-builder/fdt.c
>> @@ -257,6 +257,18 @@ static int __init process_domain_node(
>>              bd->max_vcpus = val;
>>              printk("  max vcpus: %d\n", bd->max_vcpus);
>>          }
>> +        else if ( strncmp(prop_name, "capabilities", name_len) == 0 )
>> +        {
>> +            if ( fdt_prop_as_u32(prop, &bd->capabilities) != 0 )
>> +            {
>> +                printk("  failed processing domain id for domain %s\n", name);
>> +                return -EINVAL;
>> +            }
>> +            printk("  caps: ");
>> +            if ( bd->capabilities & BUILD_CAPS_CONTROL )
>> +                printk("c");
>> +            printk("\n");
>> +        }
>
> Like for the other patch: What about other bits being set in the value read?

I take it that the non-worded suggestion is to have a mask of reserved
bits for each case and check they are not set (giving a warning if they are)?

>
>> --- a/xen/arch/x86/setup.c
>> +++ b/xen/arch/x86/setup.c
>> @@ -1006,6 +1006,7 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>>  {
>>      char *cmdline = NULL;
>>      size_t cmdline_size;
>> +    unsigned int create_flags = 0;
>>      struct xen_domctl_createdomain dom0_cfg = {
>>          .flags = IS_ENABLED(CONFIG_TBOOT) ? XEN_DOMCTL_CDF_s3_integrity : 0,
>>          .max_evtchn_port = -1,
>> @@ -1037,7 +1038,10 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>>      if ( bd->domid == DOMID_INVALID )
>>          /* Create initial domain.  Not d0 for pvshim. */
>>          bd->domid = get_initial_domain_id();
>> -    d = domain_create(bd->domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
>> +    if ( bd->capabilities & BUILD_CAPS_CONTROL )
>> +        create_flags |= CDF_privileged;
>
> Seeing that builder_init() in the non-DT case sets the new bit unconditionally,
> isn't the shim's only domain suddenly getting CDF_privileged set this way? Oh,
> no, you then ...
>
>> +    d = domain_create(bd->domid, &dom0_cfg,
>> +                      pv_shim ? 0 : create_flags);
>
> ... hide the flag here. Any reason to have the intermediate variable in the
> first place

Well, the logic would end up fairly convoluted otherwise. As things
stand this can be encoded in an if-else fashion with 2 calls, but
there's 2 capability flags coming that need integrating together.

This is just avoiding further code motion down the line.

> (can't resist: when there's already a wall of local variables here)?

Heh :). Point taken.

Cheers,
Alejandro

Re: [PATCH v3 16/16] x86/hyperlaunch: add capabilities to boot domain

[Jan Beulich]

On 14.04.2025 21:31, Alejandro Vallejo wrote:
> On Thu Apr 10, 2025 at 1:18 PM BST, Jan Beulich wrote:
>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>> --- a/xen/arch/x86/domain-builder/fdt.c
>>> +++ b/xen/arch/x86/domain-builder/fdt.c
>>> @@ -257,6 +257,18 @@ static int __init process_domain_node(
>>>              bd->max_vcpus = val;
>>>              printk("  max vcpus: %d\n", bd->max_vcpus);
>>>          }
>>> +        else if ( strncmp(prop_name, "capabilities", name_len) == 0 )
>>> +        {
>>> +            if ( fdt_prop_as_u32(prop, &bd->capabilities) != 0 )
>>> +            {
>>> +                printk("  failed processing domain id for domain %s\n", name);
>>> +                return -EINVAL;
>>> +            }
>>> +            printk("  caps: ");
>>> +            if ( bd->capabilities & BUILD_CAPS_CONTROL )
>>> +                printk("c");
>>> +            printk("\n");
>>> +        }
>>
>> Like for the other patch: What about other bits being set in the value read?
> 
> I take it that the non-worded suggestion is to have a mask of reserved
> bits for each case and check they are not set (giving a warning if they are)?

Whether a warning is sufficient I can't tell. I would have expected such to be
outright rejected.

>>> --- a/xen/arch/x86/setup.c
>>> +++ b/xen/arch/x86/setup.c
>>> @@ -1006,6 +1006,7 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>>>  {
>>>      char *cmdline = NULL;
>>>      size_t cmdline_size;
>>> +    unsigned int create_flags = 0;
>>>      struct xen_domctl_createdomain dom0_cfg = {
>>>          .flags = IS_ENABLED(CONFIG_TBOOT) ? XEN_DOMCTL_CDF_s3_integrity : 0,
>>>          .max_evtchn_port = -1,
>>> @@ -1037,7 +1038,10 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>>>      if ( bd->domid == DOMID_INVALID )
>>>          /* Create initial domain.  Not d0 for pvshim. */
>>>          bd->domid = get_initial_domain_id();
>>> -    d = domain_create(bd->domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
>>> +    if ( bd->capabilities & BUILD_CAPS_CONTROL )
>>> +        create_flags |= CDF_privileged;
>>
>> Seeing that builder_init() in the non-DT case sets the new bit unconditionally,
>> isn't the shim's only domain suddenly getting CDF_privileged set this way? Oh,
>> no, you then ...
>>
>>> +    d = domain_create(bd->domid, &dom0_cfg,
>>> +                      pv_shim ? 0 : create_flags);
>>
>> ... hide the flag here. Any reason to have the intermediate variable in the
>> first place
> 
> Well, the logic would end up fairly convoluted otherwise. As things
> stand this can be encoded in an if-else fashion with 2 calls, but
> there's 2 capability flags coming that need integrating together.
> 
> This is just avoiding further code motion down the line.

Is it?

-    d = domain_create(bd->domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
+    d = domain_create(bd->domid, &dom0_cfg,
+                      ((bd->capabilities & BUILD_CAPS_CONTROL) && !pv_shim
+                       ? CDF_privileged : 0));

isn't really worse (imo), but is highlighting the problem more clearly: Why
would the shim have BUILD_CAPS_CONTROL set in the first place? Without that
the statement would remain pretty similar to what it was before.

Jan

Re: [PATCH v3 16/16] x86/hyperlaunch: add capabilities to boot domain

[Alejandro Vallejo]

On Tue Apr 15, 2025 at 7:38 AM BST, Jan Beulich wrote:
> On 14.04.2025 21:31, Alejandro Vallejo wrote:
>> On Thu Apr 10, 2025 at 1:18 PM BST, Jan Beulich wrote:
>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>> --- a/xen/arch/x86/domain-builder/fdt.c
>>>> +++ b/xen/arch/x86/domain-builder/fdt.c
>>>> @@ -257,6 +257,18 @@ static int __init process_domain_node(
>>>>              bd->max_vcpus = val;
>>>>              printk("  max vcpus: %d\n", bd->max_vcpus);
>>>>          }
>>>> +        else if ( strncmp(prop_name, "capabilities", name_len) == 0 )
>>>> +        {
>>>> +            if ( fdt_prop_as_u32(prop, &bd->capabilities) != 0 )
>>>> +            {
>>>> +                printk("  failed processing domain id for domain %s\n", name);
>>>> +                return -EINVAL;
>>>> +            }
>>>> +            printk("  caps: ");
>>>> +            if ( bd->capabilities & BUILD_CAPS_CONTROL )
>>>> +                printk("c");
>>>> +            printk("\n");
>>>> +        }
>>>
>>> Like for the other patch: What about other bits being set in the value read?
>> 
>> I take it that the non-worded suggestion is to have a mask of reserved
>> bits for each case and check they are not set (giving a warning if they are)?
>
> Whether a warning is sufficient I can't tell. I would have expected such to be
> outright rejected.
>
>>>> --- a/xen/arch/x86/setup.c
>>>> +++ b/xen/arch/x86/setup.c
>>>> @@ -1006,6 +1006,7 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>>>>  {
>>>>      char *cmdline = NULL;
>>>>      size_t cmdline_size;
>>>> +    unsigned int create_flags = 0;
>>>>      struct xen_domctl_createdomain dom0_cfg = {
>>>>          .flags = IS_ENABLED(CONFIG_TBOOT) ? XEN_DOMCTL_CDF_s3_integrity : 0,
>>>>          .max_evtchn_port = -1,
>>>> @@ -1037,7 +1038,10 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>>>>      if ( bd->domid == DOMID_INVALID )
>>>>          /* Create initial domain.  Not d0 for pvshim. */
>>>>          bd->domid = get_initial_domain_id();
>>>> -    d = domain_create(bd->domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
>>>> +    if ( bd->capabilities & BUILD_CAPS_CONTROL )
>>>> +        create_flags |= CDF_privileged;
>>>
>>> Seeing that builder_init() in the non-DT case sets the new bit unconditionally,
>>> isn't the shim's only domain suddenly getting CDF_privileged set this way? Oh,
>>> no, you then ...
>>>
>>>> +    d = domain_create(bd->domid, &dom0_cfg,
>>>> +                      pv_shim ? 0 : create_flags);
>>>
>>> ... hide the flag here. Any reason to have the intermediate variable in the
>>> first place
>> 
>> Well, the logic would end up fairly convoluted otherwise. As things
>> stand this can be encoded in an if-else fashion with 2 calls, but
>> there's 2 capability flags coming that need integrating together.
>> 
>> This is just avoiding further code motion down the line.
>
> Is it?
>
> -    d = domain_create(bd->domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
> +    d = domain_create(bd->domid, &dom0_cfg,
> +                      ((bd->capabilities & BUILD_CAPS_CONTROL) && !pv_shim
> +                       ? CDF_privileged : 0));
>
> isn't really worse (imo),

Not sure I agree. Long conditions on ternary operators makes the
control flow harder to follow.

A nicer alternative that also removes the auxiliary variable is to have
a helper to convert from bootcaps to whatever createdomainflags are
required. That'd extend naturally for more bits.

> but is highlighting the problem more clearly: Why
> would the shim have BUILD_CAPS_CONTROL set in the first place? Without that
> the statement would remain pretty similar to what it was before.

If the commandline is parsed early enough (I see the early parse path in
head.S?) it would be better to add this logic to builder_init() and
prevent the capability from reaching the boot_domain in the first place.

Then there's no exception for the pv shim.

Cheers,
Alejandro

Re: [PATCH v3 16/16] x86/hyperlaunch: add capabilities to boot domain

[Jan Beulich]

On 15.04.2025 14:22, Alejandro Vallejo wrote:
> On Tue Apr 15, 2025 at 7:38 AM BST, Jan Beulich wrote:
>> On 14.04.2025 21:31, Alejandro Vallejo wrote:
>>> On Thu Apr 10, 2025 at 1:18 PM BST, Jan Beulich wrote:
>>>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>>>> --- a/xen/arch/x86/setup.c
>>>>> +++ b/xen/arch/x86/setup.c
>>>>> @@ -1006,6 +1006,7 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>>>>>  {
>>>>>      char *cmdline = NULL;
>>>>>      size_t cmdline_size;
>>>>> +    unsigned int create_flags = 0;
>>>>>      struct xen_domctl_createdomain dom0_cfg = {
>>>>>          .flags = IS_ENABLED(CONFIG_TBOOT) ? XEN_DOMCTL_CDF_s3_integrity : 0,
>>>>>          .max_evtchn_port = -1,
>>>>> @@ -1037,7 +1038,10 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>>>>>      if ( bd->domid == DOMID_INVALID )
>>>>>          /* Create initial domain.  Not d0 for pvshim. */
>>>>>          bd->domid = get_initial_domain_id();
>>>>> -    d = domain_create(bd->domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
>>>>> +    if ( bd->capabilities & BUILD_CAPS_CONTROL )
>>>>> +        create_flags |= CDF_privileged;
>>>>
>>>> Seeing that builder_init() in the non-DT case sets the new bit unconditionally,
>>>> isn't the shim's only domain suddenly getting CDF_privileged set this way? Oh,
>>>> no, you then ...
>>>>
>>>>> +    d = domain_create(bd->domid, &dom0_cfg,
>>>>> +                      pv_shim ? 0 : create_flags);
>>>>
>>>> ... hide the flag here. Any reason to have the intermediate variable in the
>>>> first place
>>>
>>> Well, the logic would end up fairly convoluted otherwise. As things
>>> stand this can be encoded in an if-else fashion with 2 calls, but
>>> there's 2 capability flags coming that need integrating together.
>>>
>>> This is just avoiding further code motion down the line.
>>
>> Is it?
>>
>> -    d = domain_create(bd->domid, &dom0_cfg, pv_shim ? 0 : CDF_privileged);
>> +    d = domain_create(bd->domid, &dom0_cfg,
>> +                      ((bd->capabilities & BUILD_CAPS_CONTROL) && !pv_shim
>> +                       ? CDF_privileged : 0));
>>
>> isn't really worse (imo),
> 
> Not sure I agree. Long conditions on ternary operators makes the
> control flow harder to follow.
> 
> A nicer alternative that also removes the auxiliary variable is to have
> a helper to convert from bootcaps to whatever createdomainflags are
> required. That'd extend naturally for more bits.
> 
>> but is highlighting the problem more clearly: Why
>> would the shim have BUILD_CAPS_CONTROL set in the first place? Without that
>> the statement would remain pretty similar to what it was before.
> 
> If the commandline is parsed early enough (I see the early parse path in
> head.S?) it would be better to add this logic to builder_init() and
> prevent the capability from reaching the boot_domain in the first place.

The parsing from head.S is only partial. But surely DT is being looked at
far later than when the full parsing (cmdline_parse()) is done?

Jan

Re: [PATCH v3 00/16] Hyperlaunch device tree for dom0

[Jan Beulich]

On 08.04.2025 18:07, Alejandro Vallejo wrote:
> I've purposefully not added my S-by on anything I haven't touched
> (besides rebasing) as most of the feedback had already been addressed by
> Jason by the time I looked at it and it would be utterly nonsensical to
> give myself authorship over it.

My understanding of it is that S-o-b doesn't indicate (only) authorship.
See docs/process/sending-patches.pandoc "Developer's Certificate of Origin
1.1" point (c). And what you would want to demonstrate with adding your
own one is agreement with (d) there. With this I'm not even sure I could
commit patches becoming ready in their present form.

Jan

Re: [PATCH v3 00/16] Hyperlaunch device tree for dom0

[Alejandro Vallejo]

On Wed Apr 9, 2025 at 7:29 AM BST, Jan Beulich wrote:
> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>> I've purposefully not added my S-by on anything I haven't touched
>> (besides rebasing) as most of the feedback had already been addressed by
>> Jason by the time I looked at it and it would be utterly nonsensical to
>> give myself authorship over it.
>
> My understanding of it is that S-o-b doesn't indicate (only) authorship.
> See docs/process/sending-patches.pandoc "Developer's Certificate of Origin
> 1.1" point (c). And what you would want to demonstrate with adding your
> own one is agreement with (d) there. With this I'm not even sure I could
> commit patches becoming ready in their present form.
>
> Jan

Fair enough. Will add myself everywhere on v4, then. Seeing as there's
still changes to go through.

Cheers,
Alejandro

Re: [PATCH v3 00/16] Hyperlaunch device tree for dom0

[Alejandro Vallejo]

On Wed Apr 9, 2025 at 11:19 AM BST, Alejandro Vallejo wrote:
> On Wed Apr 9, 2025 at 7:29 AM BST, Jan Beulich wrote:
>> On 08.04.2025 18:07, Alejandro Vallejo wrote:
>>> I've purposefully not added my S-by on anything I haven't touched
>>> (besides rebasing) as most of the feedback had already been addressed by
>>> Jason by the time I looked at it and it would be utterly nonsensical to
>>> give myself authorship over it.
>>
>> My understanding of it is that S-o-b doesn't indicate (only) authorship.
>> See docs/process/sending-patches.pandoc "Developer's Certificate of Origin
>> 1.1" point (c). And what you would want to demonstrate with adding your
>> own one is agreement with (d) there. With this I'm not even sure I could
>> commit patches becoming ready in their present form.
>>
>> Jan
>
> Fair enough. Will add myself everywhere on v4, then. Seeing as there's
> still changes to go through.
>
> Cheers,
> Alejandro

And FTAOD, that also meant an implicit

    Signed-off-by: Alejandro Vallejo <agarciav@amd.com>

to everything in v3.

Cheers,
Alejandro