[PATCH v11 0/3] xen/domain: domain ID allocation

[PATCH v11 0/3] xen/domain: domain ID allocation

[dmkhn]

Patch 1 introduces new domid_{alloc,free} calls.
Patch 2 introduces some basic testing for domain ID allocator.
Patch 3 adjusts create_dom0() messages (use %pd).

Link to v10: https://lore.kernel.org/xen-devel/20250623182721.194238-1-dmukhin@ford.com/
Link to CI: https://gitlab.com/xen-project/people/dmukhin/xen/-/pipelines/1953102433

Denis Mukhin (3):
  xen/domain: unify domain ID allocation
  tools/tests: introduce unit tests for domain ID allocator
  xen/domain: update create_dom0() messages

 tools/tests/Makefile                    |   2 +-
 tools/tests/domid/.gitignore            |   2 +
 tools/tests/domid/Makefile              |  69 +++++++++++++
 tools/tests/domid/include/xen/domain.h  | 124 ++++++++++++++++++++++++
 tools/tests/domid/test-domid.c          |  88 +++++++++++++++++
 xen/arch/arm/domain_build.c             |  15 ++-
 xen/arch/x86/setup.c                    |  11 ++-
 xen/common/Makefile                     |   1 +
 xen/common/device-tree/dom0less-build.c |  15 ++-
 xen/common/domain.c                     |   2 +
 xen/common/domctl.c                     |  42 +-------
 xen/common/domid.c                      |  93 ++++++++++++++++++
 xen/include/xen/domain.h                |   3 +
 13 files changed, 412 insertions(+), 55 deletions(-)
 create mode 100644 tools/tests/domid/.gitignore
 create mode 100644 tools/tests/domid/Makefile
 create mode 100644 tools/tests/domid/include/xen/domain.h
 create mode 100644 tools/tests/domid/test-domid.c
 create mode 100644 xen/common/domid.c

-- 
2.34.1

[PATCH v11 1/3] xen/domain: unify domain ID allocation

[dmkhn]

From: Denis Mukhin <dmukhin@ford.com> 

Currently, there are two different domain ID allocation implementations:

  1) Sequential IDs allocation in dom0less Arm code based on max_init_domid;

  2) Sequential IDs allocation in XEN_DOMCTL_createdomain; does not use
     max_init_domid (both Arm and x86).

The domain ID allocation covers dom0 or late hwdom, predefined domains,
post-boot domains, excluding Xen system domains (domid >=
DOMID_FIRST_RESERVED).

It makes sense to have a common helper code for such task across architectures
(Arm and x86) and between dom0less / toolstack domU allocation.

Note, fixing dependency on max_init_domid is out of scope of this patch.

Wrap the domain ID allocation as an arch-independent function domid_alloc() in
new common/domid.c based on the bitmap.

Allocation algorithm:
- If an explicit domain ID is provided, verify its availability and use it if
  ID is not used;
- If DOMID_INVALID is provided, search the range [1..DOMID_FIRST_RESERVED-1],
  starting from the last used ID. IDs are not wrapped around in dom0less case.
  Implementation guarantees that two consecutive calls will never return the
  same ID. ID#0 is reserved for the first boot domain (currently, dom0) and
  excluded from the allocation range.

Remove is_free_domid() helper as it is not needed now.

No functional change intended.

Signed-off-by: Denis Mukhin <dmukhin@ford.com>
---
Changes since v10:
- fixup #ifdefs in domid_alloc()
- corrected use of domid_free() in domain_destroy()
- rebased
- moved domid_{alloc,free}() to common/domid.c so the functional test could be
  added later
---
 xen/arch/arm/domain_build.c             |  7 +-
 xen/arch/x86/setup.c                    |  7 +-
 xen/common/Makefile                     |  1 +
 xen/common/device-tree/dom0less-build.c | 15 ++--
 xen/common/domain.c                     |  2 +
 xen/common/domctl.c                     | 42 ++---------
 xen/common/domid.c                      | 93 +++++++++++++++++++++++++
 xen/include/xen/domain.h                |  3 +
 8 files changed, 122 insertions(+), 48 deletions(-)
 create mode 100644 xen/common/domid.c

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 463ae4474d30..789f2b9d3ce7 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2050,6 +2050,7 @@ void __init create_dom0(void)
         .grant_opts = XEN_DOMCTL_GRANT_version(opt_gnttab_max_version),
     };
     unsigned int flags = CDF_privileged | CDF_hardware;
+    domid_t domid;
     int rc;
 
     /* The vGIC for DOM0 is exactly emulating the hardware GIC */
@@ -2074,7 +2075,11 @@ void __init create_dom0(void)
     if ( !llc_coloring_enabled )
         flags |= CDF_directmap;
 
-    dom0 = domain_create(0, &dom0_cfg, flags);
+    domid = domid_alloc(0);
+    if ( domid == DOMID_INVALID )
+        panic("Error allocating domain ID 0\n");
+
+    dom0 = domain_create(domid, &dom0_cfg, flags);
     if ( IS_ERR(dom0) )
         panic("Error creating domain 0 (rc = %ld)\n", PTR_ERR(dom0));
 
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 1543dd251cc6..2ff7c28c277b 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1047,8 +1047,11 @@ static struct domain *__init create_dom0(struct boot_info *bi)
     if ( iommu_enabled )
         dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
 
-    /* Create initial domain.  Not d0 for pvshim. */
-    bd->domid = get_initial_domain_id();
+    /* Allocate initial domain ID.  Not d0 for pvshim. */
+    bd->domid = domid_alloc(get_initial_domain_id());
+    if ( bd->domid == DOMID_INVALID )
+        panic("Error allocating domain ID %d\n", get_initial_domain_id());
+
     d = domain_create(bd->domid, &dom0_cfg,
                       pv_shim ? 0 : CDF_privileged | CDF_hardware);
     if ( IS_ERR(d) )
diff --git a/xen/common/Makefile b/xen/common/Makefile
index c316957fcb36..0c7d0f5d46e1 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -11,6 +11,7 @@ obj-$(filter-out $(CONFIG_X86),$(CONFIG_ACPI)) += device.o
 obj-$(CONFIG_DEVICE_TREE_PARSE) += device-tree/
 obj-$(CONFIG_IOREQ_SERVER) += dm.o
 obj-y += domain.o
+obj-y += domid.o
 obj-y += event_2l.o
 obj-y += event_channel.o
 obj-$(CONFIG_EVTCHN_FIFO) += event_fifo.o
diff --git a/xen/common/device-tree/dom0less-build.c b/xen/common/device-tree/dom0less-build.c
index 6bb038111de9..1f9461d0e738 100644
--- a/xen/common/device-tree/dom0less-build.c
+++ b/xen/common/device-tree/dom0less-build.c
@@ -833,21 +833,20 @@ void __init create_domUs(void)
     {
         struct kernel_info ki = KERNEL_INFO_INIT;
         int rc = parse_dom0less_node(node, &ki.bd);
+        domid_t domid;
 
         if ( rc == -ENOENT )
             continue;
         if ( rc )
             panic("Malformed DTB: Invalid domain %s\n", dt_node_name(node));
 
-        if ( (max_init_domid + 1) >= DOMID_FIRST_RESERVED )
-            panic("No more domain IDs available\n");
+        domid = domid_alloc(DOMID_INVALID);
+        if ( domid == DOMID_INVALID )
+            panic("Error allocating ID for domain %s\n", dt_node_name(node));
+        if ( max_init_domid < domid )
+            max_init_domid = domid;
 
-        /*
-         * The variable max_init_domid is initialized with zero, so here it's
-         * very important to use the pre-increment operator to call
-         * domain_create() with a domid > 0. (domid == 0 is reserved for Dom0)
-         */
-        ki.bd.d = domain_create(++max_init_domid,
+        ki.bd.d = domain_create(domid,
                                 &ki.bd.create_cfg, ki.bd.create_flags);
         if ( IS_ERR(ki.bd.d) )
             panic("Error creating domain %s (rc = %ld)\n",
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 3c65cca5b0ff..23dbc1f46c78 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -1466,6 +1466,8 @@ void domain_destroy(struct domain *d)
     /* Remove from the domlist/hash. */
     domlist_remove(d);
 
+    domid_free(d->domain_id);
+
     /* Schedule RCU asynchronous completion of domain destroy. */
     call_rcu(&d->rcu, complete_domain_destroy);
 }
diff --git a/xen/common/domctl.c b/xen/common/domctl.c
index f2a7caaf853c..5509998aa139 100644
--- a/xen/common/domctl.c
+++ b/xen/common/domctl.c
@@ -51,20 +51,6 @@ static int xenctl_bitmap_to_nodemask(nodemask_t *nodemask,
                                    MAX_NUMNODES);
 }
 
-static inline int is_free_domid(domid_t dom)
-{
-    struct domain *d;
-
-    if ( dom >= DOMID_FIRST_RESERVED )
-        return 0;
-
-    if ( (d = rcu_lock_domain_by_id(dom)) == NULL )
-        return 1;
-
-    rcu_unlock_domain(d);
-    return 0;
-}
-
 void getdomaininfo(struct domain *d, struct xen_domctl_getdomaininfo *info)
 {
     struct vcpu *v;
@@ -423,36 +409,18 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
 
     case XEN_DOMCTL_createdomain:
     {
-        domid_t        dom;
-        static domid_t rover = 0;
+        domid_t domid = domid_alloc(op->domain);
 
-        dom = op->domain;
-        if ( (dom > 0) && (dom < DOMID_FIRST_RESERVED) )
+        if ( domid == DOMID_INVALID )
         {
             ret = -EEXIST;
-            if ( !is_free_domid(dom) )
-                break;
-        }
-        else
-        {
-            for ( dom = rover + 1; dom != rover; dom++ )
-            {
-                if ( dom == DOMID_FIRST_RESERVED )
-                    dom = 1;
-                if ( is_free_domid(dom) )
-                    break;
-            }
-
-            ret = -ENOMEM;
-            if ( dom == rover )
-                break;
-
-            rover = dom;
+            break;
         }
 
-        d = domain_create(dom, &op->u.createdomain, false);
+        d = domain_create(domid, &op->u.createdomain, false);
         if ( IS_ERR(d) )
         {
+            domid_free(domid);
             ret = PTR_ERR(d);
             d = NULL;
             break;
diff --git a/xen/common/domid.c b/xen/common/domid.c
new file mode 100644
index 000000000000..e553ab6e5468
--- /dev/null
+++ b/xen/common/domid.c
@@ -0,0 +1,93 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Domain ID allocator.
+ * Covers dom0 or late hwdom, predefined domains, post-boot domains; excludes
+ * Xen system domains (ID >= DOMID_FIRST_RESERVED).
+ *
+ * Copyright 2025 Ford Motor Company
+ */
+
+#include <xen/domain.h>
+
+static DEFINE_SPINLOCK(domid_lock);
+static DECLARE_BITMAP(domid_bitmap, DOMID_FIRST_RESERVED);
+
+/*
+ * Allocate domain ID.
+ *
+ * @param[in] domid Exact domain ID within [0..DOMID_FIRST_RESERVED-1] range or
+ *                  DOMID_INVALID for exhaustive search within
+ *                  [1..DOMID_FIRST_RESERVED-1].
+ * @return Valid domain ID in case of successful allocation,
+ *         DOMID_INVALID - otherwise.
+ */
+domid_t cf_check domid_alloc(domid_t domid)
+{
+    static domid_t domid_last;
+
+    spin_lock(&domid_lock);
+
+    /* Exact match. */
+    if ( domid < DOMID_FIRST_RESERVED )
+    {
+        if ( __test_and_set_bit(domid, domid_bitmap) )
+            domid = DOMID_INVALID;
+    }
+    /*
+     * Exhaustive search.
+     *
+     * Domain ID#0 is reserved for the first boot domain (e.g. control domain)
+     * and excluded from allocation.
+     *
+     * In dom0less build, domains are not dynamically destroyed, so there's no
+     * need to do a wraparound of the IDs.
+     */
+#ifdef CONFIG_DOM0LESS_BOOT
+    else if ( domid_last + 1 >= DOMID_FIRST_RESERVED )
+        domid = DOMID_INVALID;
+#endif
+    else
+    {
+        domid = find_next_zero_bit(domid_bitmap,
+                                   DOMID_FIRST_RESERVED,
+                                   domid_last + 1);
+#ifndef CONFIG_DOM0LESS_BOOT
+        ASSERT(domid <= DOMID_FIRST_RESERVED);
+        if ( domid == DOMID_FIRST_RESERVED )
+            domid = find_next_zero_bit(domid_bitmap,
+                                       DOMID_FIRST_RESERVED,
+                                       1);
+#endif
+
+        if ( domid < DOMID_FIRST_RESERVED )
+        {
+            __set_bit(domid, domid_bitmap);
+            domid_last = domid;
+        }
+        else
+            domid = DOMID_INVALID;
+    }
+
+    spin_unlock(&domid_lock);
+
+    return domid;
+}
+
+void cf_check domid_free(domid_t domid)
+{
+    ASSERT(domid <= DOMID_FIRST_RESERVED);
+
+    spin_lock(&domid_lock);
+    __clear_bit(domid, domid_bitmap);
+    spin_unlock(&domid_lock);
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/xen/domain.h b/xen/include/xen/domain.h
index e10baf2615fd..31946bb1b653 100644
--- a/xen/include/xen/domain.h
+++ b/xen/include/xen/domain.h
@@ -38,6 +38,9 @@ void arch_get_domain_info(const struct domain *d,
 
 domid_t get_initial_domain_id(void);
 
+domid_t cf_check domid_alloc(domid_t domid);
+void cf_check domid_free(domid_t domid);
+
 /* CDF_* constant. Internal flags for domain creation. */
 /* Is this a privileged domain? */
 #define CDF_privileged           (1U << 0)
-- 
2.34.1

[PATCH v11 2/3] tools/tests: introduce unit tests for domain ID allocator

[dmkhn]

From: Denis Mukhin <dmukhin@ford.com> 

Introduce some basic infrastructure for doing domain ID allocation unit tests,
and add a few tests that ensure correctness of the domain ID allocator in
dom0less and non-dom0less scenarios.

Signed-off-by: Denis Mukhin <dmukhin@ford.com>
---
Changes since v10:
- new patch
---
 tools/tests/Makefile                   |   2 +-
 tools/tests/domid/.gitignore           |   2 +
 tools/tests/domid/Makefile             |  69 ++++++++++++++
 tools/tests/domid/include/xen/domain.h | 124 +++++++++++++++++++++++++
 tools/tests/domid/test-domid.c         |  88 ++++++++++++++++++
 5 files changed, 284 insertions(+), 1 deletion(-)
 create mode 100644 tools/tests/domid/.gitignore
 create mode 100644 tools/tests/domid/Makefile
 create mode 100644 tools/tests/domid/include/xen/domain.h
 create mode 100644 tools/tests/domid/test-domid.c

diff --git a/tools/tests/Makefile b/tools/tests/Makefile
index 36928676a666..ff1666425436 100644
--- a/tools/tests/Makefile
+++ b/tools/tests/Makefile
@@ -1,7 +1,7 @@
 XEN_ROOT = $(CURDIR)/../..
 include $(XEN_ROOT)/tools/Rules.mk
 
-SUBDIRS-y :=
+SUBDIRS-y := domid
 SUBDIRS-y += resource
 SUBDIRS-$(CONFIG_X86) += cpu-policy
 SUBDIRS-$(CONFIG_X86) += tsx
diff --git a/tools/tests/domid/.gitignore b/tools/tests/domid/.gitignore
new file mode 100644
index 000000000000..91ac43232518
--- /dev/null
+++ b/tools/tests/domid/.gitignore
@@ -0,0 +1,2 @@
+*.o
+test-domid-*
diff --git a/tools/tests/domid/Makefile b/tools/tests/domid/Makefile
new file mode 100644
index 000000000000..e0b6a0e0a49c
--- /dev/null
+++ b/tools/tests/domid/Makefile
@@ -0,0 +1,69 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Unit tests for domain ID allocator.
+#
+# There are two flavors of the test:
+# - default; domain ID can be reused.
+# - dom0less: each domain ID can be allocated only once.
+#
+# Copyright 2025 Ford Motor Company
+
+XEN_ROOT=$(CURDIR)/../../..
+include $(XEN_ROOT)/tools/Rules.mk
+
+TESTS := test-domid-default test-domid-CONFIG_DOM0LESS_BOOT
+
+vpath domid.c $(XEN_ROOT)/xen/common/
+
+.PHONY: all
+all: $(TESTS)
+
+.PHONY: run
+run: $(TESTS)
+	$(foreach t,$(TESTS),./$(t);)
+
+.PHONY: clean
+clean:
+	$(RM) -- *.o $(TESTS) $(DEPS_RM)
+
+.PHONY: distclean
+distclean: clean
+	$(RM) -- *~
+
+.PHONY: install
+install: all
+	$(INSTALL_DIR) $(DESTDIR)$(LIBEXEC)/tests
+	$(INSTALL_PROG) test-domid-* $(DESTDIR)$(LIBEXEC)/tests
+
+.PHONY: uninstall
+uninstall:
+	$(RM) -- $(DESTDIR)$(LIBEXEC)/tests/test-domid-*
+
+CFLAGS += -D__XEN_TOOLS__
+CFLAGS += $(APPEND_CFLAGS)
+CFLAGS += $(CFLAGS_xeninclude)
+CFLAGS += -I./include/
+
+LDFLAGS += $(APPEND_LDFLAGS)
+
+# default (no dom0less) scenario
+domid-default.o: domid.c
+	$(CC) $(CFLAGS) -c $^ -o $@ $(LDFLAGS)
+
+test-domid-default.o: test-domid.c
+	$(CC) $(CFLAGS) -c $^ -o $@ $(LDFLAGS)
+
+test-domid-default: domid-default.o test-domid-default.o
+	$(CC) $^ -o $@ $(LDFLAGS)
+
+# dom0less scenario
+domid-CONFIG_DOM0LESS_BOOT.o: domid.c
+	$(CC) $(CFLAGS) -DCONFIG_DOM0LESS_BOOT -c $^ -o $@ $(LDFLAGS)
+
+test-domid-CONFIG_DOM0LESS_BOOT.o: test-domid.c
+	$(CC) $(CFLAGS) -DCONFIG_DOM0LESS_BOOT -c $^ -o $@ $(LDFLAGS)
+
+test-domid-CONFIG_DOM0LESS_BOOT: domid-CONFIG_DOM0LESS_BOOT.o test-domid-CONFIG_DOM0LESS_BOOT.o
+	$(CC) -DCONFIG_DOM0LESS_BOOT $^ -o $@ $(LDFLAGS)
+
+-include $(DEPS_INCLUDE)
diff --git a/tools/tests/domid/include/xen/domain.h b/tools/tests/domid/include/xen/domain.h
new file mode 100644
index 000000000000..465568c0dd02
--- /dev/null
+++ b/tools/tests/domid/include/xen/domain.h
@@ -0,0 +1,124 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Unit test harness for domain ID allocator.
+ *
+ * Copyright 2025 Ford Motor Company
+ */
+
+#ifndef _TEST_HARNESS_
+#define _TEST_HARNESS_
+
+#include <assert.h>
+#include <errno.h>
+#include <stdbool.h>
+#include <stddef.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <xen-tools/common-macros.h>
+
+#define BUG_ON(x)               assert(!(x))
+#define ASSERT(x)               assert(x)
+
+#define __xen_mk_uint(x)        x ## U
+#define xen_mk_uint(x)          __xen_mk_uint(x)
+
+#define DOMID_FIRST_RESERVED    xen_mk_uint(10)
+#define DOMID_INVALID           xen_mk_uint(11)
+
+#define DEFINE_SPINLOCK(x)      unsigned long *(x)
+#define spin_lock(x)            ((*(x))++)
+#define spin_unlock(x)          ((*(x))--)
+
+#define BITS_PER_LONG           sizeof(unsigned long)
+#define BITS_PER_WORD           (8U * BITS_PER_LONG)
+#define BITS_TO_LONGS(bits) \
+    (((bits) + BITS_PER_LONG - 1) / BITS_PER_LONG)
+#define DECLARE_BITMAP(name, bits) \
+    unsigned long name[BITS_TO_LONGS(bits)]
+
+static inline int __test_and_set_bit(unsigned int nr, unsigned long *addr)
+{
+    unsigned long mask = 1UL << (nr % BITS_PER_WORD);
+    unsigned long *p = addr + (nr / BITS_PER_WORD);
+    int old = (*p & mask) != 0;
+
+    *p |= mask;
+
+    return old;
+}
+
+static inline void __set_bit(unsigned int nr, volatile unsigned long *addr)
+{
+    unsigned long mask = 1UL << (nr % BITS_PER_WORD);
+    unsigned long *p = (unsigned long *)addr + (nr / BITS_PER_WORD);
+
+    *p |= mask;
+}
+
+static inline void __clear_bit(unsigned int nr, volatile unsigned long *addr)
+{
+    unsigned long mask = 1UL << (nr % BITS_PER_WORD);
+    unsigned long *p = (unsigned long *)addr + (nr / BITS_PER_WORD);
+
+    *p &= ~mask;
+}
+
+static inline unsigned long find_next_zero_bit(const unsigned long *addr,
+                                               unsigned long size,
+                                               unsigned long offset)
+{
+    unsigned long idx = offset / BITS_PER_WORD;
+    unsigned long bit = offset % BITS_PER_WORD;
+
+    if (offset >= size)
+        return size;
+
+    while (offset < size)
+    {
+        unsigned long val = addr[idx] | (~0UL >> (BITS_PER_WORD - bit));
+
+        if (~val)
+        {
+            unsigned long pos = __builtin_ffsl(~val);
+
+            if (pos > 0)
+            {
+                unsigned long rc = idx * BITS_PER_WORD + (pos - 1);
+
+                if (rc < size)
+                    return rc;
+            }
+        }
+
+        offset = (idx + 1) * BITS_PER_WORD;
+        idx++;
+        bit = 0;
+    }
+
+    return size;
+}
+
+#define printk printf
+
+#define cf_check
+
+typedef bool spinlock_t;
+typedef uint16_t domid_t;
+
+/* See include/xen/domain.h */
+extern domid_t domid_alloc(domid_t domid);
+extern void domid_free(domid_t domid);
+
+#endif /* _TEST_HARNESS_ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/tools/tests/domid/test-domid.c b/tools/tests/domid/test-domid.c
new file mode 100644
index 000000000000..f077d77be0df
--- /dev/null
+++ b/tools/tests/domid/test-domid.c
@@ -0,0 +1,88 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Unit tests for domain ID allocator.
+ *
+ * Copyright 2025 Ford Motor Company
+ */
+
+/* Local test include replicating hypervisor includes. */
+#include <xen/domain.h>
+
+int main(int argc, char **argv)
+{
+    domid_t expected, allocated;
+
+    printk("%s DOMID_FIRST_RESERVED=%u DOMID_INVALID=%u\n",
+            argv[0], DOMID_FIRST_RESERVED, DOMID_INVALID);
+
+    /* Test ID#0 cannot be allocated twice. */
+    allocated = domid_alloc(0);
+    printk("expected %u allocated %u\n", 0, allocated);
+    ASSERT(allocated == 0);
+    allocated = domid_alloc(0);
+    printk("expected %u allocated %u\n", DOMID_INVALID, allocated);
+    ASSERT(allocated == DOMID_INVALID);
+
+    /* Ensure ID is not allocated. */
+    domid_free(0);
+
+    /*
+     * Test that that two consecutive calls of domid_alloc(DOMID_INVALID)
+     * will never return the same ID.
+     * NB: ID#0 is reserved and shall not be allocated by
+     * domid_alloc(DOMID_INVALID).
+     */
+    for ( expected = 0; expected < DOMID_FIRST_RESERVED - 1; expected++ )
+    {
+        allocated = domid_alloc(DOMID_INVALID);
+        printk("expected %u allocated %u\n", expected + 1, allocated);
+        ASSERT(allocated == expected + 1);
+    }
+    for ( expected = 0; expected < DOMID_FIRST_RESERVED; expected++ )
+    {
+        allocated = domid_alloc(DOMID_INVALID);
+        printk("expected %u allocated %u\n", DOMID_INVALID, allocated);
+        ASSERT(allocated == DOMID_INVALID);
+    }
+
+    /* Re-allocate first ID from [1..DOMID_FIRST_RESERVED - 1]. */
+#ifdef CONFIG_DOM0LESS_BOOT
+    /* Each ID is allowed to be allocated only once. */
+    expected = DOMID_INVALID;
+#else
+    expected = 1;
+#endif
+    domid_free(1);
+    allocated = domid_alloc(DOMID_INVALID);
+    printk("expected %u allocated %u\n", expected, allocated);
+    ASSERT(allocated == expected);
+
+    /* Re-allocate last ID from [1..DOMID_FIRST_RESERVED - 1]. */
+#ifdef CONFIG_DOM0LESS_BOOT
+    /* Each ID is allowed to be allocated only once. */
+    expected = DOMID_INVALID;
+#else
+    expected = DOMID_FIRST_RESERVED - 1;
+#endif
+    domid_free(DOMID_FIRST_RESERVED - 1);
+    allocated = domid_alloc(DOMID_INVALID);
+    printk("expected %u allocated %u\n", expected, allocated);
+    ASSERT(allocated == expected);
+
+    /* Allocate an invalid ID. */
+    expected = DOMID_INVALID;
+    allocated = domid_alloc(DOMID_FIRST_RESERVED);
+    printk("expected %u allocated %u\n", expected, allocated);
+    ASSERT(allocated == expected);
+
+    return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.34.1

[PATCH v11 3/3] xen/domain: update create_dom0() messages

[dmkhn]

From: Denis Mukhin <dmukhin@ford.com> 

Use %pd for domain identification in error/panic messages in create_dom0().

No functional change.

Signed-off-by: Denis Mukhin <dmukhin@ford.com>
Reviewed-by: Alejandro Vallejo <alejandro.garciavallejo@amd.com>
---
Changed since v10:
- corrected messages in create_dom0()
- added Alejandro's R-b
---
 xen/arch/arm/domain_build.c | 8 ++++----
 xen/arch/x86/setup.c        | 4 ++--
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 789f2b9d3ce7..9422b5e1827b 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2081,17 +2081,17 @@ void __init create_dom0(void)
 
     dom0 = domain_create(domid, &dom0_cfg, flags);
     if ( IS_ERR(dom0) )
-        panic("Error creating domain 0 (rc = %ld)\n", PTR_ERR(dom0));
+        panic("Error creating d%d (rc = %ld)\n", domid, PTR_ERR(dom0));
 
     if ( llc_coloring_enabled && (rc = dom0_set_llc_colors(dom0)) )
-        panic("Error initializing LLC coloring for domain 0 (rc = %d)\n", rc);
+        panic("Error initializing LLC coloring for %pd (rc = %d)\n", dom0, rc);
 
     if ( vcpu_create(dom0, 0) == NULL )
-        panic("Error creating domain 0 vcpu0\n");
+        panic("Error creating %pdv0\n", dom0);
 
     rc = construct_dom0(dom0);
     if ( rc )
-        panic("Could not set up DOM0 guest OS (rc = %d)\n", rc);
+        panic("Could not set up %pd guest OS (rc = %d)\n", dom0, rc);
 
     set_xs_domain(dom0);
 }
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 2ff7c28c277b..a740c6f60c63 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1084,7 +1084,7 @@ static struct domain *__init create_dom0(struct boot_info *bi)
 
         if ( (strlen(acpi_param) == 0) && acpi_disabled )
         {
-            printk("ACPI is disabled, notifying Domain 0 (acpi=off)\n");
+            printk("ACPI is disabled, notifying %pd (acpi=off)\n", d);
             safe_strcpy(acpi_param, "off");
         }
 
@@ -1099,7 +1099,7 @@ static struct domain *__init create_dom0(struct boot_info *bi)
 
     bd->d = d;
     if ( construct_dom0(bd) != 0 )
-        panic("Could not construct domain 0\n");
+        panic("Could not construct %pd\n", d);
 
     bd->cmdline = NULL;
     xfree(cmdline);
-- 
2.34.1

Re: [PATCH v11 3/3] xen/domain: update create_dom0() messages

[Jan Beulich]

On 28.07.2025 20:34, dmkhn@proton.me wrote:
> From: Denis Mukhin <dmukhin@ford.com> 
> 
> Use %pd for domain identification in error/panic messages in create_dom0().

Except that ...

> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -2081,17 +2081,17 @@ void __init create_dom0(void)
>  
>      dom0 = domain_create(domid, &dom0_cfg, flags);
>      if ( IS_ERR(dom0) )
> -        panic("Error creating domain 0 (rc = %ld)\n", PTR_ERR(dom0));
> +        panic("Error creating d%d (rc = %ld)\n", domid, PTR_ERR(dom0));

... here you don't (and can't). To avoid people wondering when they later
come across a commit, I think descriptions would better be accurate. It'll
be Arm maintainers to judge whether they let you get away with this; as it
covers x86 only anyway:
Acked-by: Jan Beulich <jbeulich@suse.com>

Jan

Re: [PATCH v11 1/3] xen/domain: unify domain ID allocation

[Alejandro Vallejo]

On Mon Jul 28, 2025 at 8:34 PM CEST, dmkhn wrote:
> From: Denis Mukhin <dmukhin@ford.com> 
>
> Currently, there are two different domain ID allocation implementations:
>
>   1) Sequential IDs allocation in dom0less Arm code based on max_init_domid;
>
>   2) Sequential IDs allocation in XEN_DOMCTL_createdomain; does not use
>      max_init_domid (both Arm and x86).
>
> The domain ID allocation covers dom0 or late hwdom, predefined domains,
> post-boot domains, excluding Xen system domains (domid >=
> DOMID_FIRST_RESERVED).
>
> It makes sense to have a common helper code for such task across architectures
> (Arm and x86) and between dom0less / toolstack domU allocation.
>
> Note, fixing dependency on max_init_domid is out of scope of this patch.

I can see why. The console switch code is a bit annoying. It's unfortunate it
relies on that global for the early wraparound.

>
> Wrap the domain ID allocation as an arch-independent function domid_alloc() in
> new common/domid.c based on the bitmap.
>
> Allocation algorithm:
> - If an explicit domain ID is provided, verify its availability and use it if
>   ID is not used;
> - If DOMID_INVALID is provided, search the range [1..DOMID_FIRST_RESERVED-1],
>   starting from the last used ID. IDs are not wrapped around in dom0less case.
>   Implementation guarantees that two consecutive calls will never return the
>   same ID. ID#0 is reserved for the first boot domain (currently, dom0) and
>   excluded from the allocation range.
>
> Remove is_free_domid() helper as it is not needed now.
>
> No functional change intended.
>
> Signed-off-by: Denis Mukhin <dmukhin@ford.com>
> ---
> Changes since v10:
> - fixup #ifdefs in domid_alloc()
> - corrected use of domid_free() in domain_destroy()
> - rebased
> - moved domid_{alloc,free}() to common/domid.c so the functional test could be
>   added later
> ---
>  xen/arch/arm/domain_build.c             |  7 +-
>  xen/arch/x86/setup.c                    |  7 +-
>  xen/common/Makefile                     |  1 +
>  xen/common/device-tree/dom0less-build.c | 15 ++--
>  xen/common/domain.c                     |  2 +
>  xen/common/domctl.c                     | 42 ++---------
>  xen/common/domid.c                      | 93 +++++++++++++++++++++++++
>  xen/include/xen/domain.h                |  3 +
>  8 files changed, 122 insertions(+), 48 deletions(-)
>  create mode 100644 xen/common/domid.c
>
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 463ae4474d30..789f2b9d3ce7 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -2050,6 +2050,7 @@ void __init create_dom0(void)
>          .grant_opts = XEN_DOMCTL_GRANT_version(opt_gnttab_max_version),
>      };
>      unsigned int flags = CDF_privileged | CDF_hardware;
> +    domid_t domid;
>      int rc;
>  
>      /* The vGIC for DOM0 is exactly emulating the hardware GIC */
> @@ -2074,7 +2075,11 @@ void __init create_dom0(void)
>      if ( !llc_coloring_enabled )
>          flags |= CDF_directmap;
>  
> -    dom0 = domain_create(0, &dom0_cfg, flags);
> +    domid = domid_alloc(0);
> +    if ( domid == DOMID_INVALID )
> +        panic("Error allocating domain ID 0\n");
> +
> +    dom0 = domain_create(domid, &dom0_cfg, flags);
>      if ( IS_ERR(dom0) )
>          panic("Error creating domain 0 (rc = %ld)\n", PTR_ERR(dom0));
>  
> diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
> index 1543dd251cc6..2ff7c28c277b 100644
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -1047,8 +1047,11 @@ static struct domain *__init create_dom0(struct boot_info *bi)
>      if ( iommu_enabled )
>          dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
>  
> -    /* Create initial domain.  Not d0 for pvshim. */
> -    bd->domid = get_initial_domain_id();
> +    /* Allocate initial domain ID.  Not d0 for pvshim. */
> +    bd->domid = domid_alloc(get_initial_domain_id());
> +    if ( bd->domid == DOMID_INVALID )
> +        panic("Error allocating domain ID %d\n", get_initial_domain_id());
> +
>      d = domain_create(bd->domid, &dom0_cfg,
>                        pv_shim ? 0 : CDF_privileged | CDF_hardware);
>      if ( IS_ERR(d) )
> diff --git a/xen/common/Makefile b/xen/common/Makefile
> index c316957fcb36..0c7d0f5d46e1 100644
> --- a/xen/common/Makefile
> +++ b/xen/common/Makefile
> @@ -11,6 +11,7 @@ obj-$(filter-out $(CONFIG_X86),$(CONFIG_ACPI)) += device.o
>  obj-$(CONFIG_DEVICE_TREE_PARSE) += device-tree/
>  obj-$(CONFIG_IOREQ_SERVER) += dm.o
>  obj-y += domain.o
> +obj-y += domid.o
>  obj-y += event_2l.o
>  obj-y += event_channel.o
>  obj-$(CONFIG_EVTCHN_FIFO) += event_fifo.o
> diff --git a/xen/common/device-tree/dom0less-build.c b/xen/common/device-tree/dom0less-build.c
> index 6bb038111de9..1f9461d0e738 100644
> --- a/xen/common/device-tree/dom0less-build.c
> +++ b/xen/common/device-tree/dom0less-build.c
> @@ -833,21 +833,20 @@ void __init create_domUs(void)
>      {
>          struct kernel_info ki = KERNEL_INFO_INIT;
>          int rc = parse_dom0less_node(node, &ki.bd);
> +        domid_t domid;
>  
>          if ( rc == -ENOENT )
>              continue;
>          if ( rc )
>              panic("Malformed DTB: Invalid domain %s\n", dt_node_name(node));
>  
> -        if ( (max_init_domid + 1) >= DOMID_FIRST_RESERVED )
> -            panic("No more domain IDs available\n");
> +        domid = domid_alloc(DOMID_INVALID);
> +        if ( domid == DOMID_INVALID )
> +            panic("Error allocating ID for domain %s\n", dt_node_name(node));
> +        if ( max_init_domid < domid )
> +            max_init_domid = domid;

nit: This is open-coding max_init_domid = max(max_init_domid, domid);

>  
> -        /*
> -         * The variable max_init_domid is initialized with zero, so here it's
> -         * very important to use the pre-increment operator to call
> -         * domain_create() with a domid > 0. (domid == 0 is reserved for Dom0)
> -         */
> -        ki.bd.d = domain_create(++max_init_domid,
> +        ki.bd.d = domain_create(domid,
>                                  &ki.bd.create_cfg, ki.bd.create_flags);

nit: With this change, the wrapped line can be unfolded into a single one. 

>          if ( IS_ERR(ki.bd.d) )
>              panic("Error creating domain %s (rc = %ld)\n",
> diff --git a/xen/common/domain.c b/xen/common/domain.c
> index 3c65cca5b0ff..23dbc1f46c78 100644
> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -1466,6 +1466,8 @@ void domain_destroy(struct domain *d)
>      /* Remove from the domlist/hash. */
>      domlist_remove(d);
>  
> +    domid_free(d->domain_id);
> +
>      /* Schedule RCU asynchronous completion of domain destroy. */
>      call_rcu(&d->rcu, complete_domain_destroy);
>  }
> diff --git a/xen/common/domctl.c b/xen/common/domctl.c
> index f2a7caaf853c..5509998aa139 100644
> --- a/xen/common/domctl.c
> +++ b/xen/common/domctl.c
> @@ -51,20 +51,6 @@ static int xenctl_bitmap_to_nodemask(nodemask_t *nodemask,
>                                     MAX_NUMNODES);
>  }
q  
> -static inline int is_free_domid(domid_t dom)
> -{
> -    struct domain *d;
> -
> -    if ( dom >= DOMID_FIRST_RESERVED )
> -        return 0;
> -
> -    if ( (d = rcu_lock_domain_by_id(dom)) == NULL )
> -        return 1;
> -
> -    rcu_unlock_domain(d);
> -    return 0;
> -}
> -
>  void getdomaininfo(struct domain *d, struct xen_domctl_getdomaininfo *info)
>  {
>      struct vcpu *v;
> @@ -423,36 +409,18 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
>  
>      case XEN_DOMCTL_createdomain:
>      {
> -        domid_t        dom;
> -        static domid_t rover = 0;
> +        domid_t domid = domid_alloc(op->domain);
>  
> -        dom = op->domain;
> -        if ( (dom > 0) && (dom < DOMID_FIRST_RESERVED) )
> +        if ( domid == DOMID_INVALID )
>          {
>              ret = -EEXIST;
> -            if ( !is_free_domid(dom) )
> -                break;
> -        }
> -        else
> -        {
> -            for ( dom = rover + 1; dom != rover; dom++ )
> -            {
> -                if ( dom == DOMID_FIRST_RESERVED )
> -                    dom = 1;
> -                if ( is_free_domid(dom) )
> -                    break;
> -            }
> -
> -            ret = -ENOMEM;
> -            if ( dom == rover )
> -                break;
> -
> -            rover = dom;
> +            break;
>          }
>  
> -        d = domain_create(dom, &op->u.createdomain, false);
> +        d = domain_create(domid, &op->u.createdomain, false);
>          if ( IS_ERR(d) )
>          {
> +            domid_free(domid);
>              ret = PTR_ERR(d);
>              d = NULL;
>              break;
> diff --git a/xen/common/domid.c b/xen/common/domid.c
> new file mode 100644
> index 000000000000..e553ab6e5468
> --- /dev/null
> +++ b/xen/common/domid.c
> @@ -0,0 +1,93 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Domain ID allocator.
> + * Covers dom0 or late hwdom, predefined domains, post-boot domains; excludes
> + * Xen system domains (ID >= DOMID_FIRST_RESERVED).
> + *
> + * Copyright 2025 Ford Motor Company
> + */
> +
> +#include <xen/domain.h>
> +
> +static DEFINE_SPINLOCK(domid_lock);
> +static DECLARE_BITMAP(domid_bitmap, DOMID_FIRST_RESERVED);
> +
> +/*
q + * Allocate domain ID.
> + *
> + * @param[in] domid Exact domain ID within [0..DOMID_FIRST_RESERVED-1] range or

nit: [in] is inconsequential, not being a pointer.

> + *                  DOMID_INVALID for exhaustive search within
> + *                  [1..DOMID_FIRST_RESERVED-1].
> + * @return Valid domain ID in case of successful allocation,
> + *         DOMID_INVALID - otherwise.
> + */
> +domid_t cf_check domid_alloc(domid_t domid)

Why cf_check? That's only needed when using indirection (i.e) function pointers
so the call works with IBT. I don't think that's needed here?

> +{
> +    static domid_t domid_last;
> +
> +    spin_lock(&domid_lock);
> +
> +    /* Exact match. */
> +    if ( domid < DOMID_FIRST_RESERVED )
> +    {
> +        if ( __test_and_set_bit(domid, domid_bitmap) )
> +            domid = DOMID_INVALID;
> +    }
> +    /*
> +     * Exhaustive search.
> +     *
> +     * Domain ID#0 is reserved for the first boot domain (e.g. control domain)
> +     * and excluded from allocation.
> +     *
> +     * In dom0less build, domains are not dynamically destroyed, so there's no
> +     * need to do a wraparound of the IDs.
> +     */
> +#ifdef CONFIG_DOM0LESS_BOOT
> +    else if ( domid_last + 1 >= DOMID_FIRST_RESERVED )
> +        domid = DOMID_INVALID;
> +#endif

This hunk with the guards breaks dom0less-capable Xen booting a non-dom0less
system (which effectively means any defconfig arm build). dom0 boots must wrap
around, irrespective of whether that same build could boot from a DTB.

The point of not wrapping around is merely a conservative check to ensure
dom0less-build can find a free domid by bump-allocating. This is already the
case with your code, so there's no need to special-case dom0less.

Just remove it.

> +    else
> +    {
> +        domid = find_next_zero_bit(domid_bitmap,
> +                                   DOMID_FIRST_RESERVED,
> +                                   domid_last + 1);
> +#ifndef CONFIG_DOM0LESS_BOOT
> +        ASSERT(domid <= DOMID_FIRST_RESERVED);
> +        if ( domid == DOMID_FIRST_RESERVED )
> +            domid = find_next_zero_bit(domid_bitmap,
> +                                       DOMID_FIRST_RESERVED,
> +                                       1);
> +#endif

Like before, this must be without the ifdef guards. Otherwise it breaks
non-dom0less runs on dom0less-enabled builds.

> +
> +        if ( domid < DOMID_FIRST_RESERVED )
> +        {
> +            __set_bit(domid, domid_bitmap);
> +            domid_last = domid;
> +        }
> +        else
> +            domid = DOMID_INVALID;
> +    }
> +
> +    spin_unlock(&domid_lock);
> +
> +    return domid;
> +}
> +
> +void cf_check domid_free(domid_t domid)
> +{
> +    ASSERT(domid <= DOMID_FIRST_RESERVED);
> +
> +    spin_lock(&domid_lock);

Might be worth asserting the bit is set. That way we'll catch XSAs in CI if
we try to incorrectly move the current calls to domid_free().

> +    __clear_bit(domid, domid_bitmap);
> +    spin_unlock(&domid_lock);
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/xen/domain.h b/xen/include/xen/domain.h
> index e10baf2615fd..31946bb1b653 100644
> --- a/xen/include/xen/domain.h
> +++ b/xen/include/xen/domain.h
> @@ -38,6 +38,9 @@ void arch_get_domain_info(const struct domain *d,
>  
>  domid_t get_initial_domain_id(void);
>  
> +domid_t cf_check domid_alloc(domid_t domid);
> +void cf_check domid_free(domid_t domid);

Neither of them should need the cf_check. Also, could you perhaps move the
explanation of how the allocation scheme works from the commit message into
the domid_alloc() prototype? That way callers can reason about the API without
git-blaming to find out.

> +
>  /* CDF_* constant. Internal flags for domain creation. */
>  /* Is this a privileged domain? */
>  #define CDF_privileged           (1U << 0)

Re: [PATCH v11 1/3] xen/domain: unify domain ID allocation

[dmkhn]

On Tue, Jul 29, 2025 at 12:34:25PM +0200, Alejandro Vallejo wrote:

Thanks for review!

> On Mon Jul 28, 2025 at 8:34 PM CEST, dmkhn wrote:
> > From: Denis Mukhin <dmukhin@ford.com>
> >
> > Currently, there are two different domain ID allocation implementations:
> >
> >   1) Sequential IDs allocation in dom0less Arm code based on max_init_domid;
> >
> >   2) Sequential IDs allocation in XEN_DOMCTL_createdomain; does not use
> >      max_init_domid (both Arm and x86).
> >
> > The domain ID allocation covers dom0 or late hwdom, predefined domains,
> > post-boot domains, excluding Xen system domains (domid >=
> > DOMID_FIRST_RESERVED).
> >
> > It makes sense to have a common helper code for such task across architectures
> > (Arm and x86) and between dom0less / toolstack domU allocation.
> >
> > Note, fixing dependency on max_init_domid is out of scope of this patch.
> 
> I can see why. The console switch code is a bit annoying. It's unfortunate it
> relies on that global for the early wraparound.

Yep, I have a series which removes `max_init_domid` here:
  https://lore.kernel.org/xen-devel/20250530231841.73386-1-dmukhin@ford.com/

Re: [PATCH v11 3/3] xen/domain: update create_dom0() messages

[dmkhn]

On Tue, Jul 29, 2025 at 10:11:31AM +0200, Jan Beulich wrote:
> On 28.07.2025 20:34, dmkhn@proton.me wrote:
> > From: Denis Mukhin <dmukhin@ford.com>
> >
> > Use %pd for domain identification in error/panic messages in create_dom0().
> 
> Except that ...
> 
> > --- a/xen/arch/arm/domain_build.c
> > +++ b/xen/arch/arm/domain_build.c
> > @@ -2081,17 +2081,17 @@ void __init create_dom0(void)
> >
> >      dom0 = domain_create(domid, &dom0_cfg, flags);
> >      if ( IS_ERR(dom0) )
> > -        panic("Error creating domain 0 (rc = %ld)\n", PTR_ERR(dom0));
> > +        panic("Error creating d%d (rc = %ld)\n", domid, PTR_ERR(dom0));
> 
> ... here you don't (and can't). To avoid people wondering when they later

Thanks for catching this! Will update.

> come across a commit, I think descriptions would better be accurate. It'll
> be Arm maintainers to judge whether they let you get away with this; as it
> covers x86 only anyway:
> Acked-by: Jan Beulich <jbeulich@suse.com>

Thanks!

> 
> Jan