[PATCH v2 00/26] Introduce enablemenant of dom0less

[PATCH v2 00/26] Introduce enablemenant of dom0less

[Oleksii Kurochko]

This patch series reprensent a bunch of patches necessary to enable common part
of Dom0less.
The stuff necessary to start/launch domains will be introduced separately.

This patch series is based on [1], but a lot of patch could go even without
it.

[1] https://lore.kernel.org/xen-devel/cover.1778140240.git.oleksii.kurochko@gmail.com/T/#t

CI tests: https://gitlab.com/xen-project/people/olkur/xen/-/pipelines/2510638764

---
Changes in v2:
 - Move patch "[PATCH v1 04/27] xen/riscv: rework G-stage mode handling" to
   patch series [1]
 - Address the comments from ML.
 - The following patches were folded into one:
   # xen/riscv: implement init_intc_phandle()
   # xen/riscv: call do_initcalls() in start_xen()
   # xen/riscv: setup system domains
 - The following patch were folded into one:
   # xen/riscv: add vaplic access check
   # xen/riscv: emulate guest writes to virtual APLIC MMIO
   # xen/riscv: emulate guest reads from virtual APLIC MMIO
 - Add new bug fix, not really necessary to this patch series:
   xen/riscv: manage IRQ_DISABLED flag in APLIC irq enable/disable callbacks
---

Oleksii Kurochko (26):
  xen: arm: update p2m_set_allocation() prototype
  xen/riscv: Implement ARCH_PAGING_MEMPOOL
  xen/riscv: Implement construct_domain()
  xen/riscv: implement prerequisites for domain_create()
  xen/riscv: introduce guest riscv,isa string
  xen/riscv: implement make_cpus_node()
  xen/riscv: implement make_timer_node()
  xen/riscv: implement make_arch_nodes()
  xen/riscv: introduce init interrupt controller operations
  xen/riscv: implement make_intc_domU_node()
  xen/riscv: introduce aia_init() and aia_usable()
  xen/riscv: add basic VGEIN management for AIA guests
  xen/riscv: introduce per-vCPU IMSIC state
  xen/riscv: add very early virtual APLIC (vAPLIC) initialization
    support
  xen/riscv: introduce (de)initialization helpers for vINTC
  xen/riscv: create APLIC DT node for guest domains
  xen/riscv: generate IMSIC DT node for guest domains
  xen: move declaration of map_device_irqs_to_domain() to common header
  xen/riscv: implement IRQ routing for device passthrough
  xen/riscv: add missing APLIC register offsets, masks to asm/aplic.h.
  xen/riscv: implement virtual APLIC MMIO emulation
  xen/riscv: implement init_intc_phandle()
  xen/riscv: initialize RCU, scheduler, and system domains in
    start_xen()
  xen/riscv: provide init_vuart()
  xen/riscv: add initial dom0less infrastructure support
  xen/riscv: manage IRQ_DISABLED flag in APLIC irq enable/disable
    callbacks

 automation/gitlab-ci/build.yaml           |   1 +
 xen/arch/arm/include/asm/p2m.h            |   1 -
 xen/arch/arm/include/asm/setup.h          |   3 -
 xen/arch/arm/mmu/p2m.c                    |  22 +-
 xen/arch/riscv/Kconfig                    |   3 +
 xen/arch/riscv/Makefile                   |   4 +
 xen/arch/riscv/aia.c                      | 167 +++++++++
 xen/arch/riscv/aplic.c                    |  49 ++-
 xen/arch/riscv/cpufeature.c               |  59 ++-
 xen/arch/riscv/device.c                   | 108 ++++++
 xen/arch/riscv/dom0less-build.c           |  39 ++
 xen/arch/riscv/domain-build.c             | 179 +++++++++
 xen/arch/riscv/domain.c                   |  44 ++-
 xen/arch/riscv/imsic.c                    | 167 +++++++++
 xen/arch/riscv/include/asm/aia.h          |  18 +
 xen/arch/riscv/include/asm/aplic.h        |  53 +++
 xen/arch/riscv/include/asm/cpufeature.h   |   5 +
 xen/arch/riscv/include/asm/domain.h       |   8 +
 xen/arch/riscv/include/asm/guest-layout.h |   7 +
 xen/arch/riscv/include/asm/imsic.h        |  26 ++
 xen/arch/riscv/include/asm/intc.h         |  58 ++-
 xen/arch/riscv/include/asm/irq.h          |   3 +
 xen/arch/riscv/include/asm/paging.h       |   2 +-
 xen/arch/riscv/include/asm/setup.h        |   8 +-
 xen/arch/riscv/include/asm/vaplic.h       |  37 ++
 xen/arch/riscv/intc.c                     | 103 ++++-
 xen/arch/riscv/irq.c                      | 178 ++++++++-
 xen/arch/riscv/p2m.c                      |  31 +-
 xen/arch/riscv/paging.c                   |   7 +-
 xen/arch/riscv/setup.c                    |  14 +
 xen/arch/riscv/stubs.c                    |  12 -
 xen/arch/riscv/vaplic.c                   | 436 ++++++++++++++++++++++
 xen/common/device-tree/dom0less-build.c   |   2 +-
 xen/include/xen/fdt-domain-build.h        |  13 +
 xen/include/xen/p2m-common.h              |   2 +
 35 files changed, 1807 insertions(+), 62 deletions(-)
 create mode 100644 xen/arch/riscv/aia.c
 create mode 100644 xen/arch/riscv/device.c
 create mode 100644 xen/arch/riscv/domain-build.c
 create mode 100644 xen/arch/riscv/include/asm/aia.h
 create mode 100644 xen/arch/riscv/include/asm/vaplic.h
 create mode 100644 xen/arch/riscv/vaplic.c

-- 
2.54.0

[PATCH v2 01/26] xen: arm: update p2m_set_allocation() prototype

[Oleksii Kurochko]

p2m_set_allocation() signals preemption redundantly: via a bool *preempted
out-argument (set to true) and via -ERESTART return code, both at the same
time. This led to the caller-side ASSERT(preempted == (rc == -ERESTART))
added solely to document their agreement.

Drop the out-argument entirely. The return code alone is sufficient to
distinguish preemption (-ERESTART) from success (0) or other failures.
Replace the pointer with a plain bool can_preempt that controls whether
the preemption check is performed at all, making the previous
NULL-to-suppress-preemption calling convention explicit and type-safe.

Since p2m_set_allocation() is called by the common dom0less build code,
move its declaration from the ARM-specific asm/p2m.h to xen/p2m-common.h.

Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - new patch
---
 xen/arch/arm/include/asm/p2m.h          |  1 -
 xen/arch/arm/mmu/p2m.c                  | 22 ++++++++--------------
 xen/arch/riscv/include/asm/paging.h     |  2 +-
 xen/arch/riscv/p2m.c                    |  7 ++++---
 xen/arch/riscv/paging.c                 |  7 ++-----
 xen/common/device-tree/dom0less-build.c |  2 +-
 xen/include/xen/p2m-common.h            |  2 ++
 7 files changed, 18 insertions(+), 25 deletions(-)

diff --git a/xen/arch/arm/include/asm/p2m.h b/xen/arch/arm/include/asm/p2m.h
index 7957dbd96e57..62b37a05f79e 100644
--- a/xen/arch/arm/include/asm/p2m.h
+++ b/xen/arch/arm/include/asm/p2m.h
@@ -232,7 +232,6 @@ void p2m_restore_state(struct vcpu *n);
 /* Print debugging/statistial info about a domain's p2m */
 void p2m_dump_info(struct domain *d);
 
-int p2m_set_allocation(struct domain *d, unsigned long pages, bool *preempted);
 int p2m_teardown_allocation(struct domain *d);
 
 static inline void p2m_write_lock(struct p2m_domain *p2m)
diff --git a/xen/arch/arm/mmu/p2m.c b/xen/arch/arm/mmu/p2m.c
index 51abf3504fcf..e5c6be7c0890 100644
--- a/xen/arch/arm/mmu/p2m.c
+++ b/xen/arch/arm/mmu/p2m.c
@@ -67,10 +67,11 @@ int arch_get_paging_mempool_size(struct domain *d, uint64_t *size)
 
 /*
  * Set the pool of pages to the required number of pages.
- * Returns 0 for success, non-zero for failure.
+ * Returns 0 for success, -ERESTART if preempted, or a negative error code on
+ * failure.
  * Call with d->arch.paging.lock held.
  */
-int p2m_set_allocation(struct domain *d, unsigned long pages, bool *preempted)
+int p2m_set_allocation(struct domain *d, unsigned long pages, bool can_preempt)
 {
     struct page_info *pg;
 
@@ -112,11 +113,8 @@ int p2m_set_allocation(struct domain *d, unsigned long pages, bool *preempted)
             break;
 
         /* Check to see if we need to yield and try again */
-        if ( preempted && general_preempt_check() )
-        {
-            *preempted = true;
+        if ( can_preempt && general_preempt_check() )
             return -ERESTART;
-        }
     }
 
     return 0;
@@ -125,7 +123,6 @@ int p2m_set_allocation(struct domain *d, unsigned long pages, bool *preempted)
 int arch_set_paging_mempool_size(struct domain *d, uint64_t size)
 {
     unsigned long pages = size >> PAGE_SHIFT;
-    bool preempted = false;
     int rc;
 
     if ( (size & ~PAGE_MASK) ||          /* Non page-sized request? */
@@ -133,27 +130,24 @@ int arch_set_paging_mempool_size(struct domain *d, uint64_t size)
         return -EINVAL;
 
     spin_lock(&d->arch.paging.lock);
-    rc = p2m_set_allocation(d, pages, &preempted);
+    rc = p2m_set_allocation(d, pages, true);
     spin_unlock(&d->arch.paging.lock);
 
-    ASSERT(preempted == (rc == -ERESTART));
-
     return rc;
 }
 
 int p2m_teardown_allocation(struct domain *d)
 {
     int ret = 0;
-    bool preempted = false;
 
     spin_lock(&d->arch.paging.lock);
     if ( d->arch.paging.p2m_total_pages != 0 )
     {
-        ret = p2m_set_allocation(d, 0, &preempted);
-        if ( preempted )
+        ret = p2m_set_allocation(d, 0, true);
+        if ( ret == -ERESTART )
         {
             spin_unlock(&d->arch.paging.lock);
-            return -ERESTART;
+            return ret;
         }
         ASSERT(d->arch.paging.p2m_total_pages == 0);
     }
diff --git a/xen/arch/riscv/include/asm/paging.h b/xen/arch/riscv/include/asm/paging.h
index e487c89a4ccd..103384723dc5 100644
--- a/xen/arch/riscv/include/asm/paging.h
+++ b/xen/arch/riscv/include/asm/paging.h
@@ -9,7 +9,7 @@ struct page_info;
 int paging_domain_init(struct domain *d);
 
 int paging_freelist_adjust(struct domain *d, unsigned long pages,
-                           bool *preempted);
+                           bool can_preempt);
 
 int paging_ret_to_domheap(struct domain *d, unsigned int nr_pages);
 int paging_refill_from_domheap(struct domain *d, unsigned int nr_pages);
diff --git a/xen/arch/riscv/p2m.c b/xen/arch/riscv/p2m.c
index 703b9f4d2540..41d6d3d5e699 100644
--- a/xen/arch/riscv/p2m.c
+++ b/xen/arch/riscv/p2m.c
@@ -430,15 +430,16 @@ int p2m_init(struct domain *d, const struct xen_domctl_createdomain *config)
 
 /*
  * Set the pool of pages to the required number of pages.
- * Returns 0 for success, non-zero for failure.
+ * Returns 0 for success, -ERESTART if preempted, or a negative error code on
+ * failure.
  * Call with d->arch.paging.lock held.
  */
-int p2m_set_allocation(struct domain *d, unsigned long pages, bool *preempted)
+int p2m_set_allocation(struct domain *d, unsigned long pages, bool can_preempt)
 {
     struct p2m_domain *p2m = p2m_get_hostp2m(d);
     int rc;
 
-    if ( (rc = paging_freelist_adjust(d, pages, preempted)) )
+    if ( (rc = paging_freelist_adjust(d, pages, can_preempt)) )
         return rc;
 
     /*
diff --git a/xen/arch/riscv/paging.c b/xen/arch/riscv/paging.c
index 76a203edbb0c..35f572689a7c 100644
--- a/xen/arch/riscv/paging.c
+++ b/xen/arch/riscv/paging.c
@@ -47,7 +47,7 @@ static int _paging_add_to_freelist(struct domain *d)
 }
 
 int paging_freelist_adjust(struct domain *d, unsigned long pages,
-                           bool *preempted)
+                           bool can_preempt)
 {
     ASSERT(spin_is_locked(&d->arch.paging.lock));
 
@@ -66,11 +66,8 @@ int paging_freelist_adjust(struct domain *d, unsigned long pages,
             return rc;
 
         /* Check to see if we need to yield and try again */
-        if ( preempted && general_preempt_check() )
-        {
-            *preempted = true;
+        if ( can_preempt && general_preempt_check() )
             return -ERESTART;
-        }
     }
 
     return 0;
diff --git a/xen/common/device-tree/dom0less-build.c b/xen/common/device-tree/dom0less-build.c
index 6d6882a34b5a..1058699ab25b 100644
--- a/xen/common/device-tree/dom0less-build.c
+++ b/xen/common/device-tree/dom0less-build.c
@@ -747,7 +747,7 @@ static int __init domain_p2m_set_allocation(struct domain *d, uint64_t mem,
                 domain_p2m_pages(mem, d->max_vcpus);
 
     spin_lock(&d->arch.paging.lock);
-    rc = p2m_set_allocation(d, p2m_pages, NULL);
+    rc = p2m_set_allocation(d, p2m_pages, false);
     spin_unlock(&d->arch.paging.lock);
 
     return rc;
diff --git a/xen/include/xen/p2m-common.h b/xen/include/xen/p2m-common.h
index f0bd9a6b9896..1b44ec8ce36c 100644
--- a/xen/include/xen/p2m-common.h
+++ b/xen/include/xen/p2m-common.h
@@ -43,5 +43,7 @@ int __must_check check_get_page_from_gfn(struct domain *d, gfn_t gfn,
                                          bool readonly, p2m_type_t *p2mt_p,
                                          struct page_info **page_p);
 
+int p2m_set_allocation(struct domain *d, unsigned long pages,
+                       bool can_preempt);
 
 #endif /* _XEN_P2M_COMMON_H */
-- 
2.54.0

[PATCH v2 02/26] xen/riscv: Implement ARCH_PAGING_MEMPOOL

[Oleksii Kurochko]

The p2m_freelist is used to allocate pages for the P2M. To initialize
this list, domain_p2m_set_allocation() may be called from construct_domU()
in the common Dom0less code, so RISC-V provides an implementation and
enables CONFIG_ARCH_PAGING_MEMPOOL unconditionally.

Additionally, implement arch_{set,get}_paging_mempool_size(). They are
not directly used yet, but are required to support the
XEN_DOMCTL_{get,set}_paging_mempool_size hypercalls.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - Turn on CONFIG_ARCH_PAGING_MEMPOOL=y unconditionally and drop all ifdef-s
   related to this config.
 - Optimize check inside arch_set_paging_mempool_size which verify size
   argument.
 - Use pfn_to_paddr() inside arch_get_paging_mempool_size() instead of open
   coding the stuff.
 - Drop ASSERT() from arch_set_paging_mempool_size() as it is impossible to
   have here preempted = true and rc != -ERESTART.
---
---
 xen/arch/riscv/Kconfig |  1 +
 xen/arch/riscv/p2m.c   | 24 ++++++++++++++++++++++++
 2 files changed, 25 insertions(+)

diff --git a/xen/arch/riscv/Kconfig b/xen/arch/riscv/Kconfig
index 41426c205292..48520588fe40 100644
--- a/xen/arch/riscv/Kconfig
+++ b/xen/arch/riscv/Kconfig
@@ -1,5 +1,6 @@
 config RISCV
 	def_bool y
+	select ARCH_PAGING_MEMPOOL
 	select DOMAIN_BUILD_HELPERS
 	select FUNCTION_ALIGNMENT_16B
 	select GENERIC_BUG_FRAME
diff --git a/xen/arch/riscv/p2m.c b/xen/arch/riscv/p2m.c
index 41d6d3d5e699..bed997bd9d5a 100644
--- a/xen/arch/riscv/p2m.c
+++ b/xen/arch/riscv/p2m.c
@@ -1606,3 +1606,27 @@ struct page_info *get_page_from_gfn(struct domain *d, unsigned long gfn,
 
     return p2m_get_page_from_gfn(p2m_get_hostp2m(d), _gfn(gfn), t);
 }
+
+int arch_set_paging_mempool_size(struct domain *d, uint64_t size)
+{
+    unsigned long pages = PFN_DOWN(size);
+    int rc;
+
+    /* Non page-sized request or 32-bit overflow? */
+    if ( pfn_to_paddr(pages) != size )
+        return -EINVAL;
+
+    spin_lock(&d->arch.paging.lock);
+    rc = p2m_set_allocation(d, pages, true);
+    spin_unlock(&d->arch.paging.lock);
+
+    return rc;
+}
+
+/* Return the size of the pool, in bytes. */
+int arch_get_paging_mempool_size(struct domain *d, uint64_t *size)
+{
+    *size =  pfn_to_paddr(ACCESS_ONCE(d->arch.paging.total_pages));
+
+    return 0;
+}
-- 
2.54.0

[PATCH v2 03/26] xen/riscv: Implement construct_domain()

[Oleksii Kurochko]

Implement construct_domain() function for RISC-V, which performs initial setup
for the domain's first vCPU, loads the kernel, initrd, and device tree,
and sets up guest CPU registers for boot.

It also creates additional vCPUs up to max_vcpus and assigns the device tree
address and boot cpuid in registers.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - Rework construct_domain() to print that vCPU1...n are created using %pv.
 - Use true instead of 1 for initialization of v->is_initialised.
 - Drop unnessary BUG_ON() in construct_domain().
 - Add TODO comment above *_load() functions.
---
---
 xen/arch/riscv/Makefile       |  1 +
 xen/arch/riscv/domain-build.c | 52 +++++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)
 create mode 100644 xen/arch/riscv/domain-build.c

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 8f7fd625dddd..fd5e499eb4b9 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -1,6 +1,7 @@
 obj-y += aplic.o
 obj-y += cpufeature.o
 obj-y += domain.o
+obj-y += domain-build.init.o
 obj-$(CONFIG_DOM0LESS_BOOT) += dom0less-build.init.o
 obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
 obj-y += entry.o
diff --git a/xen/arch/riscv/domain-build.c b/xen/arch/riscv/domain-build.c
new file mode 100644
index 000000000000..492327631e33
--- /dev/null
+++ b/xen/arch/riscv/domain-build.c
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include <xen/fdt-domain-build.h>
+#include <xen/fdt-kernel.h>
+#include <xen/init.h>
+#include <xen/sched.h>
+
+#include <asm/current.h>
+#include <asm/guest_access.h>
+
+int __init construct_domain(struct domain *d, struct kernel_info *kinfo)
+{
+    struct vcpu *v = d->vcpu[0];
+    struct cpu_user_regs *regs = vcpu_guest_cpu_user_regs(v);
+
+    BUG_ON(v->is_initialised);
+
+    /*
+     * At the moment *_load() don't return value and will just panic()
+     * inside.
+     * TODO: it will be good to change that.
+     */
+    kernel_load(kinfo);
+    initrd_load(kinfo, copy_to_guest_phys);
+    dtb_load(kinfo, copy_to_guest_phys);
+
+    regs->sepc = kinfo->entry;
+
+    /* Guest boot cpuid = 0 */
+    regs->a0 = 0;
+    regs->a1 = kinfo->dtb_paddr;
+
+    for ( unsigned int i = 1; i < d->max_vcpus; i++ )
+    {
+        const struct vcpu *tmp_v = vcpu_create(d, i);
+
+        if ( !tmp_v )
+        {
+            printk("Failed to allocate %pd v%d\n", d, i);
+            break;
+        }
+
+        dprintk(XENLOG_INFO, "Created vcpu %pv\n", tmp_v);
+    }
+
+    domain_update_node_affinity(d);
+
+    v->is_initialised = true;
+    clear_bit(_VPF_down, &v->pause_flags);
+
+    return 0;
+}
-- 
2.54.0

[PATCH v2 04/26] xen/riscv: implement prerequisites for domain_create()

[Oleksii Kurochko]

arch_domain_create() and arch_sanitise_domain_config() are prerequisites for
domain_create().

arch_sanitise_domain_config() currently returns 0, as there is no specific
work required at this stage.

arch_domain_create() performs basic initialization, such as setting up the P2M
and initializing of next unused phandle.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - update the commit message.
 - Drop vcpu_switch_to_aarch64_mode() from riscv/stubs. It shouldn't be under
   riscv/ at all.
 - Drop next_phandle as it is now in common code.
---
 xen/arch/riscv/domain.c | 25 +++++++++++++++++++++++++
 xen/arch/riscv/stubs.c  | 12 ------------
 2 files changed, 25 insertions(+), 12 deletions(-)

diff --git a/xen/arch/riscv/domain.c b/xen/arch/riscv/domain.c
index c77be3b827eb..669dd27d79a3 100644
--- a/xen/arch/riscv/domain.c
+++ b/xen/arch/riscv/domain.c
@@ -289,6 +289,31 @@ void sync_vcpu_execstate(struct vcpu *v)
     /* Nothing to do -- no lazy switching */
 }
 
+int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
+{
+    return 0;
+}
+
+int arch_domain_create(struct domain *d,
+                       struct xen_domctl_createdomain *config,
+                       unsigned int flags)
+{
+    int rc = 0;
+
+    if ( is_idle_domain(d) )
+        return 0;
+
+    if ( (rc = p2m_init(d, config)) != 0)
+        goto fail;
+
+    return rc;
+
+ fail:
+    d->is_dying = DOMDYING_dead;
+    arch_domain_destroy(d);
+    return rc;
+}
+
 static void __init __maybe_unused build_assertions(void)
 {
     /*
diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
index acbb5b9123ea..cfb12d51d00b 100644
--- a/xen/arch/riscv/stubs.c
+++ b/xen/arch/riscv/stubs.c
@@ -101,18 +101,6 @@ void dump_pageframe_info(struct domain *d)
     BUG_ON("unimplemented");
 }
 
-int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
-{
-    BUG_ON("unimplemented");
-}
-
-int arch_domain_create(struct domain *d,
-                       struct xen_domctl_createdomain *config,
-                       unsigned int flags)
-{
-    BUG_ON("unimplemented");
-}
-
 int arch_domain_teardown(struct domain *d)
 {
     BUG_ON("unimplemented");
-- 
2.54.0

[PATCH v2 05/26] xen/riscv: introduce guest riscv,isa string

[Oleksii Kurochko]

Introduce generation of the riscv,isa string passed to the guest via the
Device Tree riscv,isa property.

Introduce the per-domain isa string and guest isa bitmap, populated
during domain creation by calling init_guest_isa().

Introduce guest_unsupp to filter out ISA extensions that should not be
exposed to guests:

- f/d/q/v: FPU and vector context save/restore are not yet implemented
  for guests.
- h: Nested virtualisation is not supported.
- sstc: Xen owns the supervisor timer; guests must use SBI.
- svade: Xen manages hardware A/D bit updates in stage-2 page tables.
- svpbmt: Page-based memory types are not yet wired up in stage-2 code.

Drop __initconst for riscv_isa_ext() as it can be used after init stage
by init_guest_isa().

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - s/guest_unsupp_bmp/guest_unsupp.
 - Drop guest_isa_str.
 - Provide init_guest_isa() instead of polluting match_isa_ext().
 - Drop xlen.
 - Add the comment about guest_unsupp.
 - Update the way how guest_unsupp is init-ed.
 - Drop __initconst for riscv_isa_ext[] as it is used in init_guest_isa()
   which isn't marked as __init as it could be used after init stage.
---
---
 xen/arch/riscv/cpufeature.c             | 59 ++++++++++++++++++++++++-
 xen/arch/riscv/domain.c                 |  3 ++
 xen/arch/riscv/include/asm/cpufeature.h |  5 +++
 xen/arch/riscv/include/asm/domain.h     |  4 ++
 4 files changed, 70 insertions(+), 1 deletion(-)

diff --git a/xen/arch/riscv/cpufeature.c b/xen/arch/riscv/cpufeature.c
index 92235fdfd5ab..618592e6096a 100644
--- a/xen/arch/riscv/cpufeature.c
+++ b/xen/arch/riscv/cpufeature.c
@@ -14,6 +14,7 @@
 #include <xen/errno.h>
 #include <xen/init.h>
 #include <xen/lib.h>
+#include <xen/sched.h>
 #include <xen/sections.h>
 
 #include <asm/cpufeature.h>
@@ -120,7 +121,7 @@ static int __init dt_get_cpuid_from_node(const struct dt_device_node *cpu,
  * and strncmp() is used in match_isa_ext() to compare extension names instead
  * of strncasecmp().
  */
-const struct riscv_isa_ext_data __initconst riscv_isa_ext[] = {
+const struct riscv_isa_ext_data riscv_isa_ext[] = {
     RISCV_ISA_EXT_DATA(i),
     RISCV_ISA_EXT_DATA(m),
     RISCV_ISA_EXT_DATA(a),
@@ -128,6 +129,7 @@ const struct riscv_isa_ext_data __initconst riscv_isa_ext[] = {
     RISCV_ISA_EXT_DATA(d),
     RISCV_ISA_EXT_DATA(q),
     RISCV_ISA_EXT_DATA(c),
+    RISCV_ISA_EXT_DATA(v),
     RISCV_ISA_EXT_DATA(h),
     RISCV_ISA_EXT_DATA(zicntr),
     RISCV_ISA_EXT_DATA(zicsr),
@@ -160,6 +162,12 @@ static const struct riscv_isa_ext_data __initconst required_extensions[] = {
     RISCV_ISA_EXT_DATA(svpbmt),
 };
 
+/*
+ * Everything in riscv_isa_ext[] which shouldn't be exposed to guests should
+ * appear here.
+ */
+static __ro_after_init DECLARE_BITMAP(guest_unsupp, RISCV_ISA_EXT_MAX);
+
 static bool __init is_lowercase_extension_name(const char *str)
 {
     /*
@@ -480,6 +488,53 @@ bool riscv_isa_extension_available(const unsigned long *isa_bitmap,
     return test_bit(id, isa_bitmap);
 }
 
+int init_guest_isa(struct domain *d)
+{
+    char *buf = d->arch.guest_isa_str;
+    size_t len = sizeof(d->arch.guest_isa_str);
+
+    bitmap_andnot(d->arch.guest_isa, riscv_isa, guest_unsupp,
+                  RISCV_ISA_EXT_MAX);
+
+#if defined(CONFIG_RISCV_32)
+    if ( snprintf(buf, len, "rv32") >= len )
+        return -ENOBUFS;
+#elif defined(CONFIG_RISCV_64)
+    if ( snprintf(buf, len, "rv64") >= len )
+        return -ENOBUFS;
+#else
+#   error "Unsupported RISC-V bitness"
+#endif
+
+    for ( unsigned int i = 0; i < ARRAY_SIZE(riscv_isa_ext); i++ )
+    {
+        const struct riscv_isa_ext_data *ext = &riscv_isa_ext[i];
+
+        if ( !riscv_isa_extension_available(d->arch.guest_isa, ext->id) )
+            continue;
+
+        if ( ext->id >= RISCV_ISA_EXT_BASE && strlcat(buf, "_", len) >= len )
+            return -ENOBUFS;
+
+        if ( strlcat(buf, ext->name, len) >= len )
+            return -ENOBUFS;
+    }
+
+    return 0;
+}
+
+static void __init init_guest_unsupp(void)
+{
+    set_bit(RISCV_ISA_EXT_f, guest_unsupp);
+    set_bit(RISCV_ISA_EXT_d, guest_unsupp);
+    set_bit(RISCV_ISA_EXT_q, guest_unsupp);
+    set_bit(RISCV_ISA_EXT_v, guest_unsupp);
+    set_bit(RISCV_ISA_EXT_h, guest_unsupp);
+    set_bit(RISCV_ISA_EXT_sstc, guest_unsupp);
+    set_bit(RISCV_ISA_EXT_svade, guest_unsupp);
+    set_bit(RISCV_ISA_EXT_svpbmt, guest_unsupp);
+}
+
 void __init riscv_fill_hwcap(void)
 {
     unsigned int i;
@@ -527,4 +582,6 @@ void __init riscv_fill_hwcap(void)
     if ( !all_extns_available )
         panic("Look why the extensions above are needed in "
               "https://xenbits.xenproject.org/docs/unstable/misc/riscv/booting.txt\n");
+
+    init_guest_unsupp();
 }
diff --git a/xen/arch/riscv/domain.c b/xen/arch/riscv/domain.c
index 669dd27d79a3..041aed176f32 100644
--- a/xen/arch/riscv/domain.c
+++ b/xen/arch/riscv/domain.c
@@ -303,6 +303,9 @@ int arch_domain_create(struct domain *d,
     if ( is_idle_domain(d) )
         return 0;
 
+    if ( (rc = init_guest_isa(d)) != 0 )
+        goto fail;
+
     if ( (rc = p2m_init(d, config)) != 0)
         goto fail;
 
diff --git a/xen/arch/riscv/include/asm/cpufeature.h b/xen/arch/riscv/include/asm/cpufeature.h
index 0c48d57a03bb..42d13d9dbdbe 100644
--- a/xen/arch/riscv/include/asm/cpufeature.h
+++ b/xen/arch/riscv/include/asm/cpufeature.h
@@ -17,6 +17,8 @@
  */
 #define RISCV_ISA_EXT_BASE  26
 
+#define RISCV_GUEST_ISA_STR_MAX 256
+
 enum riscv_isa_ext_id {
     RISCV_ISA_EXT_a,
     RISCV_ISA_EXT_c,
@@ -44,7 +46,10 @@ enum riscv_isa_ext_id {
     RISCV_ISA_EXT_MAX
 };
 
+struct domain;
+
 void riscv_fill_hwcap(void);
+int init_guest_isa(struct domain *d);
 
 bool riscv_isa_extension_available(const unsigned long *isa_bitmap,
                                    enum riscv_isa_ext_id id);
diff --git a/xen/arch/riscv/include/asm/domain.h b/xen/arch/riscv/include/asm/domain.h
index 6044ce0feee0..664b0b9f9129 100644
--- a/xen/arch/riscv/include/asm/domain.h
+++ b/xen/arch/riscv/include/asm/domain.h
@@ -7,6 +7,7 @@
 #include <xen/xmalloc.h>
 #include <public/hvm/params.h>
 
+#include <asm/cpufeature.h>
 #include <asm/guest-layout.h>
 #include <asm/p2m.h>
 #include <asm/vtimer.h>
@@ -94,6 +95,9 @@ struct arch_domain {
     struct p2m_domain p2m;
 
     struct paging_domain paging;
+
+    DECLARE_BITMAP(guest_isa, RISCV_ISA_EXT_MAX);
+    char guest_isa_str[RISCV_GUEST_ISA_STR_MAX];
 };
 
 #include <xen/sched.h>
-- 
2.54.0

[PATCH v2 06/26] xen/riscv: implement make_cpus_node()

[Oleksii Kurochko]

Implement make_cpus_node() to create cpus node for a guest domain.

This function is going to be use by common dom0less code during
construction domain.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - s/u32/uint32_t for timebase_frequency local variable.
 - Drop +1 from BUILD_BUG_ON().
 - return fdt_end_node(fdt); instead of res at the end of the function.
---
---
 xen/arch/riscv/domain-build.c | 106 ++++++++++++++++++++++++++++++++++
 1 file changed, 106 insertions(+)

diff --git a/xen/arch/riscv/domain-build.c b/xen/arch/riscv/domain-build.c
index 492327631e33..3063328d96e4 100644
--- a/xen/arch/riscv/domain-build.c
+++ b/xen/arch/riscv/domain-build.c
@@ -3,8 +3,10 @@
 #include <xen/fdt-domain-build.h>
 #include <xen/fdt-kernel.h>
 #include <xen/init.h>
+#include <xen/libfdt/libfdt.h>
 #include <xen/sched.h>
 
+#include <asm/cpufeature.h>
 #include <asm/current.h>
 #include <asm/guest_access.h>
 
@@ -50,3 +52,107 @@ int __init construct_domain(struct domain *d, struct kernel_info *kinfo)
 
     return 0;
 }
+int __init make_cpus_node(const struct domain *d, struct kernel_info *kinfo)
+{
+    int res;
+    const struct dt_device_node *cpus = dt_find_node_by_path("/cpus");
+    unsigned int cpu;
+    uint32_t timebase_frequency;
+    bool frequency_valid;
+    void *fdt = kinfo->fdt;
+
+    dt_dprintk("Create cpus node\n");
+
+    if ( !cpus )
+    {
+        dprintk(XENLOG_ERR, "Missing /cpus node in the device tree?\n");
+        return -ENOENT;
+    }
+
+    frequency_valid = dt_property_read_u32(cpus, "timebase-frequency",
+                                           &timebase_frequency);
+
+    res = fdt_begin_node(fdt, "cpus");
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#address-cells", 1);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#size-cells", 0);
+    if ( res )
+        return res;
+
+    if ( frequency_valid )
+        res = fdt_property_cell(fdt, "timebase-frequency", timebase_frequency);
+
+    for ( cpu = 0; cpu < d->max_vcpus; cpu++ )
+    {
+        char buf[64];
+        uint32_t reg = cpu_to_fdt32(cpu);
+
+        snprintf(buf, sizeof(buf), "cpu@%u", cpu);
+        res = fdt_begin_node(fdt, buf);
+        if ( res )
+            return res;
+
+        res = fdt_property(fdt, "reg", &reg, sizeof(reg));
+        if ( res )
+            return res;
+
+        res = fdt_property_string(fdt, "status", "okay");
+        if ( res )
+            return res;
+
+        res = fdt_property_string(fdt, "compatible", "riscv");
+        if ( res )
+            return res;
+
+        BUILD_BUG_ON((sizeof("riscv,") +
+                      sizeof_field(struct gstage_mode_desc, name)) >= sizeof(buf));
+        snprintf(buf, sizeof(buf), "riscv,%s", max_gstage_mode->name);
+        res = fdt_property_string(fdt, "mmu-type", buf);
+        if ( res )
+            return res;
+
+        res = fdt_property_string(fdt, "riscv,isa", d->arch.guest_isa_str);
+        if ( res )
+            return res;
+
+        res = fdt_property_string(fdt, "device_type", "cpu");
+        if ( res )
+            return res;
+
+        res = fdt_begin_node(fdt, "interrupt-controller");
+        if ( res )
+            return res;
+
+        res = fdt_property_string(fdt, "compatible", "riscv,cpu-intc");
+        if ( res )
+            return res;
+
+        res = fdt_property_cell(fdt, "#interrupt-cells", 1);
+        if ( res )
+            return res;
+
+        res = fdt_property(fdt, "interrupt-controller", NULL, 0);
+        if ( res )
+            return res;
+
+        res = fdt_property_u32(fdt, "phandle", alloc_phandle(kinfo));
+        if ( res )
+            return res;
+
+        /* end of interrupt-controller */
+        res = fdt_end_node(fdt);
+        if ( res )
+            return res;
+
+        res = fdt_end_node(fdt);
+        if ( res )
+            return res;
+    }
+
+    return fdt_end_node(fdt);
+}
-- 
2.54.0

[PATCH v2 07/26] xen/riscv: implement make_timer_node()

[Oleksii Kurochko]

Generally, in DT for RISC-V there is a document which describes a timer
node (riscv,timer.yaml or sifive,clint.yaml), but the Linux timer driver
is declared with TIMER_OF_DECLARE(riscv_timer, "riscv", ...).
It matches the CPU node (compatible "riscv"), not the timer node itself.
It then calls of_find_compatible_node(NULL, NULL, "riscv,timer") only to
read the optional riscv,timer-cannot-wake-cpu property.

Since Xen does not care about that property for now, make_timer_node() is
implemented to return 0, as no timer node needs to be created for RISC-V
guests.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in v2:
 - Acked-by: Jan Beulich <jbeulich@suse.com>
 - Update the commit message.
---
---
 xen/arch/riscv/domain-build.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/xen/arch/riscv/domain-build.c b/xen/arch/riscv/domain-build.c
index 3063328d96e4..02b293239142 100644
--- a/xen/arch/riscv/domain-build.c
+++ b/xen/arch/riscv/domain-build.c
@@ -3,6 +3,7 @@
 #include <xen/fdt-domain-build.h>
 #include <xen/fdt-kernel.h>
 #include <xen/init.h>
+#include <xen/fdt-kernel.h>
 #include <xen/libfdt/libfdt.h>
 #include <xen/sched.h>
 
@@ -156,3 +157,10 @@ int __init make_cpus_node(const struct domain *d, struct kernel_info *kinfo)
 
     return fdt_end_node(fdt);
 }
+
+int __init make_timer_node(const struct kernel_info *kinfo)
+{
+    /* There is no need for timer node for RISC-V. */
+
+    return 0;
+}
-- 
2.54.0

[PATCH v2 08/26] xen/riscv: implement make_arch_nodes()

[Oleksii Kurochko]

No RISC-V-specific nodes need to be created at the moment,
so make_arch_nodes() is implemented to simply return 0.

It is placed in dom0less-build.c as make_arch_nodes() is
only used in the dom0less code path. In the future, it will
be extended to create an emulated UART node.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - Update the commit message.
---
---
 xen/arch/riscv/dom0less-build.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/xen/arch/riscv/dom0less-build.c b/xen/arch/riscv/dom0less-build.c
index a683972e9235..4cc00012aa8d 100644
--- a/xen/arch/riscv/dom0less-build.c
+++ b/xen/arch/riscv/dom0less-build.c
@@ -2,10 +2,18 @@
 
 #include <xen/bootfdt.h>
 #include <xen/device_tree.h>
+#include <xen/fdt-kernel.h>
 #include <xen/init.h>
 
 #include <asm/p2m.h>
 
+int __init make_arch_nodes(struct kernel_info *kinfo)
+{
+    /* No RISC-V specific nodes need to be made, at the moment. */
+
+    return 0;
+}
+
 int __init arch_parse_dom0less_node(struct dt_device_node *node,
                                     struct boot_domain *bd)
 {
-- 
2.54.0

[PATCH v2 09/26] xen/riscv: introduce init interrupt controller operations

[Oleksii Kurochko]

Introduce intc_hw_init_ops structure to avoid risky mix of init
function and non-init function.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - New patch.
---
 xen/arch/riscv/aplic.c            |  7 +++++--
 xen/arch/riscv/include/asm/intc.h | 10 +++++++---
 xen/arch/riscv/intc.c             | 10 ++++++++--
 3 files changed, 20 insertions(+), 7 deletions(-)

diff --git a/xen/arch/riscv/aplic.c b/xen/arch/riscv/aplic.c
index 739e8dab3498..97dc0ef731f0 100644
--- a/xen/arch/riscv/aplic.c
+++ b/xen/arch/riscv/aplic.c
@@ -306,12 +306,15 @@ static const hw_irq_controller aplic_xen_irq_type = {
 
 static const struct intc_hw_operations aplic_ops = {
     .info                = &aplic_info,
-    .init                = aplic_init,
     .host_irq_type       = &aplic_xen_irq_type,
     .handle_interrupt    = aplic_handle_interrupt,
     .set_irq_type        = aplic_set_irq_type,
 };
 
+static const struct intc_hw_init_ops __initdata aplic_init_ops = {
+    .init                = aplic_init,
+};
+
 static int cf_check aplic_irq_xlate(const uint32_t *intspec,
                                     unsigned int intsize,
                                     unsigned int *out_hwirq,
@@ -347,7 +350,7 @@ static int __init aplic_preinit(struct dt_device_node *node, const void *dat)
 
     dt_irq_xlate = aplic_irq_xlate;
 
-    register_intc_ops(&aplic_ops);
+    register_intc_ops(&aplic_ops, &aplic_init_ops);
 
     /* Enable supervisor external interrupt */
     csr_set(CSR_SIE, BIT(IRQ_S_EXT, UL));
diff --git a/xen/arch/riscv/include/asm/intc.h b/xen/arch/riscv/include/asm/intc.h
index ecdc8a5e6577..8b498e43b33f 100644
--- a/xen/arch/riscv/include/asm/intc.h
+++ b/xen/arch/riscv/include/asm/intc.h
@@ -28,8 +28,6 @@ struct intc_info {
 struct intc_hw_operations {
     /* Hold intc hw information */
     const struct intc_info *info;
-    /* Initialize the intc and the boot CPU */
-    int (*init)(void);
 
     /* hw_irq_controller to enable/disable/eoi host irq */
     const struct hw_interrupt_type *host_irq_type;
@@ -43,9 +41,15 @@ struct intc_hw_operations {
     void (*handle_interrupt)(struct cpu_user_regs *regs);
 };
 
+struct intc_hw_init_ops {
+    /* Initialize the intc and the boot CPU */
+    int (*init)(void);
+};
+
 void intc_preinit(void);
 
-void register_intc_ops(const struct intc_hw_operations *ops);
+void register_intc_ops(const struct intc_hw_operations *ops,
+                       const struct intc_hw_init_ops *init_ops);
 
 void intc_init(void);
 
diff --git a/xen/arch/riscv/intc.c b/xen/arch/riscv/intc.c
index ea317aea5ad8..8649160403f7 100644
--- a/xen/arch/riscv/intc.c
+++ b/xen/arch/riscv/intc.c
@@ -12,9 +12,13 @@
 
 static const struct intc_hw_operations *__ro_after_init intc_hw_ops;
 
-void __init register_intc_ops(const struct intc_hw_operations *ops)
+static const struct intc_hw_init_ops *__initdata intc_hw_init_ops;
+
+void __init register_intc_ops(const struct intc_hw_operations *ops,
+                              const struct intc_hw_init_ops *init_ops)
 {
     intc_hw_ops = ops;
+    intc_hw_init_ops = init_ops;
 }
 
 void __init intc_preinit(void)
@@ -27,7 +31,9 @@ void __init intc_preinit(void)
 
 void __init intc_init(void)
 {
-    if ( intc_hw_ops->init() )
+    ASSERT(intc_hw_init_ops && intc_hw_init_ops->init);
+
+    if ( intc_hw_init_ops->init() )
         panic("Failed to initialize the interrupt controller drivers\n");
 }
 
-- 
2.54.0

[PATCH v2 10/26] xen/riscv: implement make_intc_domU_node()

[Oleksii Kurochko]

Introduce a RISC-V specific function to create an interrupt controller
Device Tree node for DomU domains during dom0less build.

Add make_intc_domU_node() to the dom0less build path and wire it to
a new generic helper, intc_make_domu_dt_node(), which delegates DT
node creation to the active interrupt controller implementation via
vintc_init_ops.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - s/intc_make_domu_dt_node/make_intc_domU_node.
 - introduce separate intc_hw_init_ops structure for init operations.
 - Return -EOPNOTSUPP instead of -ENOSYS.
 - Drop const for kinfo argument as it could be changed by interrupt
   controller node creation code.
 - Refactor make_domu_dt_node().
 - Make make_domu_dt_node part of vintc structure as it looks more logical to be
   there.
---
 xen/arch/riscv/include/asm/domain.h |  2 ++
 xen/arch/riscv/include/asm/intc.h   | 12 ++++++++++--
 xen/arch/riscv/intc.c               | 11 +++++++++++
 3 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/xen/arch/riscv/include/asm/domain.h b/xen/arch/riscv/include/asm/domain.h
index 664b0b9f9129..136d9e816a44 100644
--- a/xen/arch/riscv/include/asm/domain.h
+++ b/xen/arch/riscv/include/asm/domain.h
@@ -98,6 +98,8 @@ struct arch_domain {
 
     DECLARE_BITMAP(guest_isa, RISCV_ISA_EXT_MAX);
     char guest_isa_str[RISCV_GUEST_ISA_STR_MAX];
+
+    struct vintc *vintc;
 };
 
 #include <xen/sched.h>
diff --git a/xen/arch/riscv/include/asm/intc.h b/xen/arch/riscv/include/asm/intc.h
index 8b498e43b33f..70df461a2a51 100644
--- a/xen/arch/riscv/include/asm/intc.h
+++ b/xen/arch/riscv/include/asm/intc.h
@@ -8,14 +8,13 @@
 #ifndef ASM__RISCV__INTERRUPT_CONTOLLER_H
 #define ASM__RISCV__INTERRUPT_CONTOLLER_H
 
-struct dt_device_node;
-
 enum intc_version {
     INTC_APLIC,
 };
 
 struct cpu_user_regs;
 struct irq_desc;
+struct kernel_info;
 
 struct intc_info {
     enum intc_version hw_version;
@@ -46,6 +45,15 @@ struct intc_hw_init_ops {
     int (*init)(void);
 };
 
+struct vintc_init_ops {
+    /* Create interrupt controller node for domain */
+    int (*make_domu_dt_node)(struct kernel_info *kinfo);
+};
+
+struct vintc {
+    struct vintc_init_ops *init_ops;
+};
+
 void intc_preinit(void);
 
 void register_intc_ops(const struct intc_hw_operations *ops,
diff --git a/xen/arch/riscv/intc.c b/xen/arch/riscv/intc.c
index 8649160403f7..0994deddcb2c 100644
--- a/xen/arch/riscv/intc.c
+++ b/xen/arch/riscv/intc.c
@@ -3,6 +3,7 @@
 #include <xen/acpi.h>
 #include <xen/bug.h>
 #include <xen/device_tree.h>
+#include <xen/fdt-kernel.h>
 #include <xen/init.h>
 #include <xen/irq.h>
 #include <xen/lib.h>
@@ -73,3 +74,13 @@ void intc_route_irq_to_xen(struct irq_desc *desc, unsigned int priority)
     intc_set_irq_type(desc, desc->arch.type);
     intc_set_irq_priority(desc, priority);
 }
+
+int __init make_intc_domU_node(struct kernel_info *kinfo)
+{
+    struct vintc *vintc = kinfo->bd.d->arch.vintc;
+
+    if ( intc_hw_ops && vintc->init_ops && vintc->init_ops->make_domu_dt_node )
+        return vintc->init_ops->make_domu_dt_node(kinfo);
+
+    return -EOPNOTSUPP;
+}
-- 
2.54.0

[PATCH v2 11/26] xen/riscv: introduce aia_init() and aia_usable()

[Oleksii Kurochko]

aia_init() is going to contain all the logic related to AIA initialization.

At the moment, it only checks whether the SSAIA extension is available,
and if so, sets is_aia_usable (which  indicates more than just the
availability of the extension) to true; it also signifies that the necessary
components (to be introduced in follow-up patches) have been initialized.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - s/is_aia_available/is_aia_usable.
 - Drop return value for aia_init().
 - s/aia_available()/aia_usable().
---
 xen/arch/riscv/Makefile          |  1 +
 xen/arch/riscv/aia.c             | 23 +++++++++++++++++++++++
 xen/arch/riscv/include/asm/aia.h | 10 ++++++++++
 xen/arch/riscv/intc.c            |  3 +++
 4 files changed, 37 insertions(+)
 create mode 100644 xen/arch/riscv/aia.c
 create mode 100644 xen/arch/riscv/include/asm/aia.h

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index fd5e499eb4b9..9df8b72b5494 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -1,3 +1,4 @@
+obj-y += aia.o
 obj-y += aplic.o
 obj-y += cpufeature.o
 obj-y += domain.o
diff --git a/xen/arch/riscv/aia.c b/xen/arch/riscv/aia.c
new file mode 100644
index 000000000000..f67f422c5a45
--- /dev/null
+++ b/xen/arch/riscv/aia.c
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#include <xen/errno.h>
+#include <xen/init.h>
+#include <xen/sections.h>
+#include <xen/types.h>
+
+#include <asm/cpufeature.h>
+
+static bool __ro_after_init is_aia_usable;
+
+bool aia_usable(void)
+{
+    return is_aia_usable;
+}
+
+void __init aia_init(void)
+{
+    if ( !riscv_isa_extension_available(NULL, RISCV_ISA_EXT_ssaia) )
+        return;
+
+    is_aia_usable = true;
+}
diff --git a/xen/arch/riscv/include/asm/aia.h b/xen/arch/riscv/include/asm/aia.h
new file mode 100644
index 000000000000..ca42c3086126
--- /dev/null
+++ b/xen/arch/riscv/include/asm/aia.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#ifndef ASM__RISCV__AIA_H
+#define ASM__RISCV__AIA_H
+
+bool aia_usable(void);
+
+void aia_init(void);
+
+#endif /* ASM__RISCV__ACPI_H */
diff --git a/xen/arch/riscv/intc.c b/xen/arch/riscv/intc.c
index 0994deddcb2c..a611c9dc5f32 100644
--- a/xen/arch/riscv/intc.c
+++ b/xen/arch/riscv/intc.c
@@ -9,6 +9,7 @@
 #include <xen/lib.h>
 #include <xen/spinlock.h>
 
+#include <asm/aia.h>
 #include <asm/intc.h>
 
 static const struct intc_hw_operations *__ro_after_init intc_hw_ops;
@@ -34,6 +35,8 @@ void __init intc_init(void)
 {
     ASSERT(intc_hw_init_ops && intc_hw_init_ops->init);
 
+    aia_init();
+
     if ( intc_hw_init_ops->init() )
         panic("Failed to initialize the interrupt controller drivers\n");
 }
-- 
2.54.0

[PATCH v2 12/26] xen/riscv: add basic VGEIN management for AIA guests

[Oleksii Kurochko]

It was decided to add support for IMSIC from the start instead of having APLIC
operate in direct delivery mode, as it requires a trap-and-emulation approach,
which is not optimal from a performance standpoint.

AIA provides a hardware-accelerated mechanism for delivering external
interrupts to domains via "guest interrupt files" located in IMSIC.
A single physical hart can implement multiple such files (up to GEILEN),
allowing several virtual harts to receive interrupts directly from hardware.

Introduce per-CPU tracking of guest interrupt file identifiers (VGEIN)
for systems implementing AIA specification. Each CPU maintains
a bitmap describing which guest interrupt files are currently in use.

Add helpers to initialize the bitmap based on the number of available
guest interrupt files (GEILEN), assign a VGEIN to a vCPU, and release it
when no longer needed. When assigning a VGEIN, the corresponding value
is written to the VGEIN field of the guest hstatus register so that
VS-level external interrupts are delivered from the selected interrupt
file.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - add static for defintion of vgein_bmp;
 - Drop declarartion of vgein_bmp from aia.h.
 - Move declaration of 'struct vgein_bmp' from aia.h to aia.c as all the
   management is inside aia.c.
 - Instead of decrement of vgein->geilen just update the wait how it is
   initialized.
 - Return -EOPNOTSUPP in vgein_init() instead of BUG_ON().
 - Use %u to print unsigned int.
 - make bmp field in vgein_bmp not a pointer.
 - allocate owners dynamically.
 - Drop unnessary blank lines.
 - use find_first_zero_bit() instead of bitmap_weight() to find a free slot
   for vgein number.
 - Drop the section number for the comment.
 - Start to search from bitnum 1 for free vgein_id, as bitnum 0 is reserved to
   tell that no guest extrenal interrupt number is used. Thereby drop vgein_id++
   at the end of vgein_assign().
 - s/bitmap_set/__set_bit.
 - s/bitmap_clear/__clear_bit.
 - as vgein_init() is needed to be invoked once per CPU being brought up, drop
   __init for it.
 - Return vgein_id == 0 if vgein_id is higher then maximun supported by h/w
   VGEIN.
 - Add check in vgein_relase() that vgein is 0 and if it is there is nothing
   is needed to do.
 - Use gdprintk instead of printk() in vgein_{assign,release}.
 - Add the claryfing comment above geilen field.
 - Drop ASSERT in vgein_assign() and return just vgein_id = 0 in the case when
   there is no aviablable h/w VGEINs.
 - Make vgein_init() static.
---
 xen/arch/riscv/aia.c             | 144 +++++++++++++++++++++++++++++++
 xen/arch/riscv/include/asm/aia.h |   8 ++
 2 files changed, 152 insertions(+)

diff --git a/xen/arch/riscv/aia.c b/xen/arch/riscv/aia.c
index f67f422c5a45..f7f44961e0f5 100644
--- a/xen/arch/riscv/aia.c
+++ b/xen/arch/riscv/aia.c
@@ -1,11 +1,33 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 
+#include <xen/bitmap.h>
+#include <xen/cpu.h>
 #include <xen/errno.h>
 #include <xen/init.h>
 #include <xen/sections.h>
+#include <xen/sched.h>
+#include <xen/spinlock.h>
 #include <xen/types.h>
+#include <xen/xvmalloc.h>
 
+#include <asm/aia.h>
 #include <asm/cpufeature.h>
+#include <asm/csr.h>
+#include <asm/current.h>
+
+struct vgein_ctrl {
+    unsigned long bmp;
+    spinlock_t lock;
+    struct vcpu **owners;
+    /* The least-significant bits are implemented first, apart from bit 0 */
+    unsigned int geilen;
+};
+
+/*
+ * Bitmap for each physical cpus to detect which VS (guest)
+ * interrupt file id was used.
+ */
+static DEFINE_PER_CPU(struct vgein_ctrl, vgein);
 
 static bool __ro_after_init is_aia_usable;
 
@@ -14,10 +36,132 @@ bool aia_usable(void)
     return is_aia_usable;
 }
 
+static int vgein_init(unsigned int cpu)
+{
+    struct vgein_ctrl *vgein = &per_cpu(vgein, cpu);
+
+    csr_write(CSR_HGEIE, -1UL);
+    vgein->geilen = flsl(csr_read(CSR_HGEIE) >> 1);
+    csr_write(CSR_HGEIE, 0);
+
+    printk("cpu%d.geilen=%u\n", cpu, vgein->geilen);
+
+    if ( !vgein->geilen )
+        return -EOPNOTSUPP;
+
+    vgein->owners = xvzalloc_array(struct vcpu *, vgein->geilen);
+    if ( !vgein->owners )
+        return -ENOMEM;
+
+    spin_lock_init(&vgein->lock);
+
+    return 0;
+}
+
+static int cf_check cpu_callback(struct notifier_block *nfb, unsigned long action,
+                        void *hcpu)
+{
+    unsigned int cpu = (unsigned long)hcpu;
+    int rc = 0;
+
+    switch ( action )
+    {
+    case CPU_STARTING:
+        rc = vgein_init(cpu);
+        if ( rc )
+            printk("AIA: failed to init vgein for CPU%un", cpu);
+        break;
+    }
+
+    return notifier_from_errno(rc);
+}
+
+static struct notifier_block cpu_nfb = {
+    .notifier_call = cpu_callback,
+};
+
 void __init aia_init(void)
 {
+    int rc;
+
     if ( !riscv_isa_extension_available(NULL, RISCV_ISA_EXT_ssaia) )
+    {
+        dprintk(XENLOG_WARNING, "SSAIA isn't present in riscv,isa\n");
+        return;
+    }
+
+    if ( (rc = vgein_init(0)) )
+    {
+        dprintk(XENLOG_ERR, "vgein_init() failed with rc(%d)\n", rc);
         return;
+    }
 
     is_aia_usable = true;
+
+    register_cpu_notifier(&cpu_nfb);
+}
+
+unsigned int vgein_assign(struct vcpu *v)
+{
+    unsigned int vgein_id;
+    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
+    unsigned long *bmp = &vgein->bmp;
+    unsigned long flags;
+
+    spin_lock_irqsave(&vgein->lock, flags);
+    /*
+     * The vgein_id shouldn't be zero, as it will indicate that no guest
+     * external interrupt source is selected for VS-level external interrupts
+     * according to RISC-V priviliged spec:
+     *   Hypervisor Status Register (hstatus) in RISC-V priviliged spec:
+     *
+     *   The VGEIN (Virtual Guest External Interrupt Number) field selects
+     *   a guest external interrupt source for VS-level external interrupts.
+     *   VGEIN is a WLRL field that must be able to hold values between zero
+     *   and the maximum guest external interrupt number (known as GEILEN),
+     *   inclusive.
+     *   When VGEIN=0, no guest external interrupt source is selected for
+     *   VS-level external interrupts.
+     *
+     * So start to search from bit number 1.
+     */
+    vgein_id = find_next_zero_bit(bmp, vgein->geilen + 1, 1);
+
+    if ( vgein_id > vgein->geilen )
+        vgein_id = 0;
+    else
+        __set_bit(vgein_id, bmp);
+
+    spin_unlock_irqrestore(&vgein->lock, flags);
+
+#ifdef VGEIN_DEBUG
+    gprintk(XENLOG_DEBUG, "%s: %pv: vgein_id(%u), xen_cpu%d_bmp=%#lx\n",
+           __func__, v, vgein_id, v->processor, *bmp);
+#endif
+
+    vcpu_guest_cpu_user_regs(v)->hstatus &= ~HSTATUS_VGEIN;
+    vcpu_guest_cpu_user_regs(v)->hstatus |=
+        MASK_INSR(vgein_id, HSTATUS_VGEIN);
+
+    return vgein_id;
+}
+
+void vgein_release(struct vcpu *v, unsigned int vgen_id)
+{
+    unsigned long flags;
+    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
+
+    if ( !vgen_id )
+        return;
+
+    spin_lock_irqsave(&vgein->lock, flags);
+     __clear_bit(vgen_id, &vgein->bmp);
+    spin_unlock_irqrestore(&vgein->lock, flags);
+
+#ifdef VGEIN_DEBUG
+    gprintk(XENLOG_DEBUG, "%s: vgein_id(%u), xen_cpu%d_bmp=%#lx\n",
+           __func__, vgen_id, v->processor, vgein->bmp);
+#endif
+
+    vcpu_guest_cpu_user_regs(v)->hstatus &= ~HSTATUS_VGEIN;
 }
diff --git a/xen/arch/riscv/include/asm/aia.h b/xen/arch/riscv/include/asm/aia.h
index ca42c3086126..6073c89774bb 100644
--- a/xen/arch/riscv/include/asm/aia.h
+++ b/xen/arch/riscv/include/asm/aia.h
@@ -3,8 +3,16 @@
 #ifndef ASM__RISCV__AIA_H
 #define ASM__RISCV__AIA_H
 
+#include <xen/percpu.h>
+#include <xen/spinlock.h>
+
+struct vcpu;
+
 bool aia_usable(void);
 
 void aia_init(void);
 
+unsigned int vgein_assign(struct vcpu *v);
+void vgein_release(struct vcpu *v, unsigned int vgen_id);
+
 #endif /* ASM__RISCV__ACPI_H */
-- 
2.54.0

[PATCH v2 13/26] xen/riscv: introduce per-vCPU IMSIC state

[Oleksii Kurochko]

Each vCPU interacting with the IMSIC requires state to track the
associated guest interrupt file and its backing context.

Introduce a per-vCPU structure to hold IMSIC-related state, including
the guest interrupt file identifier and the CPU providing the backing
VS-file. Access to the guest file identifier is protected by a lock.

Initialize this structure during vCPU setup and store it in arch_vcpu.
The initial state marks the VS-file as software-backed until it becomes
associated with a physical CPU.

Add helpers to retrieve and update the guest interrupt file identifier:
- imsic_set_guest_file_id() is going to be used during vCPU creation to
  initialize IMSIC interrupt controller things for this vCPU.
- vcpu_guest_file_id() is going to be used during update of APLIC's
  target register with the pair of information <guest_file_id, cpu_id>
  (to have MSI delivery mode work properly) when guest is trying to
  access vAPLIC's target register.
It will be used in the follow up patches.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - Rename imsic_state to vimsic_state.
 - Use 'unsigned int' for vsfile_pcpu.
 - Drop initialzation of ->guest_file_id as it will be by default zero.
 - Add the comment about ->guest_file_id field.
 - Drop __init for vcpu_imsic_init() as it could be used during post-boot
   vCPU creation.
 - Update the commit message.
 - Drop locks around ->guest_file_id() in  vcpu_guest_file_id() and imsic_set_guest_file_id().
---
 xen/arch/riscv/imsic.c              | 35 +++++++++++++++++++++++++++++
 xen/arch/riscv/include/asm/domain.h |  2 ++
 xen/arch/riscv/include/asm/imsic.h  | 23 +++++++++++++++++++
 3 files changed, 60 insertions(+)

diff --git a/xen/arch/riscv/imsic.c b/xen/arch/riscv/imsic.c
index a4460576f620..8b46419ca23b 100644
--- a/xen/arch/riscv/imsic.c
+++ b/xen/arch/riscv/imsic.c
@@ -16,6 +16,7 @@
 #include <xen/errno.h>
 #include <xen/init.h>
 #include <xen/macros.h>
+#include <xen/sched.h>
 #include <xen/smp.h>
 #include <xen/spinlock.h>
 #include <xen/xvmalloc.h>
@@ -56,6 +57,16 @@ do {                            \
     csr_clear(CSR_SIREG, v);    \
 } while (0)
 
+unsigned int vcpu_guest_file_id(const struct vcpu *v)
+{
+    return ACCESS_ONCE(v->arch.vimsic_state->guest_file_id);
+}
+
+void imsic_set_guest_file_id(const struct vcpu *v, unsigned int guest_file_id)
+{
+    ACCESS_ONCE(v->arch.vimsic_state->guest_file_id) = guest_file_id;
+}
+
 void __init imsic_ids_local_delivery(bool enable)
 {
     if ( enable )
@@ -312,6 +323,30 @@ static int imsic_parse_node(const struct dt_device_node *node,
     return 0;
 }
 
+int vcpu_imsic_init(struct vcpu *v)
+{
+    struct vimsic_state *imsic_state;
+
+    /* Allocate IMSIC context */
+    imsic_state = xvzalloc(struct vimsic_state);
+    if ( !imsic_state )
+        return -ENOMEM;
+
+    v->arch.vimsic_state = imsic_state;
+
+    /* Setup IMSIC context  */
+    rwlock_init(&imsic_state->vsfile_lock);
+
+    imsic_state->vsfile_pcpu = NR_CPUS;
+
+    return 0;
+}
+
+void vcpu_imsic_deinit(const struct vcpu *v)
+{
+    xvfree(v->arch.vimsic_state);
+}
+
 /*
  * Initialize the imsic_cfg structure based on the IMSIC DT node.
  *
diff --git a/xen/arch/riscv/include/asm/domain.h b/xen/arch/riscv/include/asm/domain.h
index 136d9e816a44..42c1db91bd86 100644
--- a/xen/arch/riscv/include/asm/domain.h
+++ b/xen/arch/riscv/include/asm/domain.h
@@ -54,6 +54,8 @@ struct arch_vcpu {
 
     struct vtimer vtimer;
 
+    struct vimsic_state *vimsic_state;
+
     register_t hcounteren;
     register_t hedeleg;
     register_t hideleg;
diff --git a/xen/arch/riscv/include/asm/imsic.h b/xen/arch/riscv/include/asm/imsic.h
index c6c59215df20..2b84824cd377 100644
--- a/xen/arch/riscv/include/asm/imsic.h
+++ b/xen/arch/riscv/include/asm/imsic.h
@@ -11,6 +11,7 @@
 #ifndef ASM_RISCV_IMSIC_H
 #define ASM_RISCV_IMSIC_H
 
+#include <xen/rwlock.h>
 #include <xen/spinlock.h>
 #include <xen/stdbool.h>
 #include <xen/types.h>
@@ -61,7 +62,24 @@ struct imsic_config {
     spinlock_t lock;
 };
 
+struct vimsic_state {
+    /* IMSIC VS-file */
+    rwlock_t vsfile_lock;
+    /*
+     * (guest_file_id == 0) -> s/w IMSIC SW-file
+     * (guest_file_id > 0) -> h/w IMSIC VS-file
+     */
+    unsigned int guest_file_id;
+    /*
+     * (vsfile_pcpu >= 0) => h/w IMSIC VS-file
+     * (vsfile_pcpu == NR_CPUS) => s/w IMSIC SW-file
+     */
+    unsigned int vsfile_pcpu;
+};
+
 struct dt_device_node;
+struct vcpu;
+
 int imsic_init(const struct dt_device_node *node);
 
 const struct imsic_config *imsic_get_config(void);
@@ -71,4 +89,9 @@ void imsic_irq_disable(unsigned int hwirq);
 
 void imsic_ids_local_delivery(bool enable);
 
+int vcpu_imsic_init(struct vcpu *v);
+void vcpu_imsic_deinit(const struct vcpu *v);
+unsigned int vcpu_guest_file_id(const struct vcpu *v);
+void imsic_set_guest_file_id(const struct vcpu *v, unsigned int guest_file_id);
+
 #endif /* ASM_RISCV_IMSIC_H */
-- 
2.54.0

[PATCH v2 14/26] xen/riscv: add very early virtual APLIC (vAPLIC) initialization support

[Oleksii Kurochko]

At the current development stage, only domain vINTC init and deinit
operations are required, so implement those first.

Initialize vAPLIC's domaincfg to with the interrupt-enable bit set and
MSI delivery mode selected as the current solution is exepcted to have
always IMSIC, and initialize vintc->ops.

Other operations such as emulate_load(), emulate_store(), and is_access()
will be needed once guests are running and MMIO accesses to APLIC MMIO
range must be handled. These will be introduced separately later.

Introduce a structure to describe a virtual interrupt controller (vINTC)
and a vintc_ops structure, which provides operations to emulate load and
store accesses to interrupt controller MMIOs and to check whether a given
address falls within the MMIO range of a specific virtual interrupt
controller.

The vAPLIC implementation of these operations will be provided later
once guests can be run and these operations are actually needed.

Introduce these structures here as they are required for the implementation
of domain_vaplic_init() and domain_vaplic_alloc(). Also, introduce
vcpu_vaplic_init() and init vintc_ops->vcpu_init() with it.

Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - s/vcpu/v for function arguments in struct vintc_ops().
 - Update the comment above is_access() and drop const for addr argument.
 - Update to_vaplic() to work with 'struct domain *'.
 - Drop smsiaddrcfg{h} from vaplic_regs struct as they aren't used for now.
 - Drop inclusion of xen/schec.h from intc.c.
 - use result of xvzalloc() as initializer in vpalic_alloc().
 - Drop goto in domain_vaplic_init().
 - s/XVFREE/xvfree.
 - s/aplic/vintc.
 - Drop __init for vcpu_vaplic_init() as it could be called for secondary CPU bring up.
 - Drop vaplic_alloc().
 - Drop vintc_ops struct, embed callbacks iniside struct vintc.
 - Introduce and init vintc irqs for vAPLIC.
 - Introduce intc_irq_nums() to properly initialize number of vAPLIC's irqs.
---
 xen/arch/riscv/Makefile             |  1 +
 xen/arch/riscv/aplic.c              |  6 +++
 xen/arch/riscv/domain.c             | 13 ++---
 xen/arch/riscv/include/asm/intc.h   | 17 ++++++-
 xen/arch/riscv/include/asm/vaplic.h | 34 +++++++++++++
 xen/arch/riscv/intc.c               |  7 +++
 xen/arch/riscv/vaplic.c             | 78 +++++++++++++++++++++++++++++
 7 files changed, 147 insertions(+), 9 deletions(-)
 create mode 100644 xen/arch/riscv/include/asm/vaplic.h
 create mode 100644 xen/arch/riscv/vaplic.c

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 9df8b72b5494..9d8d21b65188 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -25,6 +25,7 @@ obj-y += smpboot.o
 obj-y += stubs.o
 obj-y += time.o
 obj-y += traps.o
+obj-y += vaplic.o
 obj-y += vmid.o
 obj-y += vm_event.o
 obj-y += vsbi/
diff --git a/xen/arch/riscv/aplic.c b/xen/arch/riscv/aplic.c
index 97dc0ef731f0..aba9f3945236 100644
--- a/xen/arch/riscv/aplic.c
+++ b/xen/arch/riscv/aplic.c
@@ -295,6 +295,11 @@ static void cf_check aplic_set_irq_type(struct irq_desc *desc,
     spin_unlock(&aplic.lock);
 }
 
+static unsigned int cf_check aplic_irq_num(void)
+{
+    return aplic_info.num_irqs;
+}
+
 static const hw_irq_controller aplic_xen_irq_type = {
     .typename     = "aplic",
     .startup      = aplic_irq_startup,
@@ -309,6 +314,7 @@ static const struct intc_hw_operations aplic_ops = {
     .host_irq_type       = &aplic_xen_irq_type,
     .handle_interrupt    = aplic_handle_interrupt,
     .set_irq_type        = aplic_set_irq_type,
+    .irq_nums            = aplic_irq_num,
 };
 
 static const struct intc_hw_init_ops __initdata aplic_init_ops = {
diff --git a/xen/arch/riscv/domain.c b/xen/arch/riscv/domain.c
index 041aed176f32..7aa00d6cfd95 100644
--- a/xen/arch/riscv/domain.c
+++ b/xen/arch/riscv/domain.c
@@ -11,6 +11,7 @@
 #include <asm/bitops.h>
 #include <asm/cpufeature.h>
 #include <asm/csr.h>
+#include <asm/intc.h>
 #include <asm/riscv_encoding.h>
 #include <asm/vtimer.h>
 
@@ -155,14 +156,10 @@ int arch_vcpu_create(struct vcpu *v)
     if ( (rc = vcpu_vtimer_init(v)) )
         goto fail;
 
-    /*
-     * As interrupt controller (IC) is not yet implemented,
-     * return an error.
-     *
-     * TODO: Drop this once IC is implemented.
-     */
-    rc = -EOPNOTSUPP;
-    goto fail;
+    ASSERT(v->domain->arch.vintc->ops->vcpu_init);
+
+    if ( (rc = v->domain->arch.vintc->ops->vcpu_init(v)) )
+        goto fail;
 
     return rc;
 
diff --git a/xen/arch/riscv/include/asm/intc.h b/xen/arch/riscv/include/asm/intc.h
index 70df461a2a51..3b30d000066a 100644
--- a/xen/arch/riscv/include/asm/intc.h
+++ b/xen/arch/riscv/include/asm/intc.h
@@ -15,6 +15,7 @@ enum intc_version {
 struct cpu_user_regs;
 struct irq_desc;
 struct kernel_info;
+struct vcpu;
 
 struct intc_info {
     enum intc_version hw_version;
@@ -38,6 +39,8 @@ struct intc_hw_operations {
 
     /* handle external interrupt */
     void (*handle_interrupt)(struct cpu_user_regs *regs);
+
+    unsigned int (*irq_nums)(void);
 };
 
 struct intc_hw_init_ops {
@@ -50,8 +53,18 @@ struct vintc_init_ops {
     int (*make_domu_dt_node)(struct kernel_info *kinfo);
 };
 
+struct vintc_ops {
+    /* Initialize some vINTC-related stuff for a vCPU */
+    int (*vcpu_init)(struct vcpu *v);
+
+    /* Check if a address is virtual interrupt controller MMIO */
+    int (*is_access)(const struct vcpu *v, unsigned long addr);
+};
+
 struct vintc {
-    struct vintc_init_ops *init_ops;
+    unsigned int irq_nums;
+    const struct vintc_init_ops *init_ops;
+    const struct vintc_ops *ops;
 };
 
 void intc_preinit(void);
@@ -65,4 +78,6 @@ void intc_route_irq_to_xen(struct irq_desc *desc, unsigned int priority);
 
 void intc_handle_external_irqs(struct cpu_user_regs *regs);
 
+unsigned int intc_irq_nums(void);
+
 #endif /* ASM__RISCV__INTERRUPT_CONTOLLER_H */
diff --git a/xen/arch/riscv/include/asm/vaplic.h b/xen/arch/riscv/include/asm/vaplic.h
new file mode 100644
index 000000000000..630ca14657f2
--- /dev/null
+++ b/xen/arch/riscv/include/asm/vaplic.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * xen/arch/riscv/vaplic.c
+ *
+ * Virtual RISC-V Advanced Platform-Level Interrupt Controller support
+ *
+ * Copyright (c) Microchip.
+ */
+
+#ifndef ASM__RISCV__VAPLIC_H
+#define ASM__RISCV__VAPLIC_H
+
+#include <xen/kernel.h>
+#include <xen/types.h>
+
+#include <asm/intc.h>
+
+struct domain;
+
+#define to_vaplic(d) container_of(d->arch.vintc, struct vaplic, vintc)
+
+struct vaplic_regs {
+    uint32_t domaincfg;
+};
+
+struct vaplic {
+    struct vintc vintc;
+    struct vaplic_regs regs;
+};
+
+int domain_vaplic_init(struct domain *d);
+void domain_vaplic_deinit(struct domain *d);
+
+#endif /* ASM__RISCV__VAPLIC_H */
diff --git a/xen/arch/riscv/intc.c b/xen/arch/riscv/intc.c
index a611c9dc5f32..122adf78a391 100644
--- a/xen/arch/riscv/intc.c
+++ b/xen/arch/riscv/intc.c
@@ -78,6 +78,13 @@ void intc_route_irq_to_xen(struct irq_desc *desc, unsigned int priority)
     intc_set_irq_priority(desc, priority);
 }
 
+unsigned int intc_irq_nums(void)
+{
+    ASSERT(intc_hw_ops && intc_hw_ops->irq_nums);
+
+    return intc_hw_ops->irq_nums();
+}
+
 int __init make_intc_domU_node(struct kernel_info *kinfo)
 {
     struct vintc *vintc = kinfo->bd.d->arch.vintc;
diff --git a/xen/arch/riscv/vaplic.c b/xen/arch/riscv/vaplic.c
new file mode 100644
index 000000000000..d2ec196668bc
--- /dev/null
+++ b/xen/arch/riscv/vaplic.c
@@ -0,0 +1,78 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * xen/arch/riscv/vaplic.c
+ *
+ * Virtual RISC-V Advanced Platform-Level Interrupt Controller support
+ *
+ * Copyright (c) Microchip.
+ * Copyright (c) Vates
+ */
+
+#include <xen/errno.h>
+#include <xen/sched.h>
+#include <xen/xvmalloc.h>
+
+#include <asm/aia.h>
+#include <asm/imsic.h>
+#include <asm/intc.h>
+#include <asm/vaplic.h>
+
+#include "aplic-priv.h"
+
+#define VAPLIC_NUM_SOURCES 96
+
+static int cf_check vcpu_vaplic_init(struct vcpu *v)
+{
+    int rc = 0;
+    unsigned int vgein_id;
+
+    rc = vcpu_imsic_init(v);
+    if ( rc )
+        return rc;
+
+    if ( !(vgein_id = vgein_assign(v)) )
+    {
+        printk("Software interrupt files aren't supported\n");
+        rc = -EOPNOTSUPP;
+        goto fail;
+    }
+
+    imsic_set_guest_file_id(v, vgein_id);
+
+    return rc;
+
+ fail:
+    vcpu_imsic_deinit(v);
+
+    return rc;
+}
+
+static const struct vintc_ops vintc_ops = {
+    .vcpu_init = vcpu_vaplic_init,
+};
+
+int __init domain_vaplic_init(struct domain *d)
+{
+    struct vaplic *vaplic = xvzalloc(struct vaplic);
+
+    if ( !vaplic )
+        return -ENOMEM;
+
+    d->arch.vintc = &vaplic->vintc;
+    d->arch.vintc->ops = &vintc_ops;
+
+    vaplic->regs.domaincfg = APLIC_DOMAINCFG_IE | APLIC_DOMAINCFG_DM;
+
+    d->arch.vintc->irq_nums = min(intc_irq_nums(),
+                                  VAPLIC_NUM_SOURCES + 0U);
+
+
+    return 0;
+}
+
+void __init domain_vaplic_deinit(struct domain *d)
+{
+    struct vaplic *vaplic = to_vaplic(d);
+
+    xvfree(vaplic);
+}
-- 
2.54.0

[PATCH v2 15/26] xen/riscv: introduce (de)initialization helpers for vINTC

[Oleksii Kurochko]

Add common helpers domain_vintc_init() and domain_vintc_deinit() to
allocate and deallocate a virtual interrupt controller (vINTC)
structure and initialize basic virtual interrupt controller registers.

domain_vintc_deinit() isn't called at the momemnt as arch_domain_destroy()
is implemented as stub at the moment.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - Drop __init for domain_vintc_(de)init().
 - Update the commit message.
---
 xen/arch/riscv/domain.c           |  3 +++
 xen/arch/riscv/include/asm/intc.h |  3 +++
 xen/arch/riscv/intc.c             | 36 +++++++++++++++++++++++++++++++
 3 files changed, 42 insertions(+)

diff --git a/xen/arch/riscv/domain.c b/xen/arch/riscv/domain.c
index 7aa00d6cfd95..6de5b82b3724 100644
--- a/xen/arch/riscv/domain.c
+++ b/xen/arch/riscv/domain.c
@@ -306,6 +306,9 @@ int arch_domain_create(struct domain *d,
     if ( (rc = p2m_init(d, config)) != 0)
         goto fail;
 
+    if ( (rc = domain_vintc_init(d)) )
+        goto fail;
+
     return rc;
 
  fail:
diff --git a/xen/arch/riscv/include/asm/intc.h b/xen/arch/riscv/include/asm/intc.h
index 3b30d000066a..89a9c6806e55 100644
--- a/xen/arch/riscv/include/asm/intc.h
+++ b/xen/arch/riscv/include/asm/intc.h
@@ -80,4 +80,7 @@ void intc_handle_external_irqs(struct cpu_user_regs *regs);
 
 unsigned int intc_irq_nums(void);
 
+int domain_vintc_init(struct domain *d);
+void domain_vintc_deinit(struct domain *d);
+
 #endif /* ASM__RISCV__INTERRUPT_CONTOLLER_H */
diff --git a/xen/arch/riscv/intc.c b/xen/arch/riscv/intc.c
index 122adf78a391..7f9850416bb4 100644
--- a/xen/arch/riscv/intc.c
+++ b/xen/arch/riscv/intc.c
@@ -11,6 +11,7 @@
 
 #include <asm/aia.h>
 #include <asm/intc.h>
+#include <asm/vaplic.h>
 
 static const struct intc_hw_operations *__ro_after_init intc_hw_ops;
 
@@ -94,3 +95,38 @@ int __init make_intc_domU_node(struct kernel_info *kinfo)
 
     return -EOPNOTSUPP;
 }
+
+int domain_vintc_init(struct domain *d)
+{
+    int ret = -EOPNOTSUPP;
+    const enum intc_version ver = intc_hw_ops->info->hw_version;
+
+    switch ( ver )
+    {
+    case INTC_APLIC:
+        ret = domain_vaplic_init(d);
+        break;
+
+    default:
+        printk("vintc (ver:%d) isn't implemented\n", ver);
+        break;
+    }
+
+    return ret;
+}
+
+void domain_vintc_deinit(struct domain *d)
+{
+    const enum intc_version ver = intc_hw_ops->info->hw_version;
+
+    switch ( ver )
+    {
+    case INTC_APLIC:
+        domain_vaplic_deinit(d);
+        break;
+
+    default:
+        printk("vintc (ver:%d) isn't implemented\n", ver);
+        break;
+    }
+}
-- 
2.54.0

[PATCH v2 16/26] xen/riscv: create APLIC DT node for guest domains

[Oleksii Kurochko]

Guests require a Device Tree description of the interrupt controller
topology. Add support for creating an APLIC node when building the
guest DT.

Provide stub for imsic_make_dt_node() it will be introduced properly
in follow-up patch.

Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - Avoid as max as possible of host properties inheritance. Only number of
   APLIC's irqs are checked what leads to an introduction of
   get_aplic_irqs_num().
 - Move this patch earlier what leads to an introduction of
   vimsic_make_domu_dt_node() stub.
 - s/vimsic_make_domu_dt_node/imsic_make_domu_dt_node.
 - Refactor vimsic_make_domu_dt_node() to avoid re-usage of APLIC host
   properties.
 - Drop next_phandle as it is now in common code.
 - Drop const for kinfo argument of vimsic_make_domu_dt_node() is is
   going to be updated inside vimsic_make_domu_dt_node().
 - Use introduced before vintc->num_irqs.
---
 xen/arch/riscv/aplic.c                    |  2 +
 xen/arch/riscv/imsic.c                    |  7 +++
 xen/arch/riscv/include/asm/aplic.h        |  9 +++
 xen/arch/riscv/include/asm/guest-layout.h |  2 +
 xen/arch/riscv/include/asm/imsic.h        |  3 +
 xen/arch/riscv/vaplic.c                   | 77 ++++++++++++++++++++++-
 6 files changed, 98 insertions(+), 2 deletions(-)

diff --git a/xen/arch/riscv/aplic.c b/xen/arch/riscv/aplic.c
index aba9f3945236..6ed9118485f3 100644
--- a/xen/arch/riscv/aplic.c
+++ b/xen/arch/riscv/aplic.c
@@ -12,8 +12,10 @@
 #include <xen/const.h>
 #include <xen/device_tree.h>
 #include <xen/errno.h>
+#include <xen/fdt-kernel.h>
 #include <xen/init.h>
 #include <xen/irq.h>
+#include <xen/libfdt/libfdt.h>
 #include <xen/mm.h>
 #include <xen/sections.h>
 #include <xen/spinlock.h>
diff --git a/xen/arch/riscv/imsic.c b/xen/arch/riscv/imsic.c
index 8b46419ca23b..ceea6778d9dc 100644
--- a/xen/arch/riscv/imsic.c
+++ b/xen/arch/riscv/imsic.c
@@ -14,6 +14,7 @@
 #include <xen/cpumask.h>
 #include <xen/device_tree.h>
 #include <xen/errno.h>
+#include <xen/fdt-kernel.h>
 #include <xen/init.h>
 #include <xen/macros.h>
 #include <xen/sched.h>
@@ -522,3 +523,9 @@ int __init imsic_init(const struct dt_device_node *node)
 
     return rc;
 }
+
+int __init vimsic_make_domu_dt_node(struct kernel_info *kinfo,
+                                    unsigned int *phandle)
+{
+    return -EOPNOTSUPP;
+}
diff --git a/xen/arch/riscv/include/asm/aplic.h b/xen/arch/riscv/include/asm/aplic.h
index b0724fe6f360..b42b159496b8 100644
--- a/xen/arch/riscv/include/asm/aplic.h
+++ b/xen/arch/riscv/include/asm/aplic.h
@@ -15,6 +15,7 @@
 
 #include <asm/imsic.h>
 
+#define APLIC_DOMAINCFG_RO80    (0x80U << 24)
 #define APLIC_DOMAINCFG_IE      BIT(8, U)
 #define APLIC_DOMAINCFG_DM      BIT(2, U)
 
@@ -27,6 +28,14 @@
 
 #define APLIC_TARGET_HART_IDX_SHIFT 18
 
+#define APLIC_IDC_SIZE          32
+
+#define APLIC_MIN_SIZE          0x4000
+#define APLIC_SIZE_ALIGN(x)     ROUNDUP(x, APLIC_MIN_SIZE)
+
+#define APLIC_SIZE(nr_cpus)     (APLIC_MIN_SIZE + \
+                                 APLIC_SIZE_ALIGN(APLIC_IDC_SIZE * (nr_cpus)))
+
 struct aplic_regs {
     uint32_t domaincfg;         /* 0x0000 */
     uint32_t sourcecfg[1023];   /* 0x0004 */
diff --git a/xen/arch/riscv/include/asm/guest-layout.h b/xen/arch/riscv/include/asm/guest-layout.h
index 68d95a09394c..9fc990c057f2 100644
--- a/xen/arch/riscv/include/asm/guest-layout.h
+++ b/xen/arch/riscv/include/asm/guest-layout.h
@@ -3,6 +3,8 @@
 
 #include <public/xen.h>
 
+#define GUEST_APLIC_S_BASE 0xd000000
+
 #define GUEST_RAM_BANKS   2
 
 /*
diff --git a/xen/arch/riscv/include/asm/imsic.h b/xen/arch/riscv/include/asm/imsic.h
index 2b84824cd377..604f88db9322 100644
--- a/xen/arch/riscv/include/asm/imsic.h
+++ b/xen/arch/riscv/include/asm/imsic.h
@@ -78,6 +78,7 @@ struct vimsic_state {
 };
 
 struct dt_device_node;
+struct kernel_info;
 struct vcpu;
 
 int imsic_init(const struct dt_device_node *node);
@@ -94,4 +95,6 @@ void vcpu_imsic_deinit(const struct vcpu *v);
 unsigned int vcpu_guest_file_id(const struct vcpu *v);
 void imsic_set_guest_file_id(const struct vcpu *v, unsigned int guest_file_id);
 
+int vimsic_make_domu_dt_node(struct kernel_info *kinfo, unsigned int *phandle);
+
 #endif /* ASM_RISCV_IMSIC_H */
diff --git a/xen/arch/riscv/vaplic.c b/xen/arch/riscv/vaplic.c
index d2ec196668bc..3f967464335a 100644
--- a/xen/arch/riscv/vaplic.c
+++ b/xen/arch/riscv/vaplic.c
@@ -9,6 +9,8 @@
  */
 
 #include <xen/errno.h>
+#include <xen/fdt-kernel.h>
+#include <xen/libfdt/libfdt.h>
 #include <xen/sched.h>
 #include <xen/xvmalloc.h>
 
@@ -19,8 +21,11 @@
 
 #include "aplic-priv.h"
 
+#define VAPLIC_COMPATIBLE "riscv,aplic"
 #define VAPLIC_NUM_SOURCES 96
 
+#define FDT_VAPLIC_INT_CELLS 2
+
 static int cf_check vcpu_vaplic_init(struct vcpu *v)
 {
     int rc = 0;
@@ -47,6 +52,73 @@ static int cf_check vcpu_vaplic_init(struct vcpu *v)
     return rc;
 }
 
+static int __init cf_check vaplic_make_domu_dt_node(struct kernel_info *kinfo)
+{
+    int res = 0;
+    void *fdt = kinfo->fdt;
+    unsigned int msi_parent_phandle;
+    char vaplic_name[128];
+    paddr_t aplic_addr = GUEST_APLIC_S_BASE;
+    paddr_t aplic_size = APLIC_SIZE(kinfo->bd.d->max_vcpus);
+    const __be32 reg[] = {
+        cpu_to_be32(aplic_addr >> 32),
+        cpu_to_be32(aplic_addr),
+        cpu_to_be32(aplic_size >> 32),
+        cpu_to_be32(aplic_size),
+    };
+    struct vintc *vintc = kinfo->bd.d->arch.vintc;
+
+    res = snprintf(vaplic_name, sizeof(vaplic_name), "/soc/aplic@%x",
+                   GUEST_APLIC_S_BASE);
+    if ( res >= sizeof(vaplic_name) )
+    {
+        dprintk(XENLOG_DEBUG, "vaplic name is truncated\n");
+        return -ENOBUFS;
+    }
+
+    res = vimsic_make_domu_dt_node(kinfo, &msi_parent_phandle);
+    if ( res )
+        return res;
+
+    res = fdt_begin_node(fdt, vaplic_name);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "#interrupt-cells", FDT_VAPLIC_INT_CELLS);
+    if ( res )
+        return res;
+
+    res = fdt_property(fdt, "reg", reg, sizeof(reg));
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "riscv,num-sources", vintc->irq_nums);
+    if ( res )
+        return res;
+
+    res = fdt_property(fdt, "interrupt-controller", NULL, 0);
+    if ( res )
+        return res;
+
+    res = fdt_property_string(fdt, "compatible", VAPLIC_COMPATIBLE);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "msi-parent", msi_parent_phandle);
+    if ( res )
+        return res;
+
+    res = fdt_property_cell(fdt, "phandle", kinfo->phandle_intc);
+    if ( res )
+        return res;
+
+    return fdt_end_node(fdt);
+}
+
+static const struct vintc_init_ops __initdata init_ops = {
+    .make_domu_dt_node = vaplic_make_domu_dt_node,
+};
+
 static const struct vintc_ops vintc_ops = {
     .vcpu_init = vcpu_vaplic_init,
 };
@@ -60,13 +132,14 @@ int __init domain_vaplic_init(struct domain *d)
 
     d->arch.vintc = &vaplic->vintc;
     d->arch.vintc->ops = &vintc_ops;
+    d->arch.vintc->init_ops = &init_ops;
 
-    vaplic->regs.domaincfg = APLIC_DOMAINCFG_IE | APLIC_DOMAINCFG_DM;
+    vaplic->regs.domaincfg = APLIC_DOMAINCFG_IE | APLIC_DOMAINCFG_DM |
+                             APLIC_DOMAINCFG_RO80;
 
     d->arch.vintc->irq_nums = min(intc_irq_nums(),
                                   VAPLIC_NUM_SOURCES + 0U);
 
-
     return 0;
 }
 
-- 
2.54.0

[PATCH v2 17/26] xen/riscv: generate IMSIC DT node for guest domains

[Oleksii Kurochko]

Guests using the IMSIC interrupt controller require a corresponding
Device Tree description.

Add support for generating an IMSIC node when building the guest DT.
This allows guests to discover and use the IMSIC interrupt controller.

Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - s/imsic_make_reg_property/guest_imsic_make_reg_property.
 - s/imsic_set_interrupt_extended_prop/guest_imsic_set_interrupt_extended_prop.
 - Use initalizer for regs[] array in imsic_make_reg_property().
 - Move buf[] insde the for() loop.
 - Correct check of returned phandle.
 - Drop local variable len.
 - /s/XVFREE/xvfree in imsic_set_interrupt_extended_prop().
 - Drop initializer for local variable data.
 - s/uint32_t/unsinged int for pos and cpu in imsic_set_interrupt_extended_prop().
 - Drop next_phandle as it is now in common code.
 - Introduce vcpu_imsic_deinit.
 - Refactor vimsic_make_domu_dt_node() to avoid usage of host IMSIC dt node.
---
 xen/arch/riscv/imsic.c                    | 127 +++++++++++++++++++++-
 xen/arch/riscv/include/asm/guest-layout.h |   2 +
 2 files changed, 128 insertions(+), 1 deletion(-)

diff --git a/xen/arch/riscv/imsic.c b/xen/arch/riscv/imsic.c
index ceea6778d9dc..19cbacdf96e1 100644
--- a/xen/arch/riscv/imsic.c
+++ b/xen/arch/riscv/imsic.c
@@ -13,9 +13,12 @@
 #include <xen/const.h>
 #include <xen/cpumask.h>
 #include <xen/device_tree.h>
+#include <xen/domain.h>
 #include <xen/errno.h>
+#include <xen/fdt-domain-build.h>
 #include <xen/fdt-kernel.h>
 #include <xen/init.h>
+#include <xen/libfdt/libfdt.h>
 #include <xen/macros.h>
 #include <xen/sched.h>
 #include <xen/smp.h>
@@ -35,6 +38,11 @@ static struct imsic_config imsic_cfg = {
     .lock = SPIN_LOCK_UNLOCKED,
 };
 
+static unsigned int __ro_after_init guest_num_msis;
+
+#define GUEST_IMSIC_COMPATIBLE "riscv,imsics"
+#define GUEST_IMSIC_NUM_MSIS 255
+
 #define IMSIC_DISABLE_EIDELIVERY    0
 #define IMSIC_ENABLE_EIDELIVERY     1
 #define IMSIC_DISABLE_EITHRESHOLD   1
@@ -291,6 +299,11 @@ static int imsic_parse_node(const struct dt_device_node *node,
         return -ENOENT;
     }
 
+    if ( dt_property_read_u32(node, "riscv,num-guest-ids", &tmp) )
+        guest_num_msis = tmp;
+    else
+        guest_num_msis = imsic_cfg.nr_ids;
+
     if ( (imsic_cfg.nr_ids < IMSIC_MIN_ID) ||
          (imsic_cfg.nr_ids > IMSIC_MAX_ID) )
     {
@@ -524,8 +537,120 @@ int __init imsic_init(const struct dt_device_node *node)
     return rc;
 }
 
+static int __init guest_imsic_make_reg_property(struct domain *d, void *fdt)
+{
+    paddr_t base_addr = GUEST_IMSIC_S_BASE;
+    __be32 regs[4] = {
+        cpu_to_be32(base_addr >> 32),
+        cpu_to_be32(base_addr),
+        cpu_to_be32((IMSIC_MMIO_PAGE_SZ * d->max_vcpus) >> 32),
+        cpu_to_be32(IMSIC_MMIO_PAGE_SZ * d->max_vcpus),
+    };
+
+    return fdt_property(fdt, "reg", regs, sizeof(regs));
+}
+
+static int __init guest_imsic_set_interrupt_extended_prop(struct domain *d,
+                                                          void *fdt)
+{
+    unsigned int cpu, pos = 0;
+    uint32_t phandle;
+    uint32_t *irq_ext;
+    int res;
+
+    irq_ext = xvzalloc_array(uint32_t, d->max_vcpus * 2);
+    if ( !irq_ext )
+        return -ENOMEM;
+
+    for ( cpu = 0; cpu < d->max_vcpus; cpu++ )
+    {
+        char buf[64];
+
+        snprintf(buf, sizeof(buf), "/cpus/cpu@%u/interrupt-controller", cpu);
+        phandle = fdt_get_phandle(fdt, fdt_path_offset(fdt, buf));
+
+        if ( !phandle )
+        {
+            res = -ENODEV;
+            goto out;
+        }
+
+        irq_ext[pos++] = cpu_to_be32(phandle);
+        irq_ext[pos++] = cpu_to_be32(IRQ_S_EXT);
+    }
+
+    res = fdt_property(fdt, "interrupts-extended", irq_ext,
+                       d->max_vcpus * 2 * sizeof(*irq_ext));
+
+ out:
+    xvfree(irq_ext);
+
+    return res;
+}
+
 int __init vimsic_make_domu_dt_node(struct kernel_info *kinfo,
                                     unsigned int *phandle)
 {
-    return -EOPNOTSUPP;
+    int res;
+    void *fdt = kinfo->fdt;
+    char vimsic_name[128];
+    unsigned int vimsic_phandle;
+    unsigned int num_msis = min(GUEST_IMSIC_NUM_MSIS + 0U, guest_num_msis);
+
+    res = snprintf(vimsic_name, sizeof(vimsic_name), "/soc/imsic@%x",
+                   GUEST_IMSIC_S_BASE);
+    if ( res >= sizeof(vimsic_name) )
+    {
+        dprintk(XENLOG_DEBUG, "vimsic name is truncated\n");
+        return -ENOBUFS;
+    }
+
+    res = fdt_begin_node(fdt, vimsic_name);
+    if ( res )
+        return res;
+
+    res = fdt_property_string(fdt, "compatible", GUEST_IMSIC_COMPATIBLE);
+    if ( res )
+        return res;
+
+    res = guest_imsic_make_reg_property(kinfo->bd.d, fdt);
+    if ( res )
+        return res;
+
+    res = guest_imsic_set_interrupt_extended_prop(kinfo->bd.d, fdt);
+    if ( res )
+        return res;
+
+    res = fdt_property_u32(fdt, "riscv,num-ids", num_msis);
+    if ( res )
+        return res;
+
+    res = fdt_property(fdt, "msi-controller", NULL, 0);
+    if ( res )
+        return res;
+
+    res = fdt_property_u32(fdt, "#msi-cells", 0);
+    if ( res )
+        return res;
+
+    res = fdt_property(fdt, "interrupt-controller", NULL, 0);
+    if ( res )
+        return res;
+
+    res = fdt_property_u32(fdt, "#interrupt-cells", 0);
+    if ( res )
+        return res;
+
+    vimsic_phandle = alloc_phandle(kinfo);
+    if ( !vimsic_phandle )
+        return -EOVERFLOW;
+
+    res = fdt_property_cell(fdt, "phandle", vimsic_phandle);
+    if ( res )
+        return res;
+
+    if ( phandle )
+        *phandle = vimsic_phandle;
+
+    return fdt_end_node(fdt);
 }
diff --git a/xen/arch/riscv/include/asm/guest-layout.h b/xen/arch/riscv/include/asm/guest-layout.h
index 9fc990c057f2..b16ec79c3786 100644
--- a/xen/arch/riscv/include/asm/guest-layout.h
+++ b/xen/arch/riscv/include/asm/guest-layout.h
@@ -5,6 +5,8 @@
 
 #define GUEST_APLIC_S_BASE 0xd000000
 
+#define GUEST_IMSIC_S_BASE 0x28000000
+
 #define GUEST_RAM_BANKS   2
 
 /*
-- 
2.54.0

[PATCH v2 18/26] xen: move declaration of map_device_irqs_to_domain() to common header

[Oleksii Kurochko]

As map_device_irqs_to_domain() is used unconditionally by common part of
dom0less code it is moved to common header.

fdt-domain-build.h is chosen as map_device_irqs_to_domain() could be
also called indirectly in Arm's DOM0-related code.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - New patch.
---
 xen/arch/arm/include/asm/setup.h   |  3 ---
 xen/include/xen/fdt-domain-build.h | 13 +++++++++++++
 2 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/xen/arch/arm/include/asm/setup.h b/xen/arch/arm/include/asm/setup.h
index 0d29b46ea52b..0adfa4993a8f 100644
--- a/xen/arch/arm/include/asm/setup.h
+++ b/xen/arch/arm/include/asm/setup.h
@@ -53,9 +53,6 @@ void init_traps(void);
 int handle_device(struct domain *d, struct dt_device_node *dev, p2m_type_t p2mt,
                   struct rangeset *iomem_ranges, struct rangeset *irq_ranges);
 
-int map_device_irqs_to_domain(struct domain *d, struct dt_device_node *dev,
-                              bool need_mapping, struct rangeset *irq_ranges);
-
 int map_irq_to_domain(struct domain *d, unsigned int irq,
                       bool need_mapping, const char *devname);
 
diff --git a/xen/include/xen/fdt-domain-build.h b/xen/include/xen/fdt-domain-build.h
index 671486c1c837..8612e98dfda5 100644
--- a/xen/include/xen/fdt-domain-build.h
+++ b/xen/include/xen/fdt-domain-build.h
@@ -12,6 +12,7 @@
 
 struct domain;
 struct page_info;
+struct rangeset;
 struct membanks;
 
 typedef bool (*alloc_domheap_mem_cb)(struct domain *d, struct page_info *pg,
@@ -79,6 +80,18 @@ static inline void set_domain_type(struct domain *d, const struct kernel_info *k
 #endif
 }
 
+/*
+ * Retrieves the interrupts configuration from a device tree node and maps
+ * those interrupts to the target domain.
+ *
+ * Returns:
+ *   < 0 error
+ *   0   success
+ */
+int map_device_irqs_to_domain(struct domain *d, struct dt_device_node *dev,
+                              bool need_mapping,
+                              struct rangeset *irq_ranges);
+
 #endif /* __XEN_FDT_DOMAIN_BUILD_H__ */
 
 /*
-- 
2.54.0

[PATCH v2 19/26] xen/riscv: implement IRQ routing for device passthrough

[Oleksii Kurochko]

dom0less device passthrough requires granting guest domains access to
device interrupts.  Introduce map_device_irqs_to_domain() to enumerate
a DT node's interrupt properties, skipping those not owned by
the primary interrupt controller (as at the moment I haven't seen usages
of it), and map_irq_to_domain() to grant domain access and configure
Xen's interrupt descriptor accordingly. Sharing IRQ between domains is
rejected.

route_irq_to_guest() and release_irq() manage irq_desc ownership for
guest-assigned interrupts.  Each assignment carries a small irq_guest
structure as irqaction::dev_id, recording the owning domain and virtual
IRQ number which is 1:1 mapped to physical IRQ number.  A per-domain
vIRQ allocation bitmap (allocated_irqs in struct vintc), managed by
vintc_reserve_virq(), prevents the same vIRQ being claimed twice.

With APLIC+IMSIC, guest interrupts are delivered directly by hardware
through the IMSIC, bypassing do_IRQ(). The _IRQ_GUEST branch in
do_IRQ() is therefore left as BUG() until a platform without direct
IMSIC delivery is encountered.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - Rework IRQ mapping in more common (similar approach to Arm).
---
 xen/arch/riscv/Makefile            |   1 +
 xen/arch/riscv/aplic.c             |   4 +
 xen/arch/riscv/device.c            | 108 ++++++++++++++++++
 xen/arch/riscv/include/asm/intc.h  |  10 ++
 xen/arch/riscv/include/asm/irq.h   |   3 +
 xen/arch/riscv/include/asm/setup.h |   4 +
 xen/arch/riscv/intc.c              |  36 ++++++
 xen/arch/riscv/irq.c               | 175 +++++++++++++++++++++++++++++
 8 files changed, 341 insertions(+)
 create mode 100644 xen/arch/riscv/device.c

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 9d8d21b65188..fc6b34661111 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -1,6 +1,7 @@
 obj-y += aia.o
 obj-y += aplic.o
 obj-y += cpufeature.o
+obj-y += device.o
 obj-y += domain.o
 obj-y += domain-build.init.o
 obj-$(CONFIG_DOM0LESS_BOOT) += dom0less-build.init.o
diff --git a/xen/arch/riscv/aplic.c b/xen/arch/riscv/aplic.c
index 6ed9118485f3..1c8fd0145eb2 100644
--- a/xen/arch/riscv/aplic.c
+++ b/xen/arch/riscv/aplic.c
@@ -311,9 +311,13 @@ static const hw_irq_controller aplic_xen_irq_type = {
     .set_affinity = aplic_set_irq_affinity,
 };
 
+/* At the moment there is no difference between guest and Xen ops */
+#define aplic_guest_irq_type aplic_xen_irq_type
+
 static const struct intc_hw_operations aplic_ops = {
     .info                = &aplic_info,
     .host_irq_type       = &aplic_xen_irq_type,
+    .guest_irq_type      = &aplic_guest_irq_type,
     .handle_interrupt    = aplic_handle_interrupt,
     .set_irq_type        = aplic_set_irq_type,
     .irq_nums            = aplic_irq_num,
diff --git a/xen/arch/riscv/device.c b/xen/arch/riscv/device.c
new file mode 100644
index 000000000000..ba78032028af
--- /dev/null
+++ b/xen/arch/riscv/device.c
@@ -0,0 +1,108 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include <xen/device_tree.h>
+#include <xen/errno.h>
+#include <xen/iocap.h>
+#include <xen/rangeset.h>
+#include <xen/sched.h>
+
+#include <asm/intc.h>
+
+int map_irq_to_domain(struct domain *d, unsigned int irq,
+                      bool need_mapping, const char *devname)
+{
+    int res;
+
+    res = irq_permit_access(d, irq);
+    if ( res )
+    {
+        printk(XENLOG_ERR "Unable to permit to %pd access to IRQ %u\n", d, irq);
+        return res;
+    }
+
+    if ( need_mapping )
+    {
+        /*
+         * Checking the return of vintc_reserve_virq is not
+         * necessary. It should not fail except when we try to map
+         * the IRQ twice. This can legitimately happen if the IRQ is shared.
+         */
+        vintc_reserve_virq(d, irq);
+
+        res = route_irq_to_guest(d, irq, irq, devname);
+        if ( res < 0 )
+        {
+            printk(XENLOG_ERR "Unable to map IRQ%u to %pd\n", irq, d);
+            return res;
+        }
+    }
+
+    dt_dprintk("  - IRQ: %u\n", irq);
+
+    return 0;
+}
+
+/*
+ * map_device_irqs_to_domain retrieves the interrupts configuration from
+ * a device tree node and maps those interrupts to the target domain.
+ *
+ * Returns:
+ *   < 0 error
+ *   0   success
+ */
+int map_device_irqs_to_domain(struct domain *d,
+                              struct dt_device_node *dev,
+                              bool need_mapping,
+                              struct rangeset *irq_ranges)
+{
+    unsigned int i, nirq;
+    int res, irq;
+    struct dt_raw_irq rirq;
+
+    nirq = dt_number_of_irq(dev);
+
+    /* Give permission and map IRQs */
+    for ( i = 0; i < nirq; i++ )
+    {
+        res = dt_device_get_raw_irq(dev, i, &rirq);
+        if ( res )
+        {
+            printk(XENLOG_ERR "Unable to retrieve irq %u for %s\n",
+                   i, dt_node_full_name(dev));
+            return res;
+        }
+
+        /*
+         * Don't map IRQ that have no physical meaning
+         * ie: IRQ whose controller is not APLIC/IMSIC/PLIC.
+         */
+        if ( rirq.controller != dt_interrupt_controller )
+        {
+            dt_dprintk("irq %u not connected to primary controller."
+                       "Connected to %s\n", i,
+                       dt_node_full_name(rirq.controller));
+            continue;
+        }
+
+        irq = platform_get_irq(dev, i);
+        if ( irq < 0 )
+        {
+            printk("Unable to get irq %u for %s\n", i, dt_node_full_name(dev));
+            return irq;
+        }
+
+        res = map_irq_to_domain(d, irq, need_mapping, dt_node_name(dev));
+        if ( res )
+            return res;
+
+
+        /*
+         * At the moment there is only one user of map_device_irqs_to_domain()
+         * for RISC-V which calls it irq_ranges == NULL.
+         */
+        if ( irq_ranges )
+            return -EOPNOTSUPP;
+    }
+
+    return 0;
+}
diff --git a/xen/arch/riscv/include/asm/intc.h b/xen/arch/riscv/include/asm/intc.h
index 89a9c6806e55..1ff992b6c20b 100644
--- a/xen/arch/riscv/include/asm/intc.h
+++ b/xen/arch/riscv/include/asm/intc.h
@@ -13,8 +13,11 @@ enum intc_version {
 };
 
 struct cpu_user_regs;
+struct domain;
+struct dt_device_node;
 struct irq_desc;
 struct kernel_info;
+struct rangeset;
 struct vcpu;
 
 struct intc_info {
@@ -32,6 +35,9 @@ struct intc_hw_operations {
     /* hw_irq_controller to enable/disable/eoi host irq */
     const struct hw_interrupt_type *host_irq_type;
 
+    /* hw_irq_controller to enable/disable/eoi guest irq */
+    const struct hw_interrupt_type *guest_irq_type;
+
     /* Set IRQ type */
     void (*set_irq_type)(struct irq_desc *desc, unsigned int type);
     /* Set IRQ priority */
@@ -63,6 +69,7 @@ struct vintc_ops {
 
 struct vintc {
     unsigned int irq_nums;
+    unsigned long *allocated_irqs;
     const struct vintc_init_ops *init_ops;
     const struct vintc_ops *ops;
 };
@@ -75,6 +82,7 @@ void register_intc_ops(const struct intc_hw_operations *ops,
 void intc_init(void);
 
 void intc_route_irq_to_xen(struct irq_desc *desc, unsigned int priority);
+int intc_route_irq_to_guest(struct irq_desc *desc, unsigned int priority);
 
 void intc_handle_external_irqs(struct cpu_user_regs *regs);
 
@@ -83,4 +91,6 @@ unsigned int intc_irq_nums(void);
 int domain_vintc_init(struct domain *d);
 void domain_vintc_deinit(struct domain *d);
 
+bool vintc_reserve_virq(const struct domain *d, unsigned int virq);
+
 #endif /* ASM__RISCV__INTERRUPT_CONTOLLER_H */
diff --git a/xen/arch/riscv/include/asm/irq.h b/xen/arch/riscv/include/asm/irq.h
index f633636dc308..98dd536be86e 100644
--- a/xen/arch/riscv/include/asm/irq.h
+++ b/xen/arch/riscv/include/asm/irq.h
@@ -49,6 +49,9 @@ void init_IRQ(void);
 
 void do_IRQ(struct cpu_user_regs *regs, unsigned int irq);
 
+int route_irq_to_guest(struct domain *d, unsigned int virq,
+                       unsigned int irq, const char *devname);
+
 #endif /* ASM__RISCV__IRQ_H */
 
 /*
diff --git a/xen/arch/riscv/include/asm/setup.h b/xen/arch/riscv/include/asm/setup.h
index 2215894cfbb1..9dbd3a8cbef7 100644
--- a/xen/arch/riscv/include/asm/setup.h
+++ b/xen/arch/riscv/include/asm/setup.h
@@ -5,6 +5,10 @@
 
 #include <xen/types.h>
 
+struct domain;
+struct dt_device_node;
+struct rangeset;
+
 #define max_init_domid (0)
 
 void setup_mm(void);
diff --git a/xen/arch/riscv/intc.c b/xen/arch/riscv/intc.c
index 7f9850416bb4..a24c679c0cf8 100644
--- a/xen/arch/riscv/intc.c
+++ b/xen/arch/riscv/intc.c
@@ -7,7 +7,9 @@
 #include <xen/init.h>
 #include <xen/irq.h>
 #include <xen/lib.h>
+#include <xen/sched.h>
 #include <xen/spinlock.h>
+#include <xen/xvmalloc.h>
 
 #include <asm/aia.h>
 #include <asm/intc.h>
@@ -86,6 +88,22 @@ unsigned int intc_irq_nums(void)
     return intc_hw_ops->irq_nums();
 }
 
+int intc_route_irq_to_guest(struct irq_desc *desc,
+                            unsigned int priority)
+{
+    ASSERT(spin_is_locked(&desc->lock));
+
+    ASSERT(intc_hw_ops->guest_irq_type);
+
+    desc->handler = intc_hw_ops->guest_irq_type;
+    set_bit(_IRQ_GUEST, &desc->status);
+
+    intc_set_irq_type(desc, desc->arch.type);
+    intc_set_irq_priority(desc, priority);
+
+    return 0;
+}
+
 int __init make_intc_domU_node(struct kernel_info *kinfo)
 {
     struct vintc *vintc = kinfo->bd.d->arch.vintc;
@@ -112,6 +130,14 @@ int domain_vintc_init(struct domain *d)
         break;
     }
 
+    if ( !ret )
+    {
+        d->arch.vintc->allocated_irqs =
+            xvzalloc_array(unsigned long, BITS_TO_LONGS(d->arch.vintc->irq_nums));
+        if ( !d->arch.vintc->allocated_irqs )
+            ret = -ENOMEM;
+    }
+
     return ret;
 }
 
@@ -129,4 +155,14 @@ void domain_vintc_deinit(struct domain *d)
         printk("vintc (ver:%d) isn't implemented\n", ver);
         break;
     }
+
+    xvfree(d->arch.vintc->allocated_irqs);
+}
+
+bool vintc_reserve_virq(const struct domain *d, unsigned int virq)
+{
+    if ( virq >= d->arch.vintc->irq_nums )
+        return false;
+
+    return !test_and_set_bit(virq, d->arch.vintc->allocated_irqs);
 }
diff --git a/xen/arch/riscv/irq.c b/xen/arch/riscv/irq.c
index 25d329500212..4481ae23aa8e 100644
--- a/xen/arch/riscv/irq.c
+++ b/xen/arch/riscv/irq.c
@@ -12,11 +12,20 @@
 #include <xen/errno.h>
 #include <xen/init.h>
 #include <xen/irq.h>
+#include <xen/sched.h>
 #include <xen/spinlock.h>
+#include <xen/xvmalloc.h>
 
 #include <asm/hardirq.h>
 #include <asm/intc.h>
 
+/* Describe an IRQ assigned to a guest */
+struct irq_guest
+{
+    struct domain *d;
+    unsigned int virq;
+};
+
 static irq_desc_t irq_desc[NR_IRQS];
 
 static bool irq_validate_new_type(unsigned int curr, unsigned int new)
@@ -192,6 +201,15 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq)
     if ( desc->handler->ack )
         desc->handler->ack(desc);
 
+    if ( desc->status & IRQ_GUEST )
+        /*
+         * As at the moment APLIC + IMSIC is used for guest interrupts will
+         * be directly passed to guest. But if/when IMSIC won't be available
+         * all interrupts will go through Xenand here an irq injection
+         * will be necessary to do.
+         */
+        panic("unimplemented");
+
     if ( desc->status & IRQ_DISABLED )
         goto out;
 
@@ -221,3 +239,160 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq)
     spin_unlock(&desc->lock);
     irq_exit();
 }
+
+static inline struct irq_guest *irq_get_guest_info(struct irq_desc *desc)
+{
+    ASSERT(spin_is_locked(&desc->lock));
+    ASSERT(test_bit(_IRQ_GUEST, &desc->status));
+    ASSERT(desc->action != NULL);
+
+    return desc->action->dev_id;
+}
+
+static inline struct domain *irq_get_domain(struct irq_desc *desc)
+{
+    return irq_get_guest_info(desc)->d;
+}
+
+void release_irq(unsigned int irq, const void *dev_id)
+{
+    struct irq_desc *desc;
+    unsigned long flags;
+    struct irqaction *action, **action_ptr;
+
+    desc = irq_to_desc(irq);
+
+    spin_lock_irqsave(&desc->lock,flags);
+
+    action_ptr = &desc->action;
+#ifdef CONFIG_IRQ_HAS_MULTIPLE_ACTION
+    for ( ;; )
+    {
+        action = *action_ptr;
+        if ( !action )
+        {
+            printk(XENLOG_WARNING "Trying to free already-free IRQ %u\n", irq);
+            spin_unlock_irqrestore(&desc->lock, flags);
+            return;
+        }
+
+        if ( action->dev_id == dev_id )
+            break;
+
+        action_ptr = &action->next;
+    }
+
+    /* Found it - remove it from the action list */
+    *action_ptr = action->next;
+#else
+    action = *action_ptr;
+#endif
+
+    /* If this was the last action, shut down the IRQ */
+    if ( !desc->action )
+    {
+        desc->handler->shutdown(desc);
+        clear_bit(_IRQ_GUEST, &desc->status);
+    }
+
+    spin_unlock_irqrestore(&desc->lock,flags);
+
+    /* Wait to make sure it's not being used on another CPU */
+    do { smp_mb(); } while ( test_bit(_IRQ_INPROGRESS, &desc->status) );
+
+    if ( action->free_on_release )
+        xvfree(action);
+}
+
+/* Route an IRQ to a specific guest */
+int route_irq_to_guest(struct domain *d, unsigned int virq,
+                       unsigned int irq, const char *devname)
+{
+    struct irqaction *action;
+    struct irq_guest *info;
+    struct irq_desc *desc;
+    unsigned long flags;
+    int retval = 0;
+
+    desc = irq_to_desc(irq);
+
+    action = xvmalloc(struct irqaction);
+    if ( !action )
+        return -ENOMEM;
+
+    info = xvmalloc(struct irq_guest);
+    if ( !info )
+    {
+        xvfree(action);
+        return -ENOMEM;
+    }
+
+    info->d = d;
+    info->virq = virq;
+
+    action->dev_id = info;
+    action->name = devname;
+    action->free_on_release = 1;
+
+    spin_lock_irqsave(&desc->lock, flags);
+
+    /*
+     * If the IRQ is already used by someone
+     *  - If it's the same domain -> Xen doesn't need to update the IRQ desc.
+     *  For safety check if we are not trying to assign the IRQ to a
+     *  different vIRQ.
+     *  - Otherwise -> For now, don't allow the IRQ to be shared between
+     *  Xen and domains.
+     */
+    if ( desc->action != NULL )
+    {
+        if ( test_bit(_IRQ_GUEST, &desc->status) )
+        {
+            struct domain *ad = irq_get_domain(desc);
+
+            if ( d != ad )
+            {
+                printk(XENLOG_G_ERR "IRQ %u is already used by domain %u\n",
+                       irq, ad->domain_id);
+                retval = -EBUSY;
+            }
+            else if ( irq_get_guest_info(desc)->virq != virq )
+            {
+                printk(XENLOG_G_ERR
+                       "d%u: IRQ %u is already assigned to vIRQ %u\n",
+                       d->domain_id, irq, irq_get_guest_info(desc)->virq);
+                retval = -EBUSY;
+            }
+        }
+        else
+        {
+            printk(XENLOG_G_ERR "IRQ %u is already used by Xen\n", irq);
+            retval = -EBUSY;
+        }
+        goto out;
+    }
+
+    retval = _setup_irq(desc, 0, action);
+    if ( retval )
+        goto out;
+
+    retval = intc_route_irq_to_guest(desc, IRQ_NO_PRIORITY);
+
+    spin_unlock_irqrestore(&desc->lock, flags);
+
+    if ( retval )
+    {
+        release_irq(desc->irq, info);
+        goto free_info;
+    }
+
+    return 0;
+
+ out:
+    spin_unlock_irqrestore(&desc->lock, flags);
+    xvfree(action);
+ free_info:
+    xvfree(info);
+
+    return retval;
+}
-- 
2.54.0

[PATCH v2 20/26] xen/riscv: add missing APLIC register offsets, masks to asm/aplic.h.

[Oleksii Kurochko]

These definitions are required for correct decoding of APLIC MMIO
accesses and target configuration, and will be used by both the
physical and virtual APLIC implementations.

No functional change is intended by this patch; it only centralises
hardware definitions that were previously missing.

Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 xen/arch/riscv/include/asm/aplic.h | 35 ++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/xen/arch/riscv/include/asm/aplic.h b/xen/arch/riscv/include/asm/aplic.h
index b42b159496b8..e418fc53433b 100644
--- a/xen/arch/riscv/include/asm/aplic.h
+++ b/xen/arch/riscv/include/asm/aplic.h
@@ -15,6 +15,11 @@
 
 #include <asm/imsic.h>
 
+#define APLIC_REG_OFFSET_MASK   0x3fff
+#define APLIC_TARGET_IPRIO_MASK 0xff
+#define APLIC_TARGET_GUEST_IDX_SHIFT 12
+#define APLIC_TARGET_EIID_MASK  0x7ff
+
 #define APLIC_DOMAINCFG_RO80    (0x80U << 24)
 #define APLIC_DOMAINCFG_IE      BIT(8, U)
 #define APLIC_DOMAINCFG_DM      BIT(2, U)
@@ -26,6 +31,36 @@
 #define APLIC_SOURCECFG_SM_LEVEL_HIGH   0x6
 #define APLIC_SOURCECFG_SM_LEVEL_LOW    0x7
 
+#define APLIC_DOMAINCFG         0x0000
+#define APLIC_SOURCECFG_BASE    0x0004
+#define APLIC_SOURCECFG_LAST    0x0ffc
+
+#define APLIC_SMSICFGADDR       0x1bc8
+#define APLIC_SMSICFGADDRH      0x1bcc
+
+#define APLIC_SETIP_BASE        0x1c00
+#define APLIC_SETIP_LAST        0x1c7c
+#define APLIC_SETIPNUM          0x1cdc
+
+#define APLIC_CLRIP_BASE        0x1d00
+#define APLIC_CLRIP_LAST        0x1d7c
+#define APLIC_CLRIPNUM          0x1ddc
+
+#define APLIC_SETIE_BASE        0x1e00
+#define APLIC_SETIE_LAST        0x1e7c
+#define APLIC_SETIENUM          0x1edc
+
+#define APLIC_CLRIE_BASE        0x1f00
+#define APLIC_CLRIE_LAST        0x1f7c
+#define APLIC_CLRIENUM          0x1fdc
+
+#define APLIC_SETIPNUM_LE       0x2000
+
+#define APLIC_GENMSI            0x3000
+
+#define APLIC_TARGET_BASE       0x3004
+#define APLIC_TARGET_LAST       0x3ffc
+
 #define APLIC_TARGET_HART_IDX_SHIFT 18
 
 #define APLIC_IDC_SIZE          32
-- 
2.54.0

[PATCH v2 21/26] xen/riscv: implement virtual APLIC MMIO emulation

[Oleksii Kurochko]

Guests running under Xen program interrupt routing by writing to APLIC
MMIO registers. Xen must intercept these accesses to enforce interrupt
isolation between domains and to translate guest routing intent into the
underlying physical MSI topology.

Writes are gated by the domain's authorised interrupt bitmap so that a
guest cannot affect interrupts it does not own. TARGET register writes
additionally require translation of the hart and IMSIC guest-file
indices from virtual to physical, as the APLIC uses these fields
directly to compute the MSI delivery address.

Delegation (APLIC_SOURCECFG_D) is not yet supported.

Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - Merge the following patches into one:
    xen/riscv: add vaplic access check:
      - Add check that address is properly aligned.
      - Check vaplic range intead of APLIC one.
      - Return bool from vaplic_is_access instead of int.
    xen/riscv: emulate guest writes to virtual APLIC MMIO
      - Drop CALC_REG_VALUE.
      - Use unsigned int instead of uin32_t for offset.
      - s/.../subtracting in the comment.
      - start one line comments from the upper case.
      - Check the value before being written to sourcecfg register.
      - 'unsigned int' for loop index.
      - Omit unneessary braces.
      - s/vaplic_update_target/aplic_msi_target_gen.
      - Use IMSIC_MMIO_PAGE_SHIFT instead of 12 in aplic_msi_target_gen().
      - Drop explicit usage of APLIC register in store function.
      - Drop APLIC_REG_{GET,SET} macros and introudce APLIC specific funtcions.
      - Ignore write to SOURCECFG_BASE when value is out-of-range.
      - Drop ASSERT(!target_vcpu) inside handler of targer register setting,
        just avoid such writings + debug message.
      - domain_crash() instead of panic() in the case of default case.
      - Drop ASSERT() in APLIC_SOURCE_CFG_BASE case and use domain_crash()
        instead.
    xen/riscv: emulate guest reads from virtual APLIC MMIO:
      - s/regval_to_irqn/regindx_to_irqn.
      - pass to to_vaplic() a domain instead of vintc.
      - add check that load access is aligned.
      - instead of panic() just crash a domain().
      - use 'unsigned int' for local variable offset.
      - Return 0 in the case APLIC_CLRIE_BASE ...APLIC_CLRIE_LAST reading to
        follow AIA spec.
      - Drop explicit usage of physical APLIC registers.
---
 xen/arch/riscv/aplic.c              |  25 +++
 xen/arch/riscv/include/asm/aplic.h  |   9 +
 xen/arch/riscv/include/asm/intc.h   |  10 +-
 xen/arch/riscv/include/asm/vaplic.h |   3 +
 xen/arch/riscv/vaplic.c             | 289 +++++++++++++++++++++++++++-
 5 files changed, 333 insertions(+), 3 deletions(-)

diff --git a/xen/arch/riscv/aplic.c b/xen/arch/riscv/aplic.c
index 1c8fd0145eb2..1976733dfbaa 100644
--- a/xen/arch/riscv/aplic.c
+++ b/xen/arch/riscv/aplic.c
@@ -40,6 +40,31 @@ static struct intc_info __ro_after_init aplic_info = {
     .hw_version = INTC_APLIC,
 };
 
+uint32_t aplic_hw_read_reg(unsigned int offset, uint32_t mask)
+{
+    unsigned long flags;
+    uint32_t val;
+
+    ASSERT(offset < aplic.size);
+
+    spin_lock_irqsave(&aplic.lock, flags);
+    val = readl((void __iomem *)((uintptr_t)aplic.regs + offset)) & mask;
+    spin_unlock_irqrestore(&aplic.lock, flags);
+
+    return val;
+}
+
+void aplic_hw_write_reg(unsigned int offset, uint32_t value)
+{
+    unsigned long flags;
+
+    ASSERT(offset < aplic.size);
+
+    spin_lock_irqsave(&aplic.lock, flags);
+    writel(value, (void __iomem *)((uintptr_t)aplic.regs + offset));
+    spin_unlock_irqrestore(&aplic.lock, flags);
+}
+
 static void __init aplic_init_hw_interrupts(void)
 {
     unsigned int i;
diff --git a/xen/arch/riscv/include/asm/aplic.h b/xen/arch/riscv/include/asm/aplic.h
index e418fc53433b..7905db9876d4 100644
--- a/xen/arch/riscv/include/asm/aplic.h
+++ b/xen/arch/riscv/include/asm/aplic.h
@@ -15,6 +15,8 @@
 
 #include <asm/imsic.h>
 
+#define APLIC_NUM_REGS 32
+
 #define APLIC_REG_OFFSET_MASK   0x3fff
 #define APLIC_TARGET_IPRIO_MASK 0xff
 #define APLIC_TARGET_GUEST_IDX_SHIFT 12
@@ -24,6 +26,8 @@
 #define APLIC_DOMAINCFG_IE      BIT(8, U)
 #define APLIC_DOMAINCFG_DM      BIT(2, U)
 
+#define APLIC_SOURCECFG_D       BIT(10, U)
+
 #define APLIC_SOURCECFG_SM_INACTIVE     0x0
 #define APLIC_SOURCECFG_SM_DETACH       0x1
 #define APLIC_SOURCECFG_SM_EDGE_RISE    0x4
@@ -71,6 +75,8 @@
 #define APLIC_SIZE(nr_cpus)     (APLIC_MIN_SIZE + \
                                  APLIC_SIZE_ALIGN(APLIC_IDC_SIZE * (nr_cpus)))
 
+#define APLIC_SETCLR_OFFSET_MASK  ((32 * sizeof(uint32_t)) - 1)
+
 struct aplic_regs {
     uint32_t domaincfg;         /* 0x0000 */
     uint32_t sourcecfg[1023];   /* 0x0004 */
@@ -114,4 +120,7 @@ struct aplic_regs {
     uint32_t target[1023];      /* 0x3008 */
 };
 
+uint32_t aplic_hw_read_reg(unsigned int offset, uint32_t mask);
+void aplic_hw_write_reg(unsigned int offset, uint32_t value);
+
 #endif /* ASM_RISCV_APLIC_H */
diff --git a/xen/arch/riscv/include/asm/intc.h b/xen/arch/riscv/include/asm/intc.h
index 1ff992b6c20b..641c83125a5b 100644
--- a/xen/arch/riscv/include/asm/intc.h
+++ b/xen/arch/riscv/include/asm/intc.h
@@ -64,7 +64,15 @@ struct vintc_ops {
     int (*vcpu_init)(struct vcpu *v);
 
     /* Check if a address is virtual interrupt controller MMIO */
-    int (*is_access)(const struct vcpu *v, unsigned long addr);
+    bool (*is_access)(const struct vcpu *v, unsigned long addr);
+
+    /* Emulate load to virtual interrupt controller MMIOs */
+    int (*emulate_load)(const struct vcpu *vcpu, unsigned long addr,
+                        uint32_t *out);
+
+    /* Emulate store to virtual interrupt controller MMIOs */
+    int (*emulate_store)(const struct vcpu *vcpu, unsigned long addr,
+                         uint32_t in);
 };
 
 struct vintc {
diff --git a/xen/arch/riscv/include/asm/vaplic.h b/xen/arch/riscv/include/asm/vaplic.h
index 630ca14657f2..0fa690fcb2d7 100644
--- a/xen/arch/riscv/include/asm/vaplic.h
+++ b/xen/arch/riscv/include/asm/vaplic.h
@@ -26,6 +26,9 @@ struct vaplic_regs {
 struct vaplic {
     struct vintc vintc;
     struct vaplic_regs regs;
+
+    paddr_t regs_start;
+    paddr_t regs_size;
 };
 
 int domain_vaplic_init(struct domain *d);
diff --git a/xen/arch/riscv/vaplic.c b/xen/arch/riscv/vaplic.c
index 3f967464335a..57c3433ba03b 100644
--- a/xen/arch/riscv/vaplic.c
+++ b/xen/arch/riscv/vaplic.c
@@ -26,6 +26,283 @@
 
 #define FDT_VAPLIC_INT_CELLS 2
 
+#define AUTH_IRQ_BIT(d, irqn) ( \
+    ((irqn) <= (d)->arch.vintc->irq_nums) && \
+    test_bit(irqn, (d)->arch.vintc->allocated_irqs) )
+
+#define regindx_to_irqn(reg_val) ((reg_val) / sizeof(uint32_t))
+
+static inline uint32_t generate_auth_mask(const struct domain *d,
+                                          unsigned int irqsn)
+{
+    if ( irqsn >= DIV_ROUND_UP(d->arch.vintc->irq_nums,
+                               sizeof(uint32_t) * BITS_PER_BYTE) )
+    {
+        dprintk(XENLOG_DEBUG, "incorrect irqsn(%d) is passed\n", irqsn);
+
+        return 0U;
+    }
+
+    return *((uint32_t *)d->arch.vintc->allocated_irqs + irqsn);
+}
+
+static int vaplic_emulate_load(const struct vcpu *vcpu,
+                               const unsigned long addr, uint32_t *out)
+{
+    const struct domain *d = vcpu->domain;
+    const struct vaplic *vaplic = to_vaplic(d);
+    const unsigned int offset = addr & APLIC_REG_OFFSET_MASK;
+    uint32_t auth_mask;
+    unsigned int i;
+
+    switch ( offset )
+    {
+    case APLIC_DOMAINCFG:
+        *out = vaplic->regs.domaincfg;
+
+        return 0;
+
+    case APLIC_SETIPNUM:
+    case APLIC_SETIPNUM_LE:
+    case APLIC_CLRIPNUM:
+    case APLIC_SETIENUM:
+    case APLIC_CLRIENUM:
+    case APLIC_CLRIE_BASE ... APLIC_CLRIE_LAST:
+        /*
+         * Based on the RISC-V AIA spec a read of these registers
+         * always returns zero
+         */
+        *out = 0;
+
+        return 0;
+
+    case APLIC_SETIP_BASE ... APLIC_SETIP_LAST:
+    case APLIC_CLRIP_BASE ... APLIC_CLRIP_LAST:
+    case APLIC_SETIE_BASE ... APLIC_SETIE_LAST:
+        i = regindx_to_irqn(offset & APLIC_SETCLR_OFFSET_MASK);
+        auth_mask = generate_auth_mask(d, i);
+
+        break;
+
+    case APLIC_TARGET_BASE ... APLIC_TARGET_LAST:
+        /*
+         * As target registers start for 1:
+         *  0x3000 genmsi
+         *  0x3004 target[1]
+         *  0x3008 target[2]
+         *   ...
+         *  0x3FFC target[1023]
+         * It is necessary to calculate an interrupt number by substracting
+         * of APLIC_GENMSI instead of APLIC_TARGET_BASE.
+         */
+        i = regindx_to_irqn(offset - APLIC_GENMSI);
+
+        if ( !AUTH_IRQ_BIT(d, i) )
+        {
+            *out = 0;
+
+            return 0;
+        }
+
+        auth_mask = ~0U;
+
+        break;
+
+    default:
+        gdprintk(XENLOG_WARNING, "Unhandled APLIC read at offset %#x\n",
+                 offset);
+
+        domain_crash(vcpu->domain);
+
+        return -EINVAL;
+    }
+
+    *out = aplic_hw_read_reg(offset, auth_mask);
+
+    return 0;
+}
+
+static uint32_t aplic_msi_target_gen(const struct vcpu *target_vcpu,
+                                     uint32_t base_val)
+{
+    const struct imsic_config *imsic = imsic_get_config();
+    unsigned int guest_id = vcpu_guest_file_id(target_vcpu);
+    unsigned long hart_id = cpuid_to_hartid(target_vcpu->processor);
+    unsigned long group_index;
+    unsigned int hhxw = imsic->group_index_bits;
+    unsigned int lhxw = imsic->hart_index_bits;
+    unsigned int hhxs = imsic->group_index_shift - IMSIC_MMIO_PAGE_SHIFT * 2;
+    unsigned long base_ppn = imsic->msi[hart_id].base_addr >> IMSIC_MMIO_PAGE_SHIFT;
+
+    group_index = (base_ppn >> (hhxs + IMSIC_MMIO_PAGE_SHIFT)) & (BIT(hhxw, UL) - 1);
+
+    base_val &= APLIC_TARGET_EIID_MASK;
+    base_val |= guest_id << APLIC_TARGET_GUEST_IDX_SHIFT;
+    base_val |= hart_id << APLIC_TARGET_HART_IDX_SHIFT;
+    base_val |= group_index << (lhxw + APLIC_TARGET_HART_IDX_SHIFT);
+
+    return base_val;
+}
+
+static int cf_check vaplic_emulate_store(const struct vcpu *vcpu,
+                                         unsigned long addr, uint32_t value)
+{
+    int rc = -EINVAL;
+    const struct domain *d = vcpu->domain;
+    unsigned int offset = addr & APLIC_REG_OFFSET_MASK;
+
+    switch ( offset )
+    {
+    case APLIC_SETIP_BASE ... APLIC_SETIP_LAST:
+    case APLIC_CLRIP_BASE ... APLIC_CLRIP_LAST:
+    case APLIC_SETIE_BASE ... APLIC_SETIE_LAST:
+    case APLIC_CLRIE_BASE ... APLIC_CLRIE_LAST:
+    {
+        unsigned int irqn = regindx_to_irqn(offset & APLIC_SETCLR_OFFSET_MASK);
+        value &= generate_auth_mask(d, irqn);
+
+        break;
+    }
+
+    case APLIC_SOURCECFG_BASE ... APLIC_SOURCECFG_LAST:
+        if ( value & APLIC_SOURCECFG_D )
+        {
+            rc = -EOPNOTSUPP;
+
+            dprintk(XENLOG_ERR, "APLIC_SOURCECFG_D isn't supported\n");
+
+            goto fail;
+        }
+
+        /*
+         * As sourcecfg register starts from 1:
+         *   0x0000 domaincfg
+         *   0x0004 sourcecfg[1]
+         *   0x0008 sourcecfg[2]
+         *    ...
+         *   0x0FFC sourcecfg[1023]
+         * It is necessary to calculate an interrupt number by subtracting
+         * of APLIC_DOMAINCFG instead of APLIC_SOURCECFG_BASE.
+         */
+        if ( !AUTH_IRQ_BIT(d, regindx_to_irqn(offset - APLIC_DOMAINCFG)) )
+            /* Interrupt not enabled, ignore it */
+            return 0;
+
+        if ( value > APLIC_SOURCECFG_SM_LEVEL_LOW )
+        {
+            gdprintk(XENLOG_ERR,
+                     "value(%u) is incorrect for sourcecfg register\n", value);
+
+            return 0;
+        }
+
+        break;
+
+    case APLIC_TARGET_BASE ... APLIC_TARGET_LAST:
+    {
+        struct vcpu *target_vcpu = NULL;
+
+        /*
+         * Look at vaplic_emulate_load() for explanation why
+         * APLIC_GENMSI is subtracted.
+         */
+        if ( !AUTH_IRQ_BIT(d, regindx_to_irqn(offset - APLIC_GENMSI)) )
+            /* Interrupt not enabled, ignore it */
+            return 0;
+
+        for ( unsigned int i = 0; i < vcpu->domain->max_vcpus; i++ )
+        {
+            struct vcpu *v = vcpu->domain->vcpu[i];
+
+            if ( v->vcpu_id == (value >> APLIC_TARGET_HART_IDX_SHIFT) )
+            {
+                target_vcpu = v;
+                break;
+            }
+        }
+
+        if ( !target_vcpu )
+        {
+            dprintk(XENLOG_ERR, "Invalid vCPU id in target register\n");
+
+            /* Ignore such writings */
+            return 0;
+        }
+
+        value = aplic_msi_target_gen(target_vcpu, value);
+
+        break;
+    }
+
+    case APLIC_SETIPNUM:
+    case APLIC_SETIPNUM_LE:
+    case APLIC_CLRIPNUM:
+    case APLIC_SETIENUM:
+    case APLIC_CLRIENUM:
+        if ( !value || !AUTH_IRQ_BIT(d, value) )
+            return 0;
+
+        break;
+
+    case APLIC_DOMAINCFG:
+        /*
+         * TODO:
+         * The domaincfg register has this format:
+         * bits 31:24 read-only 0x80
+         * bit 8      IE
+         * bit 7      read-only 0
+         * bit 2      DM (WARL)
+         * bit 0      BE (WARL)
+         *
+         * The most interesting bit for us is IE(Interrupt Enable) bit.
+         * At the moment, at least, Linux doesn't use domaincfg.IE bit to
+         * disable interrupts globally, but if one day someone will use it
+         * then extra actions should be done.
+         */
+
+        printk_once("%s: Ignore writes to domaincfg as it is set by aplic "
+                    " during initialization in Xen\n", __func__);
+
+        return 0;
+
+    default:
+        goto fail;
+    }
+
+    aplic_hw_write_reg(offset, value);
+
+    return 0;
+
+ fail:
+    domain_crash(vcpu->domain,
+                 "Unhandled APLIC write at offset %#x (value %#x)\n", offset,
+                 value);
+
+    return rc;
+}
+
+static bool cf_check vaplic_is_access(const struct vcpu *vcpu,
+                                      unsigned long addr)
+{
+    const struct vaplic *vaplic = to_vaplic(vcpu->domain);
+    paddr_t start = vaplic->regs_start;
+    paddr_t end = vaplic->regs_start + vaplic->regs_size;
+
+    if ( addr & 0x3 )
+    {
+        dprintk(XENLOG_DEBUG,
+                "APLIC MMIO address should be properly aligned\n");
+
+        return false;
+    }
+
+    /* check if it is an APLIC access */
+    if ( start <= addr && addr < end )
+        return true;
+
+    return false;
+}
+
 static int cf_check vcpu_vaplic_init(struct vcpu *v)
 {
     int rc = 0;
@@ -54,19 +331,24 @@ static int cf_check vcpu_vaplic_init(struct vcpu *v)
 
 static int __init cf_check vaplic_make_domu_dt_node(struct kernel_info *kinfo)
 {
+    struct domain *d = kinfo->bd.d;
     int res = 0;
     void *fdt = kinfo->fdt;
     unsigned int msi_parent_phandle;
     char vaplic_name[128];
     paddr_t aplic_addr = GUEST_APLIC_S_BASE;
-    paddr_t aplic_size = APLIC_SIZE(kinfo->bd.d->max_vcpus);
+    paddr_t aplic_size = APLIC_SIZE(d->max_vcpus);
     const __be32 reg[] = {
         cpu_to_be32(aplic_addr >> 32),
         cpu_to_be32(aplic_addr),
         cpu_to_be32(aplic_size >> 32),
         cpu_to_be32(aplic_size),
     };
-    struct vintc *vintc = kinfo->bd.d->arch.vintc;
+    struct vintc *vintc = d->arch.vintc;
+    struct vaplic *vaplic = to_vaplic(d);
+
+    vaplic->regs_start = GUEST_APLIC_S_BASE;
+    vaplic->regs_size = aplic_size;
 
     res = snprintf(vaplic_name, sizeof(vaplic_name), "/soc/aplic@%x",
                    GUEST_APLIC_S_BASE);
@@ -121,6 +403,9 @@ static const struct vintc_init_ops __initdata init_ops = {
 
 static const struct vintc_ops vintc_ops = {
     .vcpu_init = vcpu_vaplic_init,
+    .is_access = vaplic_is_access,
+    .emulate_store = vaplic_emulate_store,
+    .emulate_load = vaplic_emulate_load,
 };
 
 int __init domain_vaplic_init(struct domain *d)
-- 
2.54.0

[PATCH v2 22/26] xen/riscv: implement init_intc_phandle()

[Oleksii Kurochko]

Implement init_intc_phandle() to read phandle of interrupt controller
node and save it in kernel->phandle_intc for the future usage during
creation of guest interrupt controller node.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
 xen/arch/riscv/dom0less-build.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/xen/arch/riscv/dom0less-build.c b/xen/arch/riscv/dom0less-build.c
index 4cc00012aa8d..a1fa51b996a7 100644
--- a/xen/arch/riscv/dom0less-build.c
+++ b/xen/arch/riscv/dom0less-build.c
@@ -4,9 +4,26 @@
 #include <xen/device_tree.h>
 #include <xen/fdt-kernel.h>
 #include <xen/init.h>
+#include <xen/libfdt/libfdt.h>
 
 #include <asm/p2m.h>
 
+int __init init_intc_phandle(struct kernel_info *kinfo, const char *name,
+                             const int node_next, const void *pfdt)
+{
+    if ( dt_node_cmp(name, "intc") == 0 )
+    {
+        uint32_t phandle_intc = fdt_get_phandle(pfdt, node_next);
+
+        if ( phandle_intc != 0 )
+            kinfo->phandle_intc = phandle_intc;
+
+        return 0;
+    }
+
+    return 1;
+}
+
 int __init make_arch_nodes(struct kernel_info *kinfo)
 {
     /* No RISC-V specific nodes need to be made, at the moment. */
-- 
2.54.0

[PATCH v2 23/26] xen/riscv: initialize RCU, scheduler, and system domains in start_xen()

[Oleksii Kurochko]

Wire up the missing early-boot initialization steps in start_xen().

The scheduler must be initialized prior to do_initcalls() because
cpupool_create_pool() is called during initcalls; without it,
BUG_ON(IS_ERR(pool)) is triggered inside cpupool_create_pool().

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - New patch. Several patches were folded into one.
---
 xen/arch/riscv/setup.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c
index 56a0907a855f..c3e98733ebc3 100644
--- a/xen/arch/riscv/setup.c
+++ b/xen/arch/riscv/setup.c
@@ -6,9 +6,12 @@
 #include <xen/compile.h>
 #include <xen/console.h>
 #include <xen/device_tree.h>
+#include <xen/domain.h>
 #include <xen/init.h>
 #include <xen/irq.h>
 #include <xen/mm.h>
+#include <xen/rcupdate.h>
+#include <xen/sched.h>
 #include <xen/serial.h>
 #include <xen/shutdown.h>
 #include <xen/smp.h>
@@ -156,12 +159,21 @@ void __init noreturn start_xen(unsigned long bootcpu_id,
 
     timer_init();
 
+    rcu_init();
+
+    setup_system_domains();
+
     local_irq_enable();
 
     console_init_postirq();
 
     guest_mm_init();
 
+    scheduler_init();
+    set_current(idle_vcpu[0]);
+
+    do_initcalls();
+
     printk("All set up\n");
 
     machine_halt();
-- 
2.54.0

[PATCH v2 24/26] xen/riscv: provide init_vuart()

[Oleksii Kurochko]

For debug purpose is enough to have only print messages from guest what is
now implemented in vsbi_legacy_ecall_handler().

For full guesst console support it will better to have something similar to
[1], thereby there is nothing specific should be done, at least, for now
and init_vuart() is provided to make dom0less code buildable.

[1] https://lore.kernel.org/xen-devel/alpine.DEB.2.22.394.2602041533440.3175371@ubuntu-linux-20-04-desktop/

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in v2:
 - Add Acked-by: Jan Beulich <jbeulich@suse.com>.
---
 xen/arch/riscv/dom0less-build.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/xen/arch/riscv/dom0less-build.c b/xen/arch/riscv/dom0less-build.c
index a1fa51b996a7..d1a51b92936a 100644
--- a/xen/arch/riscv/dom0less-build.c
+++ b/xen/arch/riscv/dom0less-build.c
@@ -8,6 +8,14 @@
 
 #include <asm/p2m.h>
 
+int __init init_vuart(struct domain *d, struct kernel_info *kinfo,
+                      const struct dt_device_node *node)
+{
+    /* Nothing to do at the moment */
+
+    return 0;
+}
+
 int __init init_intc_phandle(struct kernel_info *kinfo, const char *name,
                              const int node_next, const void *pfdt)
 {
-- 
2.54.0

[PATCH v2 25/26] xen/riscv: add initial dom0less infrastructure support

[Oleksii Kurochko]

Enable dom0less support for RISC-V by selecting HAS_DOM0LESS and
providing the minimal architecture hooks required by the common
dom0less infrastructure.

Add stub implementations for architecture-specific helpers used when
building domains from the device tree. These currently perform no
additional work but allow the generic dom0less code to build and run
on RISC-V.

Introduce max_init_domid as a runtime variable rather than a constant
so that it can be updated during dom0less domain creation.

Provide missing helpers and definitions required by the domain
construction code, including domain bitness helpers and the
p2m_set_allocation() prototype.

Additionally define the guest magic memory region in the public
RISC-V interface.

As HAS_DOM0LESS is selected for RISC-V now it could be a compilation
issue if CONFIG_STATIC_MEMORY=y as guest_physmap_add_pages() isn't
yet provided.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - Move declaration of p2m_set_allocation() to p2m-common.h.
 - Add __initdata for max_init_domid and drop initalizer for it.
 - Add CONFIG_STATIC_MEMORY=n to CI's randconfig to avoid
   compilation error because of guest_physmap_add_pages()
   isn't provided.
 - Select HAS_DOMAIN_TYPE for RISC-V and drop things which were
   introduced when HAS_DOMAIN_TYPE doesn't exist.
---
 automation/gitlab-ci/build.yaml           |  1 +
 xen/arch/riscv/Kconfig                    |  2 ++
 xen/arch/riscv/dom0less-build.c           |  6 ++++++
 xen/arch/riscv/domain-build.c             | 13 +++++++++++++
 xen/arch/riscv/include/asm/guest-layout.h |  3 +++
 xen/arch/riscv/include/asm/setup.h        |  4 +++-
 xen/arch/riscv/setup.c                    |  2 ++
 7 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index f05895729147..3c3e7cb356a4 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -805,6 +805,7 @@ debian-13-riscv64-gcc:
       CONFIG_GRANT_TABLE=n
       CONFIG_LIVEPATCH=n
       CONFIG_QEMU_PLATFORM=y
+      CONFIG_STATIC_MEMORY=n
       CONFIG_VM_EVENT=n
       CONFIG_XSM=n
 
diff --git a/xen/arch/riscv/Kconfig b/xen/arch/riscv/Kconfig
index 48520588fe40..d8a348c0cf07 100644
--- a/xen/arch/riscv/Kconfig
+++ b/xen/arch/riscv/Kconfig
@@ -6,6 +6,8 @@ config RISCV
 	select GENERIC_BUG_FRAME
 	select GENERIC_UART_INIT
 	select HAS_DEVICE_TREE_DISCOVERY
+	select HAS_DOM0LESS
+	select HAS_DOMAIN_TYPE
 	select HAS_EX_TABLE
 	select HAS_PMAP
 	select HAS_UBSAN
diff --git a/xen/arch/riscv/dom0less-build.c b/xen/arch/riscv/dom0less-build.c
index d1a51b92936a..b5390175538a 100644
--- a/xen/arch/riscv/dom0less-build.c
+++ b/xen/arch/riscv/dom0less-build.c
@@ -102,3 +102,9 @@ int __init arch_parse_dom0less_node(struct dt_device_node *node,
 
     return 0;
 }
+
+int __init arch_handle_passthrough_prop(struct kernel_info *kinfo,
+                                        struct dt_device_node *node)
+{
+    return 0;
+}
diff --git a/xen/arch/riscv/domain-build.c b/xen/arch/riscv/domain-build.c
index 02b293239142..a70306434fc9 100644
--- a/xen/arch/riscv/domain-build.c
+++ b/xen/arch/riscv/domain-build.c
@@ -158,9 +158,22 @@ int __init make_cpus_node(const struct domain *d, struct kernel_info *kinfo)
     return fdt_end_node(fdt);
 }
 
+int __init construct_hwdom(struct kernel_info *kinfo,
+                           const struct dt_device_node *node)
+{
+    return -EOPNOTSUPP;
+}
+
 int __init make_timer_node(const struct kernel_info *kinfo)
 {
     /* There is no need for timer node for RISC-V. */
 
     return 0;
 }
+
+int __init make_hypervisor_node(struct domain *d,
+                                const struct kernel_info *kinfo,
+                                int addrcells, int sizecells)
+{
+    return -EOPNOTSUPP;
+}
diff --git a/xen/arch/riscv/include/asm/guest-layout.h b/xen/arch/riscv/include/asm/guest-layout.h
index b16ec79c3786..81cc87545054 100644
--- a/xen/arch/riscv/include/asm/guest-layout.h
+++ b/xen/arch/riscv/include/asm/guest-layout.h
@@ -24,4 +24,7 @@
 #define GUEST_RAM_BANK_BASES   { GUEST_RAM0_BASE, GUEST_RAM1_BASE }
 #define GUEST_RAM_BANK_SIZES   { GUEST_RAM0_SIZE, GUEST_RAM1_SIZE }
 
+#define GUEST_MAGIC_BASE  xen_mk_ullong(0x39000000)
+#define GUEST_MAGIC_SIZE  xen_mk_ullong(0x01000000)
+
 #endif /* ASM_RISCV_GUEST_LAYOUT_H */
diff --git a/xen/arch/riscv/include/asm/setup.h b/xen/arch/riscv/include/asm/setup.h
index 9dbd3a8cbef7..416d2301dcde 100644
--- a/xen/arch/riscv/include/asm/setup.h
+++ b/xen/arch/riscv/include/asm/setup.h
@@ -5,11 +5,13 @@
 
 #include <xen/types.h>
 
+#include <public/xen.h>
+
 struct domain;
 struct dt_device_node;
 struct rangeset;
 
-#define max_init_domid (0)
+extern domid_t max_init_domid;
 
 void setup_mm(void);
 
diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c
index c3e98733ebc3..b8fa004cbf25 100644
--- a/xen/arch/riscv/setup.c
+++ b/xen/arch/riscv/setup.c
@@ -33,6 +33,8 @@
 #include <asm/traps.h>
 #include <asm/vsbi.h>
 
+domid_t __initdata max_init_domid;
+
 /* Xen stack for bringing up the first CPU. */
 unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
     __aligned(STACK_SIZE);
-- 
2.54.0

[PATCH v2 26/26] xen/riscv: manage IRQ_DISABLED flag in APLIC irq enable/disable callbacks

[Oleksii Kurochko]

desc->status is only set once during setup_irq(), but interrupts can be
enabled/disabled at runtime, so update it in the corresponding callbacks.

wmb() in aplic_irq_enable() ensures do_IRQ(), which can fire immediately
after the interrupt is enabled, sees the updated desc->status. No rmb() is
needed on the do_IRQ() side because desc->status is read under a spinlock,
which implies an acquire barrier.

No barrier is needed in aplic_irq_disable() because the hardware disables
the interrupt before the status is updated, so do_IRQ() cannot fire, and
spin_unlock() makes the updated value visible.

Fixes: d4676a1398bc5 ("xen/riscv: implementation of aplic and imsic operations")
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v2:
 - New patch.
---
 xen/arch/riscv/aplic.c | 5 +++++
 xen/arch/riscv/irq.c   | 3 ---
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/xen/arch/riscv/aplic.c b/xen/arch/riscv/aplic.c
index 1976733dfbaa..7933a00772a8 100644
--- a/xen/arch/riscv/aplic.c
+++ b/xen/arch/riscv/aplic.c
@@ -161,6 +161,9 @@ static void cf_check aplic_irq_enable(struct irq_desc *desc)
 
     spin_lock(&aplic.lock);
 
+    desc->status &= ~IRQ_DISABLED;
+    wmb();
+
     /* Enable interrupt in IMSIC */
     imsic_irq_enable(desc->irq);
 
@@ -189,6 +192,8 @@ static void cf_check aplic_irq_disable(struct irq_desc *desc)
     /* Disable interrupt in IMSIC */
     imsic_irq_disable(desc->irq);
 
+    desc->status |= IRQ_DISABLED;
+
     spin_unlock(&aplic.lock);
 }
 
diff --git a/xen/arch/riscv/irq.c b/xen/arch/riscv/irq.c
index 4481ae23aa8e..f9c10a7c9faf 100644
--- a/xen/arch/riscv/irq.c
+++ b/xen/arch/riscv/irq.c
@@ -145,9 +145,6 @@ int setup_irq(unsigned int irq, unsigned int irqflags, struct irqaction *new)
         desc->handler->set_affinity(desc, cpumask_of(smp_processor_id()));
 
         desc->handler->startup(desc);
-
-        /* Enable irq */
-        desc->status &= ~IRQ_DISABLED;
     }
 
  err:
-- 
2.54.0

Re: [PATCH v2 02/26] xen/riscv: Implement ARCH_PAGING_MEMPOOL

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> The p2m_freelist is used to allocate pages for the P2M. To initialize
> this list, domain_p2m_set_allocation() may be called from construct_domU()
> in the common Dom0less code, so RISC-V provides an implementation and
> enables CONFIG_ARCH_PAGING_MEMPOOL unconditionally.
> 
> Additionally, implement arch_{set,get}_paging_mempool_size(). They are
> not directly used yet, but are required to support the
> XEN_DOMCTL_{get,set}_paging_mempool_size hypercalls.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
with ...

> --- a/xen/arch/riscv/p2m.c
> +++ b/xen/arch/riscv/p2m.c
> @@ -1606,3 +1606,27 @@ struct page_info *get_page_from_gfn(struct domain *d, unsigned long gfn,
>  
>      return p2m_get_page_from_gfn(p2m_get_hostp2m(d), _gfn(gfn), t);
>  }
> +
> +int arch_set_paging_mempool_size(struct domain *d, uint64_t size)
> +{
> +    unsigned long pages = PFN_DOWN(size);
> +    int rc;
> +
> +    /* Non page-sized request or 32-bit overflow? */
> +    if ( pfn_to_paddr(pages) != size )
> +        return -EINVAL;
> +
> +    spin_lock(&d->arch.paging.lock);
> +    rc = p2m_set_allocation(d, pages, true);
> +    spin_unlock(&d->arch.paging.lock);
> +
> +    return rc;
> +}
> +
> +/* Return the size of the pool, in bytes. */
> +int arch_get_paging_mempool_size(struct domain *d, uint64_t *size)
> +{
> +    *size =  pfn_to_paddr(ACCESS_ONCE(d->arch.paging.total_pages));

... the stray blank dropped from here. Will take care of this when committing.

Jan

Re: [PATCH v2 03/26] xen/riscv: Implement construct_domain()

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> Implement construct_domain() function for RISC-V, which performs initial setup
> for the domain's first vCPU, loads the kernel, initrd, and device tree,
> and sets up guest CPU registers for boot.
> 
> It also creates additional vCPUs up to max_vcpus and assigns the device tree
> address and boot cpuid in registers.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
with ...

> --- /dev/null
> +++ b/xen/arch/riscv/domain-build.c
> @@ -0,0 +1,52 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +
> +#include <xen/fdt-domain-build.h>
> +#include <xen/fdt-kernel.h>
> +#include <xen/init.h>
> +#include <xen/sched.h>
> +
> +#include <asm/current.h>
> +#include <asm/guest_access.h>
> +
> +int __init construct_domain(struct domain *d, struct kernel_info *kinfo)
> +{
> +    struct vcpu *v = d->vcpu[0];
> +    struct cpu_user_regs *regs = vcpu_guest_cpu_user_regs(v);
> +
> +    BUG_ON(v->is_initialised);
> +
> +    /*
> +     * At the moment *_load() don't return value and will just panic()
> +     * inside.
> +     * TODO: it will be good to change that.
> +     */
> +    kernel_load(kinfo);
> +    initrd_load(kinfo, copy_to_guest_phys);
> +    dtb_load(kinfo, copy_to_guest_phys);
> +
> +    regs->sepc = kinfo->entry;
> +
> +    /* Guest boot cpuid = 0 */
> +    regs->a0 = 0;
> +    regs->a1 = kinfo->dtb_paddr;
> +
> +    for ( unsigned int i = 1; i < d->max_vcpus; i++ )
> +    {
> +        const struct vcpu *tmp_v = vcpu_create(d, i);
> +
> +        if ( !tmp_v )
> +        {
> +            printk("Failed to allocate %pd v%d\n", d, i);

... %u used here and ...

> +            break;
> +        }
> +
> +        dprintk(XENLOG_INFO, "Created vcpu %pv\n", tmp_v);

... this line preferably dropped. Can do when committing, provided you agree.

Jan

Re: [PATCH v2 00/26] Introduce enablemenant of dom0less

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> This patch series reprensent a bunch of patches necessary to enable common part
> of Dom0less.
> The stuff necessary to start/launch domains will be introduced separately.
> 
> This patch series is based on [1], but a lot of patch could go even without
> it.

What do I do with this kind of statement? This still leaves it to the committer
to figure out which ones could go in earlier. What helps in this specific case
it that ...

> [1] https://lore.kernel.org/xen-devel/cover.1778140240.git.oleksii.kurochko@gmail.com/T/#t
... the remainder of this looks to be ready to go in now (and I intend to commit
that tomorrow).

Jan

Re: [PATCH v2 04/26] xen/riscv: implement prerequisites for domain_create()

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> --- a/xen/arch/riscv/domain.c
> +++ b/xen/arch/riscv/domain.c
> @@ -289,6 +289,31 @@ void sync_vcpu_execstate(struct vcpu *v)
>      /* Nothing to do -- no lazy switching */
>  }
>  
> +int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
> +{
> +    return 0;
> +}
> +
> +int arch_domain_create(struct domain *d,
> +                       struct xen_domctl_createdomain *config,
> +                       unsigned int flags)
> +{
> +    int rc = 0;
> +
> +    if ( is_idle_domain(d) )
> +        return 0;
> +
> +    if ( (rc = p2m_init(d, config)) != 0)
> +        goto fail;
> +
> +    return rc;
> +
> + fail:
> +    d->is_dying = DOMDYING_dead;

domain_create() does this as well. Is it really needed here?

> +    arch_domain_destroy(d);

This continues to be a stub, i.e. upon encountering any kind of error one
would hit the BUG_ON() there.

Jan

Re: [PATCH v2 05/26] xen/riscv: introduce guest riscv,isa string

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> @@ -480,6 +488,53 @@ bool riscv_isa_extension_available(const unsigned long *isa_bitmap,
>      return test_bit(id, isa_bitmap);
>  }
>  
> +int init_guest_isa(struct domain *d)
> +{
> +    char *buf = d->arch.guest_isa_str;
> +    size_t len = sizeof(d->arch.guest_isa_str);

Seeing these uses: Is the "guest" prefix really of much use here?

> +    bitmap_andnot(d->arch.guest_isa, riscv_isa, guest_unsupp,
> +                  RISCV_ISA_EXT_MAX);

Same question here, clearly.

> +#if defined(CONFIG_RISCV_32)
> +    if ( snprintf(buf, len, "rv32") >= len )
> +        return -ENOBUFS;
> +#elif defined(CONFIG_RISCV_64)
> +    if ( snprintf(buf, len, "rv64") >= len )
> +        return -ENOBUFS;
> +#else
> +#   error "Unsupported RISC-V bitness"
> +#endif
> +
> +    for ( unsigned int i = 0; i < ARRAY_SIZE(riscv_isa_ext); i++ )
> +    {
> +        const struct riscv_isa_ext_data *ext = &riscv_isa_ext[i];
> +
> +        if ( !riscv_isa_extension_available(d->arch.guest_isa, ext->id) )
> +            continue;
> +
> +        if ( ext->id >= RISCV_ISA_EXT_BASE && strlcat(buf, "_", len) >= len )
> +            return -ENOBUFS;
> +
> +        if ( strlcat(buf, ext->name, len) >= len )
> +            return -ENOBUFS;
> +    }
> +
> +    return 0;
> +}
> +
> +static void __init init_guest_unsupp(void)
> +{
> +    set_bit(RISCV_ISA_EXT_f, guest_unsupp);
> +    set_bit(RISCV_ISA_EXT_d, guest_unsupp);
> +    set_bit(RISCV_ISA_EXT_q, guest_unsupp);
> +    set_bit(RISCV_ISA_EXT_v, guest_unsupp);
> +    set_bit(RISCV_ISA_EXT_h, guest_unsupp);
> +    set_bit(RISCV_ISA_EXT_sstc, guest_unsupp);
> +    set_bit(RISCV_ISA_EXT_svade, guest_unsupp);
> +    set_bit(RISCV_ISA_EXT_svpbmt, guest_unsupp);
> +}

These don't need to be atomic, do they? I.e. __set_bit() would suffice.

> --- a/xen/arch/riscv/include/asm/cpufeature.h
> +++ b/xen/arch/riscv/include/asm/cpufeature.h
> @@ -17,6 +17,8 @@
>   */
>  #define RISCV_ISA_EXT_BASE  26
>  
> +#define RISCV_GUEST_ISA_STR_MAX 256

This looks like it won't be good for very long, seeing how long ISA strings can
get. I wonder anyway whether ...

> @@ -94,6 +95,9 @@ struct arch_domain {
>      struct p2m_domain p2m;
>  
>      struct paging_domain paging;
> +
> +    DECLARE_BITMAP(guest_isa, RISCV_ISA_EXT_MAX);
> +    char guest_isa_str[RISCV_GUEST_ISA_STR_MAX];

... a compile-time sized buffer is suitable here. Can't you allocate a buffer
just large enough to hold the string?

Jan

Re: [PATCH v2 06/26] xen/riscv: implement make_cpus_node()

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> @@ -50,3 +52,107 @@ int __init construct_domain(struct domain *d, struct kernel_info *kinfo)
>  
>      return 0;
>  }
> +int __init make_cpus_node(const struct domain *d, struct kernel_info *kinfo)

Nit: Blank line above here, please.

> +{
> +    int res;
> +    const struct dt_device_node *cpus = dt_find_node_by_path("/cpus");
> +    unsigned int cpu;
> +    uint32_t timebase_frequency;
> +    bool frequency_valid;
> +    void *fdt = kinfo->fdt;
> +
> +    dt_dprintk("Create cpus node\n");
> +
> +    if ( !cpus )
> +    {
> +        dprintk(XENLOG_ERR, "Missing /cpus node in the device tree?\n");
> +        return -ENOENT;
> +    }
> +
> +    frequency_valid = dt_property_read_u32(cpus, "timebase-frequency",
> +                                           &timebase_frequency);
> +
> +    res = fdt_begin_node(fdt, "cpus");
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property_cell(fdt, "#address-cells", 1);
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property_cell(fdt, "#size-cells", 0);
> +    if ( res )
> +        return res;
> +
> +    if ( frequency_valid )
> +        res = fdt_property_cell(fdt, "timebase-frequency", timebase_frequency);
> +
> +    for ( cpu = 0; cpu < d->max_vcpus; cpu++ )

Limit cpu's scope to this loop?

> +    {
> +        char buf[64];
> +        uint32_t reg = cpu_to_fdt32(cpu);

Isn't this a byte-order adjustment? If so, how come ...

> +        snprintf(buf, sizeof(buf), "cpu@%u", cpu);

... the result is passed to an entirely non-FDT function? (Most pre-existing uses
of the function that I can spot store something in memory, i.e. adjusting byte-
order makes sense there.)

> +        res = fdt_begin_node(fdt, buf);
> +        if ( res )
> +            return res;
> +
> +        res = fdt_property(fdt, "reg", &reg, sizeof(reg));
> +        if ( res )
> +            return res;
> +
> +        res = fdt_property_string(fdt, "status", "okay");
> +        if ( res )
> +            return res;
> +
> +        res = fdt_property_string(fdt, "compatible", "riscv");
> +        if ( res )
> +            return res;
> +
> +        BUILD_BUG_ON((sizeof("riscv,") +
> +                      sizeof_field(struct gstage_mode_desc, name)) >= sizeof(buf));
> +        snprintf(buf, sizeof(buf), "riscv,%s", max_gstage_mode->name);
> +        res = fdt_property_string(fdt, "mmu-type", buf);
> +        if ( res )
> +            return res;
> +
> +        res = fdt_property_string(fdt, "riscv,isa", d->arch.guest_isa_str);
> +        if ( res )
> +            return res;
> +
> +        res = fdt_property_string(fdt, "device_type", "cpu");
> +        if ( res )
> +            return res;
> +
> +        res = fdt_begin_node(fdt, "interrupt-controller");
> +        if ( res )
> +            return res;
> +
> +        res = fdt_property_string(fdt, "compatible", "riscv,cpu-intc");
> +        if ( res )
> +            return res;
> +
> +        res = fdt_property_cell(fdt, "#interrupt-cells", 1);
> +        if ( res )
> +            return res;
> +
> +        res = fdt_property(fdt, "interrupt-controller", NULL, 0);
> +        if ( res )
> +            return res;
> +
> +        res = fdt_property_u32(fdt, "phandle", alloc_phandle(kinfo));
> +        if ( res )
> +            return res;
> +
> +        /* end of interrupt-controller */

Nit: Comment style. Also such a comment pretty clearly calls for a counterpart
at the start.

> +        res = fdt_end_node(fdt);
> +        if ( res )
> +            return res;
> +
> +        res = fdt_end_node(fdt);
> +        if ( res )
> +            return res;
> +    }
> +
> +    return fdt_end_node(fdt);
> +}

Re: [PATCH v2 00/26] Introduce enablemenant of dom0less

[Oleksii Kurochko]

On 5/18/26 5:38 PM, Jan Beulich wrote:
> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>> This patch series reprensent a bunch of patches necessary to enable common part
>> of Dom0less.
>> The stuff necessary to start/launch domains will be introduced separately.
>>
>> This patch series is based on [1], but a lot of patch could go even without
>> it.
> 
> What do I do with this kind of statement? This still leaves it to the committer
> to figure out which ones could go in earlier. What helps in this specific case
> it that ...
> 
>> [1] https://lore.kernel.org/xen-devel/cover.1778140240.git.oleksii.kurochko@gmail.com/T/#t
> ... the remainder of this looks to be ready to go in now (and I intend to commit
> that tomorrow).

I will be more specific in the future and specify which patches can go 
in without waiting for the dependency patch series to be fully merged.


~ Oleksii

Re: [PATCH v2 02/26] xen/riscv: Implement ARCH_PAGING_MEMPOOL

[Oleksii Kurochko]

On 5/18/26 5:13 PM, Jan Beulich wrote:
> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>> The p2m_freelist is used to allocate pages for the P2M. To initialize
>> this list, domain_p2m_set_allocation() may be called from construct_domU()
>> in the common Dom0less code, so RISC-V provides an implementation and
>> enables CONFIG_ARCH_PAGING_MEMPOOL unconditionally.
>>
>> Additionally, implement arch_{set,get}_paging_mempool_size(). They are
>> not directly used yet, but are required to support the
>> XEN_DOMCTL_{get,set}_paging_mempool_size hypercalls.
>>
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> 
> Acked-by: Jan Beulich <jbeulich@suse.com>
> with ...
> 
>> --- a/xen/arch/riscv/p2m.c
>> +++ b/xen/arch/riscv/p2m.c
>> @@ -1606,3 +1606,27 @@ struct page_info *get_page_from_gfn(struct domain *d, unsigned long gfn,
>>   
>>       return p2m_get_page_from_gfn(p2m_get_hostp2m(d), _gfn(gfn), t);
>>   }
>> +
>> +int arch_set_paging_mempool_size(struct domain *d, uint64_t size)
>> +{
>> +    unsigned long pages = PFN_DOWN(size);
>> +    int rc;
>> +
>> +    /* Non page-sized request or 32-bit overflow? */
>> +    if ( pfn_to_paddr(pages) != size )
>> +        return -EINVAL;
>> +
>> +    spin_lock(&d->arch.paging.lock);
>> +    rc = p2m_set_allocation(d, pages, true);
>> +    spin_unlock(&d->arch.paging.lock);
>> +
>> +    return rc;
>> +}
>> +
>> +/* Return the size of the pool, in bytes. */
>> +int arch_get_paging_mempool_size(struct domain *d, uint64_t *size)
>> +{
>> +    *size =  pfn_to_paddr(ACCESS_ONCE(d->arch.paging.total_pages));
> 
> ... the stray blank dropped from here. Will take care of this when committing.

I would be happy with that.

Thanks.

~ Oleksii

Re: [PATCH v2 03/26] xen/riscv: Implement construct_domain()

[Oleksii Kurochko]

On 5/18/26 5:33 PM, Jan Beulich wrote:
> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>> Implement construct_domain() function for RISC-V, which performs initial setup
>> for the domain's first vCPU, loads the kernel, initrd, and device tree,
>> and sets up guest CPU registers for boot.
>>
>> It also creates additional vCPUs up to max_vcpus and assigns the device tree
>> address and boot cpuid in registers.
>>
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> 
> Acked-by: Jan Beulich <jbeulich@suse.com>
> with ...
> 
>> --- /dev/null
>> +++ b/xen/arch/riscv/domain-build.c
>> @@ -0,0 +1,52 @@
>> +/* SPDX-License-Identifier: GPL-2.0-or-later */
>> +
>> +#include <xen/fdt-domain-build.h>
>> +#include <xen/fdt-kernel.h>
>> +#include <xen/init.h>
>> +#include <xen/sched.h>
>> +
>> +#include <asm/current.h>
>> +#include <asm/guest_access.h>
>> +
>> +int __init construct_domain(struct domain *d, struct kernel_info *kinfo)
>> +{
>> +    struct vcpu *v = d->vcpu[0];
>> +    struct cpu_user_regs *regs = vcpu_guest_cpu_user_regs(v);
>> +
>> +    BUG_ON(v->is_initialised);
>> +
>> +    /*
>> +     * At the moment *_load() don't return value and will just panic()
>> +     * inside.
>> +     * TODO: it will be good to change that.
>> +     */
>> +    kernel_load(kinfo);
>> +    initrd_load(kinfo, copy_to_guest_phys);
>> +    dtb_load(kinfo, copy_to_guest_phys);
>> +
>> +    regs->sepc = kinfo->entry;
>> +
>> +    /* Guest boot cpuid = 0 */
>> +    regs->a0 = 0;
>> +    regs->a1 = kinfo->dtb_paddr;
>> +
>> +    for ( unsigned int i = 1; i < d->max_vcpus; i++ )
>> +    {
>> +        const struct vcpu *tmp_v = vcpu_create(d, i);
>> +
>> +        if ( !tmp_v )
>> +        {
>> +            printk("Failed to allocate %pd v%d\n", d, i);
> 
> ... %u used here and ...
> 
>> +            break;
>> +        }
>> +
>> +        dprintk(XENLOG_INFO, "Created vcpu %pv\n", tmp_v);
> 
> ... this line preferably dropped. Can do when committing, provided you agree.

I agree with these changes.

Thanks a lot.

~ Oleksii

Re: [PATCH v2 04/26] xen/riscv: implement prerequisites for domain_create()

[Oleksii Kurochko]

On 5/18/26 5:43 PM, Jan Beulich wrote:
> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>> --- a/xen/arch/riscv/domain.c
>> +++ b/xen/arch/riscv/domain.c
>> @@ -289,6 +289,31 @@ void sync_vcpu_execstate(struct vcpu *v)
>>       /* Nothing to do -- no lazy switching */
>>   }
>>   
>> +int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>> +{
>> +    return 0;
>> +}
>> +
>> +int arch_domain_create(struct domain *d,
>> +                       struct xen_domctl_createdomain *config,
>> +                       unsigned int flags)
>> +{
>> +    int rc = 0;
>> +
>> +    if ( is_idle_domain(d) )
>> +        return 0;
>> +
>> +    if ( (rc = p2m_init(d, config)) != 0)
>> +        goto fail;
>> +
>> +    return rc;
>> +
>> + fail:
>> +    d->is_dying = DOMDYING_dead;
> 
> domain_create() does this as well. Is it really needed here?

Considering that domain wasn't created and thereby scheduled that 
nothing will use d->is_dying and so it could be dropped or moved to
arch_domain_destroy().

> 
>> +    arch_domain_destroy(d);
> 
> This continues to be a stub, i.e. upon encountering any kind of error one
> would hit the BUG_ON() there.

I think that for current stage of development it is fine.

Would it be better change BUG_ON() to printk()? And add proper 
implementation a little bit later?

~ Oleksii

Re: [PATCH v2 04/26] xen/riscv: implement prerequisites for domain_create()

[Jan Beulich]

On 19.05.2026 13:33, Oleksii Kurochko wrote:
> On 5/18/26 5:43 PM, Jan Beulich wrote:
>> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>>> --- a/xen/arch/riscv/domain.c
>>> +++ b/xen/arch/riscv/domain.c
>>> @@ -289,6 +289,31 @@ void sync_vcpu_execstate(struct vcpu *v)
>>>       /* Nothing to do -- no lazy switching */
>>>   }
>>>   
>>> +int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
>>> +{
>>> +    return 0;
>>> +}
>>> +
>>> +int arch_domain_create(struct domain *d,
>>> +                       struct xen_domctl_createdomain *config,
>>> +                       unsigned int flags)
>>> +{
>>> +    int rc = 0;
>>> +
>>> +    if ( is_idle_domain(d) )
>>> +        return 0;
>>> +
>>> +    if ( (rc = p2m_init(d, config)) != 0)
>>> +        goto fail;
>>> +
>>> +    return rc;
>>> +
>>> + fail:
>>> +    d->is_dying = DOMDYING_dead;
>>
>> domain_create() does this as well. Is it really needed here?
> 
> Considering that domain wasn't created and thereby scheduled that 
> nothing will use d->is_dying and so it could be dropped or moved to
> arch_domain_destroy().
> 
>>
>>> +    arch_domain_destroy(d);
>>
>> This continues to be a stub, i.e. upon encountering any kind of error one
>> would hit the BUG_ON() there.
> 
> I think that for current stage of development it is fine.
> 
> Would it be better change BUG_ON() to printk()? And add proper 
> implementation a little bit later?

That may be acceptable, yes.

Jan

Re: [PATCH v2 05/26] xen/riscv: introduce guest riscv,isa string

[Oleksii Kurochko]

On 5/18/26 5:51 PM, Jan Beulich wrote:
> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>> @@ -480,6 +488,53 @@ bool riscv_isa_extension_available(const unsigned long *isa_bitmap,
>>       return test_bit(id, isa_bitmap);
>>   }
>>   
>> +int init_guest_isa(struct domain *d)
>> +{
>> +    char *buf = d->arch.guest_isa_str;
>> +    size_t len = sizeof(d->arch.guest_isa_str);
> 
> Seeing these uses: Is the "guest" prefix really of much use here?
> 
>> +    bitmap_andnot(d->arch.guest_isa, riscv_isa, guest_unsupp,
>> +                  RISCV_ISA_EXT_MAX);
> 
> Same question here, clearly.

I will drop "guest" prefix for "d->arch.guest_isa_str".


> 
>> +#if defined(CONFIG_RISCV_32)
>> +    if ( snprintf(buf, len, "rv32") >= len )
>> +        return -ENOBUFS;
>> +#elif defined(CONFIG_RISCV_64)
>> +    if ( snprintf(buf, len, "rv64") >= len )
>> +        return -ENOBUFS;
>> +#else
>> +#   error "Unsupported RISC-V bitness"
>> +#endif
>> +
>> +    for ( unsigned int i = 0; i < ARRAY_SIZE(riscv_isa_ext); i++ )
>> +    {
>> +        const struct riscv_isa_ext_data *ext = &riscv_isa_ext[i];
>> +
>> +        if ( !riscv_isa_extension_available(d->arch.guest_isa, ext->id) )
>> +            continue;
>> +
>> +        if ( ext->id >= RISCV_ISA_EXT_BASE && strlcat(buf, "_", len) >= len )
>> +            return -ENOBUFS;
>> +
>> +        if ( strlcat(buf, ext->name, len) >= len )
>> +            return -ENOBUFS;
>> +    }
>> +
>> +    return 0;
>> +}
>> +
>> +static void __init init_guest_unsupp(void)
>> +{
>> +    set_bit(RISCV_ISA_EXT_f, guest_unsupp);
>> +    set_bit(RISCV_ISA_EXT_d, guest_unsupp);
>> +    set_bit(RISCV_ISA_EXT_q, guest_unsupp);
>> +    set_bit(RISCV_ISA_EXT_v, guest_unsupp);
>> +    set_bit(RISCV_ISA_EXT_h, guest_unsupp);
>> +    set_bit(RISCV_ISA_EXT_sstc, guest_unsupp);
>> +    set_bit(RISCV_ISA_EXT_svade, guest_unsupp);
>> +    set_bit(RISCV_ISA_EXT_svpbmt, guest_unsupp);
>> +}
> 
> These don't need to be atomic, do they? I.e. __set_bit() would suffice.

Agree, __set_bit() would be enough.


> 
>> --- a/xen/arch/riscv/include/asm/cpufeature.h
>> +++ b/xen/arch/riscv/include/asm/cpufeature.h
>> @@ -17,6 +17,8 @@
>>    */
>>   #define RISCV_ISA_EXT_BASE  26
>>   
>> +#define RISCV_GUEST_ISA_STR_MAX 256
> 
> This looks like it won't be good for very long, seeing how long ISA strings can
> get. I wonder anyway whether ...
> 
>> @@ -94,6 +95,9 @@ struct arch_domain {
>>       struct p2m_domain p2m;
>>   
>>       struct paging_domain paging;
>> +
>> +    DECLARE_BITMAP(guest_isa, RISCV_ISA_EXT_MAX);
>> +    char guest_isa_str[RISCV_GUEST_ISA_STR_MAX];
> 
> ... a compile-time sized buffer is suitable here. Can't you allocate a buffer
> just large enough to hold the string?

It could be allocated dynamically.

Does it make sense to evaluate in run-time what should be a buffer size? 
For this case I can't find analogue of realloc() in Xen. Or it would be 
fine just to take something bigger as a const (lets say 2048) and use it 
for dynamic allocation?

Thanks.

~ Oleksii

Re: [PATCH v2 05/26] xen/riscv: introduce guest riscv,isa string

[Jan Beulich]

On 19.05.2026 13:59, Oleksii Kurochko wrote:
> On 5/18/26 5:51 PM, Jan Beulich wrote:
>> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>>> --- a/xen/arch/riscv/include/asm/cpufeature.h
>>> +++ b/xen/arch/riscv/include/asm/cpufeature.h
>>> @@ -17,6 +17,8 @@
>>>    */
>>>   #define RISCV_ISA_EXT_BASE  26
>>>   
>>> +#define RISCV_GUEST_ISA_STR_MAX 256
>>
>> This looks like it won't be good for very long, seeing how long ISA strings can
>> get. I wonder anyway whether ...
>>
>>> @@ -94,6 +95,9 @@ struct arch_domain {
>>>       struct p2m_domain p2m;
>>>   
>>>       struct paging_domain paging;
>>> +
>>> +    DECLARE_BITMAP(guest_isa, RISCV_ISA_EXT_MAX);
>>> +    char guest_isa_str[RISCV_GUEST_ISA_STR_MAX];
>>
>> ... a compile-time sized buffer is suitable here. Can't you allocate a buffer
>> just large enough to hold the string?
> 
> It could be allocated dynamically.
> 
> Does it make sense to evaluate in run-time what should be a buffer size? 
> For this case I can't find analogue of realloc() in Xen.

Hmm, I see xrealloc_array(), and surely we could gain xvrealloc_array()
which we'll need anyway once xrealloc_array() uses get converted. (I also
see x{,v}realloc_flex_struct(), but that's of no use here as it looks.)

> Or it would be 
> fine just to take something bigger as a const (lets say 2048) and use it 
> for dynamic allocation?

I'd rather not. Can't you determine how much space the string is going to
occupy?

Jan

Re: [PATCH v2 05/26] xen/riscv: introduce guest riscv,isa string

[Oleksii Kurochko]

On 5/19/26 2:12 PM, Jan Beulich wrote:
> On 19.05.2026 13:59, Oleksii Kurochko wrote:
>> On 5/18/26 5:51 PM, Jan Beulich wrote:
>>> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>>>> --- a/xen/arch/riscv/include/asm/cpufeature.h
>>>> +++ b/xen/arch/riscv/include/asm/cpufeature.h
>>>> @@ -17,6 +17,8 @@
>>>>     */
>>>>    #define RISCV_ISA_EXT_BASE  26
>>>>    
>>>> +#define RISCV_GUEST_ISA_STR_MAX 256
>>>
>>> This looks like it won't be good for very long, seeing how long ISA strings can
>>> get. I wonder anyway whether ...
>>>
>>>> @@ -94,6 +95,9 @@ struct arch_domain {
>>>>        struct p2m_domain p2m;
>>>>    
>>>>        struct paging_domain paging;
>>>> +
>>>> +    DECLARE_BITMAP(guest_isa, RISCV_ISA_EXT_MAX);
>>>> +    char guest_isa_str[RISCV_GUEST_ISA_STR_MAX];
>>>
>>> ... a compile-time sized buffer is suitable here. Can't you allocate a buffer
>>> just large enough to hold the string?
>>
>> It could be allocated dynamically.
>>
>> Does it make sense to evaluate in run-time what should be a buffer size?
>> For this case I can't find analogue of realloc() in Xen.
> 
> Hmm, I see xrealloc_array(), and surely we could gain xvrealloc_array()
> which we'll need anyway once xrealloc_array() uses get converted. (I also
> see x{,v}realloc_flex_struct(), but that's of no use here as it looks.)

Oh, I missed to turned off "full match" during search...

> 
>> Or it would be
>> fine just to take something bigger as a const (lets say 2048) and use it
>> for dynamic allocation?
> 
> I'd rather not. Can't you determine how much space the string is going to
> occupy?

I thought about two options as alternatives:

1. Take as a length host RISC-V ISA string but theoretically we can 
emulate some extensions which aren't mentioned in host RISC-V ISA string 
so it could be longer. So not a good option.

2. Having two walks in init_guest_isa():
    Introduce the following function:
static size_t guest_isa_str_len(const unsigned long *isa_bitmap)
{
     size_t len = 4; /* rvX prefix */

     for ( unsigned int i = 0; i < ARRAY_SIZE(riscv_isa_ext); i++ )
     {
         const struct riscv_isa_ext_data *ext = &riscv_isa_ext[i];

         if ( !riscv_isa_extension_available(isa_bitmap, ext->id) )
             continue;

         if ( ext->id >= RISCV_ISA_EXT_BASE )
             len++; /* '_' separator */

         len += strlen(ext->name);
     }

     return len + 1; /* NUL terminator */
}

   and then:

int init_guest_isa(struct domain *d)
{
     bitmap_andnot(d->arch.guest_isa, riscv_isa, guest_unsupp,
                   RISCV_ISA_EXT_MAX);

     size_t len = guest_isa_str_len(d->arch.guest_isa);
     d->arch.guest_isa_str = xzalloc_array(char, len);
     if ( !d->arch.guest_isa_str )
         return -ENOMEM;

     /* ... existing snprintf + strlcat loop unchanged ... */
}

If approach 2 is a good one I can follow it.

~ Oleksii

Re: [PATCH v2 06/26] xen/riscv: implement make_cpus_node()

[Oleksii Kurochko]

On 5/18/26 6:00 PM, Jan Beulich wrote:
> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>> @@ -50,3 +52,107 @@ int __init construct_domain(struct domain *d, struct kernel_info *kinfo)
>>   
>>       return 0;
>>   }
>> +int __init make_cpus_node(const struct domain *d, struct kernel_info *kinfo)
> 
> Nit: Blank line above here, please.
> 
>> +{
>> +    int res;
>> +    const struct dt_device_node *cpus = dt_find_node_by_path("/cpus");
>> +    unsigned int cpu;
>> +    uint32_t timebase_frequency;
>> +    bool frequency_valid;
>> +    void *fdt = kinfo->fdt;
>> +
>> +    dt_dprintk("Create cpus node\n");
>> +
>> +    if ( !cpus )
>> +    {
>> +        dprintk(XENLOG_ERR, "Missing /cpus node in the device tree?\n");
>> +        return -ENOENT;
>> +    }
>> +
>> +    frequency_valid = dt_property_read_u32(cpus, "timebase-frequency",
>> +                                           &timebase_frequency);
>> +
>> +    res = fdt_begin_node(fdt, "cpus");
>> +    if ( res )
>> +        return res;
>> +
>> +    res = fdt_property_cell(fdt, "#address-cells", 1);
>> +    if ( res )
>> +        return res;
>> +
>> +    res = fdt_property_cell(fdt, "#size-cells", 0);
>> +    if ( res )
>> +        return res;
>> +
>> +    if ( frequency_valid )
>> +        res = fdt_property_cell(fdt, "timebase-frequency", timebase_frequency);
>> +
>> +    for ( cpu = 0; cpu < d->max_vcpus; cpu++ )
> 
> Limit cpu's scope to this loop?

I will remove 'unsigned int cpu' from the function-level declarations 
and moved it into the for loop.

> 
>> +    {
>> +        char buf[64];
>> +        uint32_t reg = cpu_to_fdt32(cpu);
> 
> Isn't this a byte-order adjustment? If so, how come ...
> 
>> +        snprintf(buf, sizeof(buf), "cpu@%u", cpu);
> 
> ... the result is passed to an entirely non-FDT function? (Most pre-existing uses
> of the function that I can spot store something in memory, i.e. adjusting byte-
> order makes sense there.)

But here pure cpu is used instead of reg variable and reg variable is 
used here ...

> 
>> +        res = fdt_begin_node(fdt, buf);
>> +        if ( res )
>> +            return res;
>> +
>> +        res = fdt_property(fdt, "reg", &reg, sizeof(reg));
>> +        if ( res )
>> +            return res;
... but it we could drop it and use just:
   res = fdt_property_cell(fdt, "reg", cpu);

fdt_property_cell will take care if a byte-order adjustment.

>> +
>> +        res = fdt_property_string(fdt, "status", "okay");
>> +        if ( res )
>> +            return res;
>> +
>> +        res = fdt_property_string(fdt, "compatible", "riscv");
>> +        if ( res )
>> +            return res;
>> +
>> +        BUILD_BUG_ON((sizeof("riscv,") +
>> +                      sizeof_field(struct gstage_mode_desc, name)) >= sizeof(buf));
>> +        snprintf(buf, sizeof(buf), "riscv,%s", max_gstage_mode->name);
>> +        res = fdt_property_string(fdt, "mmu-type", buf);
>> +        if ( res )
>> +            return res;
>> +
>> +        res = fdt_property_string(fdt, "riscv,isa", d->arch.guest_isa_str);
>> +        if ( res )
>> +            return res;
>> +
>> +        res = fdt_property_string(fdt, "device_type", "cpu");
>> +        if ( res )
>> +            return res;
>> +
>> +        res = fdt_begin_node(fdt, "interrupt-controller");
>> +        if ( res )
>> +            return res;
>> +
>> +        res = fdt_property_string(fdt, "compatible", "riscv,cpu-intc");
>> +        if ( res )
>> +            return res;
>> +
>> +        res = fdt_property_cell(fdt, "#interrupt-cells", 1);
>> +        if ( res )
>> +            return res;
>> +
>> +        res = fdt_property(fdt, "interrupt-controller", NULL, 0);
>> +        if ( res )
>> +            return res;
>> +
>> +        res = fdt_property_u32(fdt, "phandle", alloc_phandle(kinfo));
>> +        if ( res )
>> +            return res;
>> +
>> +        /* end of interrupt-controller */
> 
> Nit: Comment style. Also such a comment pretty clearly calls for a counterpart
> at the start.

Sure, I will add one.

Thanks.

~ Oleksii

Re: [PATCH v2 05/26] xen/riscv: introduce guest riscv,isa string

[Jan Beulich]

On 19.05.2026 15:24, Oleksii Kurochko wrote:
> 
> 
> On 5/19/26 2:12 PM, Jan Beulich wrote:
>> On 19.05.2026 13:59, Oleksii Kurochko wrote:
>>> On 5/18/26 5:51 PM, Jan Beulich wrote:
>>>> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>>>>> --- a/xen/arch/riscv/include/asm/cpufeature.h
>>>>> +++ b/xen/arch/riscv/include/asm/cpufeature.h
>>>>> @@ -17,6 +17,8 @@
>>>>>     */
>>>>>    #define RISCV_ISA_EXT_BASE  26
>>>>>    
>>>>> +#define RISCV_GUEST_ISA_STR_MAX 256
>>>>
>>>> This looks like it won't be good for very long, seeing how long ISA strings can
>>>> get. I wonder anyway whether ...
>>>>
>>>>> @@ -94,6 +95,9 @@ struct arch_domain {
>>>>>        struct p2m_domain p2m;
>>>>>    
>>>>>        struct paging_domain paging;
>>>>> +
>>>>> +    DECLARE_BITMAP(guest_isa, RISCV_ISA_EXT_MAX);
>>>>> +    char guest_isa_str[RISCV_GUEST_ISA_STR_MAX];
>>>>
>>>> ... a compile-time sized buffer is suitable here. Can't you allocate a buffer
>>>> just large enough to hold the string?
>>>
>>> It could be allocated dynamically.
>>>
>>> Does it make sense to evaluate in run-time what should be a buffer size?
>>> For this case I can't find analogue of realloc() in Xen.
>>
>> Hmm, I see xrealloc_array(), and surely we could gain xvrealloc_array()
>> which we'll need anyway once xrealloc_array() uses get converted. (I also
>> see x{,v}realloc_flex_struct(), but that's of no use here as it looks.)
> 
> Oh, I missed to turned off "full match" during search...
> 
>>
>>> Or it would be
>>> fine just to take something bigger as a const (lets say 2048) and use it
>>> for dynamic allocation?
>>
>> I'd rather not. Can't you determine how much space the string is going to
>> occupy?
> 
> I thought about two options as alternatives:
> 
> 1. Take as a length host RISC-V ISA string but theoretically we can 
> emulate some extensions which aren't mentioned in host RISC-V ISA string 
> so it could be longer. So not a good option.
> 
> 2. Having two walks in init_guest_isa():
>     Introduce the following function:
> static size_t guest_isa_str_len(const unsigned long *isa_bitmap)
> {
>      size_t len = 4; /* rvX prefix */
> 
>      for ( unsigned int i = 0; i < ARRAY_SIZE(riscv_isa_ext); i++ )
>      {
>          const struct riscv_isa_ext_data *ext = &riscv_isa_ext[i];
> 
>          if ( !riscv_isa_extension_available(isa_bitmap, ext->id) )
>              continue;
> 
>          if ( ext->id >= RISCV_ISA_EXT_BASE )
>              len++; /* '_' separator */
> 
>          len += strlen(ext->name);
>      }
> 
>      return len + 1; /* NUL terminator */
> }
> 
>    and then:
> 
> int init_guest_isa(struct domain *d)
> {
>      bitmap_andnot(d->arch.guest_isa, riscv_isa, guest_unsupp,
>                    RISCV_ISA_EXT_MAX);
> 
>      size_t len = guest_isa_str_len(d->arch.guest_isa);
>      d->arch.guest_isa_str = xzalloc_array(char, len);
>      if ( !d->arch.guest_isa_str )
>          return -ENOMEM;
> 
>      /* ... existing snprintf + strlcat loop unchanged ... */
> }
> 
> If approach 2 is a good one I can follow it.

This might be yet better with only a single function. Otherwise the two are
always at risk of going out of sync. After all you can use snprintf() to
determine just the size needed; if you go look, there may even be an
example or two in the tree.

Jan

Re: [PATCH v2 06/26] xen/riscv: implement make_cpus_node()

[Jan Beulich]

On 19.05.2026 15:33, Oleksii Kurochko wrote:
> On 5/18/26 6:00 PM, Jan Beulich wrote:
>> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>>> +    {
>>> +        char buf[64];
>>> +        uint32_t reg = cpu_to_fdt32(cpu);
>>
>> Isn't this a byte-order adjustment? If so, how come ...
>>
>>> +        snprintf(buf, sizeof(buf), "cpu@%u", cpu);
>>
>> ... the result is passed to an entirely non-FDT function? (Most pre-existing uses
>> of the function that I can spot store something in memory, i.e. adjusting byte-
>> order makes sense there.)
> 
> But here pure cpu is used instead of reg variable and reg variable is 
> used here ...
> 
>>
>>> +        res = fdt_begin_node(fdt, buf);
>>> +        if ( res )
>>> +            return res;
>>> +
>>> +        res = fdt_property(fdt, "reg", &reg, sizeof(reg));
>>> +        if ( res )
>>> +            return res;
> ... but it we could drop it and use just:
>    res = fdt_property_cell(fdt, "reg", cpu);
> 
> fdt_property_cell will take care if a byte-order adjustment.

I'm sorry, I clearly must have seen something that wasn't really there.

Jan

Re: [PATCH v2 05/26] xen/riscv: introduce guest riscv,isa string

[Oleksii Kurochko]

On 5/19/26 3:40 PM, Jan Beulich wrote:
> On 19.05.2026 15:24, Oleksii Kurochko wrote:
>>
>>
>> On 5/19/26 2:12 PM, Jan Beulich wrote:
>>> On 19.05.2026 13:59, Oleksii Kurochko wrote:
>>>> On 5/18/26 5:51 PM, Jan Beulich wrote:
>>>>> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>>>>>> --- a/xen/arch/riscv/include/asm/cpufeature.h
>>>>>> +++ b/xen/arch/riscv/include/asm/cpufeature.h
>>>>>> @@ -17,6 +17,8 @@
>>>>>>      */
>>>>>>     #define RISCV_ISA_EXT_BASE  26
>>>>>>     
>>>>>> +#define RISCV_GUEST_ISA_STR_MAX 256
>>>>>
>>>>> This looks like it won't be good for very long, seeing how long ISA strings can
>>>>> get. I wonder anyway whether ...
>>>>>
>>>>>> @@ -94,6 +95,9 @@ struct arch_domain {
>>>>>>         struct p2m_domain p2m;
>>>>>>     
>>>>>>         struct paging_domain paging;
>>>>>> +
>>>>>> +    DECLARE_BITMAP(guest_isa, RISCV_ISA_EXT_MAX);
>>>>>> +    char guest_isa_str[RISCV_GUEST_ISA_STR_MAX];
>>>>>
>>>>> ... a compile-time sized buffer is suitable here. Can't you allocate a buffer
>>>>> just large enough to hold the string?
>>>>
>>>> It could be allocated dynamically.
>>>>
>>>> Does it make sense to evaluate in run-time what should be a buffer size?
>>>> For this case I can't find analogue of realloc() in Xen.
>>>
>>> Hmm, I see xrealloc_array(), and surely we could gain xvrealloc_array()
>>> which we'll need anyway once xrealloc_array() uses get converted. (I also
>>> see x{,v}realloc_flex_struct(), but that's of no use here as it looks.)
>>
>> Oh, I missed to turned off "full match" during search...
>>
>>>
>>>> Or it would be
>>>> fine just to take something bigger as a const (lets say 2048) and use it
>>>> for dynamic allocation?
>>>
>>> I'd rather not. Can't you determine how much space the string is going to
>>> occupy?
>>
>> I thought about two options as alternatives:
>>
>> 1. Take as a length host RISC-V ISA string but theoretically we can
>> emulate some extensions which aren't mentioned in host RISC-V ISA string
>> so it could be longer. So not a good option.
>>
>> 2. Having two walks in init_guest_isa():
>>      Introduce the following function:
>> static size_t guest_isa_str_len(const unsigned long *isa_bitmap)
>> {
>>       size_t len = 4; /* rvX prefix */
>>
>>       for ( unsigned int i = 0; i < ARRAY_SIZE(riscv_isa_ext); i++ )
>>       {
>>           const struct riscv_isa_ext_data *ext = &riscv_isa_ext[i];
>>
>>           if ( !riscv_isa_extension_available(isa_bitmap, ext->id) )
>>               continue;
>>
>>           if ( ext->id >= RISCV_ISA_EXT_BASE )
>>               len++; /* '_' separator */
>>
>>           len += strlen(ext->name);
>>       }
>>
>>       return len + 1; /* NUL terminator */
>> }
>>
>>     and then:
>>
>> int init_guest_isa(struct domain *d)
>> {
>>       bitmap_andnot(d->arch.guest_isa, riscv_isa, guest_unsupp,
>>                     RISCV_ISA_EXT_MAX);
>>
>>       size_t len = guest_isa_str_len(d->arch.guest_isa);
>>       d->arch.guest_isa_str = xzalloc_array(char, len);
>>       if ( !d->arch.guest_isa_str )
>>           return -ENOMEM;
>>
>>       /* ... existing snprintf + strlcat loop unchanged ... */
>> }
>>
>> If approach 2 is a good one I can follow it.
> 
> This might be yet better with only a single function. Otherwise the two are
> always at risk of going out of sync. After all you can use snprintf() to
> determine just the size needed; if you go look, there may even be an
> example or two in the tree.

I will do than in the following way:

static int build_guest_isa_str(char *buf, size_t size,
                                const unsigned long *isa_bitmap)
{
     int total = 0;
     int ret;

#if defined(CONFIG_RISCV_32)
     ret = snprintf(buf, size, "rv32");
#elif defined(CONFIG_RISCV_64)
     ret = snprintf(buf, size, "rv64");
#else
#   error "Unsupported RISC-V bitness"
#endif
     if ( ret < 0 )
         return ret;
     total += ret;

     for ( unsigned int i = 0; i < ARRAY_SIZE(riscv_isa_ext); i++ )
     {
         const struct riscv_isa_ext_data *ext = &riscv_isa_ext[i];

         if ( !riscv_isa_extension_available(isa_bitmap, ext->id) )
             continue;

         ret = snprintf(buf ? buf + total : NULL,
                        buf ? size - total : 0, "%s%s",
                        ext->id >= RISCV_ISA_EXT_BASE ? "_" : "",
                        ext->name);
         if ( ret < 0 )
             return ret;
         total += ret;
     }

     return total;
}

int init_guest_isa(struct domain *d)
{
     int len;

     bitmap_andnot(d->arch.isa, riscv_isa, guest_unsupp,
                   RISCV_ISA_EXT_MAX);

     len = build_guest_isa_str(NULL, 0, d->arch.isa);
     if ( len < 0 )
         return len;

     d->arch.isa_str = xmalloc_array(char, len + 1);
     if ( !d->arch.isa_str )
         return -ENOMEM;

     build_guest_isa_str(d->arch.isa_str, len + 1, d->arch.isa);

     return 0;
}

Thanks for suggestion.

~ Oleksii

Re: [PATCH v2 05/26] xen/riscv: introduce guest riscv,isa string

[Jan Beulich]

On 19.05.2026 16:49, Oleksii Kurochko wrote:
> On 5/19/26 3:40 PM, Jan Beulich wrote:
>> On 19.05.2026 15:24, Oleksii Kurochko wrote:
>>> I thought about two options as alternatives:
>>>
>>> 1. Take as a length host RISC-V ISA string but theoretically we can
>>> emulate some extensions which aren't mentioned in host RISC-V ISA string
>>> so it could be longer. So not a good option.
>>>
>>> 2. Having two walks in init_guest_isa():
>>>      Introduce the following function:
>>> static size_t guest_isa_str_len(const unsigned long *isa_bitmap)
>>> {
>>>       size_t len = 4; /* rvX prefix */
>>>
>>>       for ( unsigned int i = 0; i < ARRAY_SIZE(riscv_isa_ext); i++ )
>>>       {
>>>           const struct riscv_isa_ext_data *ext = &riscv_isa_ext[i];
>>>
>>>           if ( !riscv_isa_extension_available(isa_bitmap, ext->id) )
>>>               continue;
>>>
>>>           if ( ext->id >= RISCV_ISA_EXT_BASE )
>>>               len++; /* '_' separator */
>>>
>>>           len += strlen(ext->name);
>>>       }
>>>
>>>       return len + 1; /* NUL terminator */
>>> }
>>>
>>>     and then:
>>>
>>> int init_guest_isa(struct domain *d)
>>> {
>>>       bitmap_andnot(d->arch.guest_isa, riscv_isa, guest_unsupp,
>>>                     RISCV_ISA_EXT_MAX);
>>>
>>>       size_t len = guest_isa_str_len(d->arch.guest_isa);
>>>       d->arch.guest_isa_str = xzalloc_array(char, len);
>>>       if ( !d->arch.guest_isa_str )
>>>           return -ENOMEM;
>>>
>>>       /* ... existing snprintf + strlcat loop unchanged ... */
>>> }
>>>
>>> If approach 2 is a good one I can follow it.
>>
>> This might be yet better with only a single function. Otherwise the two are
>> always at risk of going out of sync. After all you can use snprintf() to
>> determine just the size needed; if you go look, there may even be an
>> example or two in the tree.
> 
> I will do than in the following way:
> 
> static int build_guest_isa_str(char *buf, size_t size,
>                                 const unsigned long *isa_bitmap)
> {
>      int total = 0;
>      int ret;
> 
> #if defined(CONFIG_RISCV_32)
>      ret = snprintf(buf, size, "rv32");
> #elif defined(CONFIG_RISCV_64)
>      ret = snprintf(buf, size, "rv64");
> #else
> #   error "Unsupported RISC-V bitness"
> #endif
>      if ( ret < 0 )
>          return ret;

You can use total here right away, and limit ...

>      total += ret;
> 
>      for ( unsigned int i = 0; i < ARRAY_SIZE(riscv_isa_ext); i++ )
>      {
>          const struct riscv_isa_ext_data *ext = &riscv_isa_ext[i];

... ret's scope to this loop. This then also justifies total to be of a
signed type.

>          if ( !riscv_isa_extension_available(isa_bitmap, ext->id) )
>              continue;
> 
>          ret = snprintf(buf ? buf + total : NULL,
>                         buf ? size - total : 0, "%s%s",
>                         ext->id >= RISCV_ISA_EXT_BASE ? "_" : "",
>                         ext->name);
>          if ( ret < 0 )
>              return ret;
>          total += ret;
>      }
> 
>      return total;
> }
> 
> int init_guest_isa(struct domain *d)
> {
>      int len;
> 
>      bitmap_andnot(d->arch.isa, riscv_isa, guest_unsupp,
>                    RISCV_ISA_EXT_MAX);
> 
>      len = build_guest_isa_str(NULL, 0, d->arch.isa);
>      if ( len < 0 )
>          return len;
> 
>      d->arch.isa_str = xmalloc_array(char, len + 1);
>      if ( !d->arch.isa_str )
>          return -ENOMEM;
> 
>      build_guest_isa_str(d->arch.isa_str, len + 1, d->arch.isa);

At least ASSERT() the success of this?

Jan

Re: [PATCH v2 05/26] xen/riscv: introduce guest riscv,isa string

[Oleksii Kurochko]

On 5/19/26 4:53 PM, Jan Beulich wrote:
> On 19.05.2026 16:49, Oleksii Kurochko wrote:
>> On 5/19/26 3:40 PM, Jan Beulich wrote:
>>> On 19.05.2026 15:24, Oleksii Kurochko wrote:
>>>> I thought about two options as alternatives:
>>>>
>>>> 1. Take as a length host RISC-V ISA string but theoretically we can
>>>> emulate some extensions which aren't mentioned in host RISC-V ISA string
>>>> so it could be longer. So not a good option.
>>>>
>>>> 2. Having two walks in init_guest_isa():
>>>>       Introduce the following function:
>>>> static size_t guest_isa_str_len(const unsigned long *isa_bitmap)
>>>> {
>>>>        size_t len = 4; /* rvX prefix */
>>>>
>>>>        for ( unsigned int i = 0; i < ARRAY_SIZE(riscv_isa_ext); i++ )
>>>>        {
>>>>            const struct riscv_isa_ext_data *ext = &riscv_isa_ext[i];
>>>>
>>>>            if ( !riscv_isa_extension_available(isa_bitmap, ext->id) )
>>>>                continue;
>>>>
>>>>            if ( ext->id >= RISCV_ISA_EXT_BASE )
>>>>                len++; /* '_' separator */
>>>>
>>>>            len += strlen(ext->name);
>>>>        }
>>>>
>>>>        return len + 1; /* NUL terminator */
>>>> }
>>>>
>>>>      and then:
>>>>
>>>> int init_guest_isa(struct domain *d)
>>>> {
>>>>        bitmap_andnot(d->arch.guest_isa, riscv_isa, guest_unsupp,
>>>>                      RISCV_ISA_EXT_MAX);
>>>>
>>>>        size_t len = guest_isa_str_len(d->arch.guest_isa);
>>>>        d->arch.guest_isa_str = xzalloc_array(char, len);
>>>>        if ( !d->arch.guest_isa_str )
>>>>            return -ENOMEM;
>>>>
>>>>        /* ... existing snprintf + strlcat loop unchanged ... */
>>>> }
>>>>
>>>> If approach 2 is a good one I can follow it.
>>>
>>> This might be yet better with only a single function. Otherwise the two are
>>> always at risk of going out of sync. After all you can use snprintf() to
>>> determine just the size needed; if you go look, there may even be an
>>> example or two in the tree.
>>
>> I will do than in the following way:
>>
>> static int build_guest_isa_str(char *buf, size_t size,
>>                                  const unsigned long *isa_bitmap)
>> {
>>       int total = 0;
>>       int ret;
>>
>> #if defined(CONFIG_RISCV_32)
>>       ret = snprintf(buf, size, "rv32");
>> #elif defined(CONFIG_RISCV_64)
>>       ret = snprintf(buf, size, "rv64");
>> #else
>> #   error "Unsupported RISC-V bitness"
>> #endif
>>       if ( ret < 0 )
>>           return ret;
> 
> You can use total here right away, and limit ...
> 
>>       total += ret;
>>
>>       for ( unsigned int i = 0; i < ARRAY_SIZE(riscv_isa_ext); i++ )
>>       {
>>           const struct riscv_isa_ext_data *ext = &riscv_isa_ext[i];
> 
> ... ret's scope to this loop. This then also justifies total to be of a
> signed type.
It makes sense.

> 
>>           if ( !riscv_isa_extension_available(isa_bitmap, ext->id) )
>>               continue;
>>
>>           ret = snprintf(buf ? buf + total : NULL,
>>                          buf ? size - total : 0, "%s%s",
>>                          ext->id >= RISCV_ISA_EXT_BASE ? "_" : "",
>>                          ext->name);
>>           if ( ret < 0 )
>>               return ret;
>>           total += ret;
>>       }
>>
>>       return total;
>> }
>>
>> int init_guest_isa(struct domain *d)
>> {
>>       int len;
>>
>>       bitmap_andnot(d->arch.isa, riscv_isa, guest_unsupp,
>>                     RISCV_ISA_EXT_MAX);
>>
>>       len = build_guest_isa_str(NULL, 0, d->arch.isa);
>>       if ( len < 0 )
>>           return len;
>>
>>       d->arch.isa_str = xmalloc_array(char, len + 1);
>>       if ( !d->arch.isa_str )
>>           return -ENOMEM;
>>
>>       build_guest_isa_str(d->arch.isa_str, len + 1, d->arch.isa);
> 
> At least ASSERT() the success of this?

I will add:

ASSERT(build_guest_isa_str(d->arch.isa_str, len + 1, d->arch.isa) == len);

but I expect that we won't be here if len is incorrect or 
d->arch.isa_str wasn't allocated properly.

Thanks.

~ Oleksii

Re: [PATCH v2 05/26] xen/riscv: introduce guest riscv,isa string

[Jan Beulich]

On 19.05.2026 17:17, Oleksii Kurochko wrote:
> On 5/19/26 4:53 PM, Jan Beulich wrote:
>> On 19.05.2026 16:49, Oleksii Kurochko wrote:
>>> int init_guest_isa(struct domain *d)
>>> {
>>>       int len;
>>>
>>>       bitmap_andnot(d->arch.isa, riscv_isa, guest_unsupp,
>>>                     RISCV_ISA_EXT_MAX);
>>>
>>>       len = build_guest_isa_str(NULL, 0, d->arch.isa);
>>>       if ( len < 0 )
>>>           return len;
>>>
>>>       d->arch.isa_str = xmalloc_array(char, len + 1);
>>>       if ( !d->arch.isa_str )
>>>           return -ENOMEM;
>>>
>>>       build_guest_isa_str(d->arch.isa_str, len + 1, d->arch.isa);
>>
>> At least ASSERT() the success of this?
> 
> I will add:
> 
> ASSERT(build_guest_isa_str(d->arch.isa_str, len + 1, d->arch.isa) == len);

Ehem. Please check how ASSERT() works (and the difference to BUG_ON()).

Jan

> but I expect that we won't be here if len is incorrect or 
> d->arch.isa_str wasn't allocated properly.
> 
> Thanks.
> 
> ~ Oleksii

Re: [PATCH v2 05/26] xen/riscv: introduce guest riscv,isa string

[Oleksii Kurochko]

On 5/19/26 5:56 PM, Jan Beulich wrote:
> On 19.05.2026 17:17, Oleksii Kurochko wrote:
>> On 5/19/26 4:53 PM, Jan Beulich wrote:
>>> On 19.05.2026 16:49, Oleksii Kurochko wrote:
>>>> int init_guest_isa(struct domain *d)
>>>> {
>>>>        int len;
>>>>
>>>>        bitmap_andnot(d->arch.isa, riscv_isa, guest_unsupp,
>>>>                      RISCV_ISA_EXT_MAX);
>>>>
>>>>        len = build_guest_isa_str(NULL, 0, d->arch.isa);
>>>>        if ( len < 0 )
>>>>            return len;
>>>>
>>>>        d->arch.isa_str = xmalloc_array(char, len + 1);
>>>>        if ( !d->arch.isa_str )
>>>>            return -ENOMEM;
>>>>
>>>>        build_guest_isa_str(d->arch.isa_str, len + 1, d->arch.isa);
>>>
>>> At least ASSERT() the success of this?
>>
>> I will add:
>>
>> ASSERT(build_guest_isa_str(d->arch.isa_str, len + 1, d->arch.isa) == len);
> 
> Ehem. Please check how ASSERT() works (and the difference to BUG_ON()).
> 

Condition itself looks correct for ASSERT(). If build_guest_isa_str() 
returns value equal to len then assert_failed() shouldn't be called.

Maybe do you mean that it will never fire in release build then yes it 
should be changed to BUG_ON() and the condition should be inverted:
  BUG_ON(build_guest_isa_str(d->arch.isa_str, len + 1, d->arch.isa) != len);

~ Oleksii

Re: [PATCH v2 05/26] xen/riscv: introduce guest riscv,isa string

[Jan Beulich]

On 19.05.2026 18:21, Oleksii Kurochko wrote:
> On 5/19/26 5:56 PM, Jan Beulich wrote:
>> On 19.05.2026 17:17, Oleksii Kurochko wrote:
>>> On 5/19/26 4:53 PM, Jan Beulich wrote:
>>>> On 19.05.2026 16:49, Oleksii Kurochko wrote:
>>>>> int init_guest_isa(struct domain *d)
>>>>> {
>>>>>        int len;
>>>>>
>>>>>        bitmap_andnot(d->arch.isa, riscv_isa, guest_unsupp,
>>>>>                      RISCV_ISA_EXT_MAX);
>>>>>
>>>>>        len = build_guest_isa_str(NULL, 0, d->arch.isa);
>>>>>        if ( len < 0 )
>>>>>            return len;
>>>>>
>>>>>        d->arch.isa_str = xmalloc_array(char, len + 1);
>>>>>        if ( !d->arch.isa_str )
>>>>>            return -ENOMEM;
>>>>>
>>>>>        build_guest_isa_str(d->arch.isa_str, len + 1, d->arch.isa);
>>>>
>>>> At least ASSERT() the success of this?
>>>
>>> I will add:
>>>
>>> ASSERT(build_guest_isa_str(d->arch.isa_str, len + 1, d->arch.isa) == len);
>>
>> Ehem. Please check how ASSERT() works (and the difference to BUG_ON()).
> 
> Condition itself looks correct for ASSERT(). If build_guest_isa_str() 
> returns value equal to len then assert_failed() shouldn't be called.
> 
> Maybe do you mean that it will never fire in release build then yes it 
> should be changed to BUG_ON() and the condition should be inverted:
>   BUG_ON(build_guest_isa_str(d->arch.isa_str, len + 1, d->arch.isa) != len);

No. Unlike in BUG_ON(), you can't use expressions with side effects (i.e.
also function calls, unless they're const/pure) in ASSERT(). That's
true for standard C's assert() as well, i.e. not Xen specific at all.
(We do, however, diverge from assert() in another aspect.)

Jan

Re: [PATCH v2 05/26] xen/riscv: introduce guest riscv,isa string

[Oleksii Kurochko]

On 5/20/26 8:13 AM, Jan Beulich wrote:
> On 19.05.2026 18:21, Oleksii Kurochko wrote:
>> On 5/19/26 5:56 PM, Jan Beulich wrote:
>>> On 19.05.2026 17:17, Oleksii Kurochko wrote:
>>>> On 5/19/26 4:53 PM, Jan Beulich wrote:
>>>>> On 19.05.2026 16:49, Oleksii Kurochko wrote:
>>>>>> int init_guest_isa(struct domain *d)
>>>>>> {
>>>>>>         int len;
>>>>>>
>>>>>>         bitmap_andnot(d->arch.isa, riscv_isa, guest_unsupp,
>>>>>>                       RISCV_ISA_EXT_MAX);
>>>>>>
>>>>>>         len = build_guest_isa_str(NULL, 0, d->arch.isa);
>>>>>>         if ( len < 0 )
>>>>>>             return len;
>>>>>>
>>>>>>         d->arch.isa_str = xmalloc_array(char, len + 1);
>>>>>>         if ( !d->arch.isa_str )
>>>>>>             return -ENOMEM;
>>>>>>
>>>>>>         build_guest_isa_str(d->arch.isa_str, len + 1, d->arch.isa);
>>>>>
>>>>> At least ASSERT() the success of this?
>>>>
>>>> I will add:
>>>>
>>>> ASSERT(build_guest_isa_str(d->arch.isa_str, len + 1, d->arch.isa) == len);
>>>
>>> Ehem. Please check how ASSERT() works (and the difference to BUG_ON()).
>>
>> Condition itself looks correct for ASSERT(). If build_guest_isa_str()
>> returns value equal to len then assert_failed() shouldn't be called.
>>
>> Maybe do you mean that it will never fire in release build then yes it
>> should be changed to BUG_ON() and the condition should be inverted:
>>    BUG_ON(build_guest_isa_str(d->arch.isa_str, len + 1, d->arch.isa) != len);
> 
> No. Unlike in BUG_ON(), you can't use expressions with side effects (i.e.
> also function calls, unless they're const/pure) in ASSERT(). That's
> true for standard C's assert() as well, i.e. not Xen specific at all.
> (We do, however, diverge from assert() in another aspect.)

Got you.

Also it could be in release build that build_guest_isa_str() won't be 
called at all because of how ASSERT() is open-coded:  if ( 0 && (p) ...

Then probably it would be better to do in the following way:

if ( build_guest_isa_str(d->arch.isa_str, len + 1, d->arch.isa) != len )
{
     XVFREE(d->arch.isa_str);
     return -EINVAL;
}

return 0;

I am not sure that -EINVAL is the best one option but I don't see any 
better now.

~ Oleksii

Re: [PATCH v2 08/26] xen/riscv: implement make_arch_nodes()

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> No RISC-V-specific nodes need to be created at the moment,
> so make_arch_nodes() is implemented to simply return 0.
> 
> It is placed in dom0less-build.c as make_arch_nodes() is
> only used in the dom0less code path. In the future, it will
> be extended to create an emulated UART node.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

Re: [PATCH v2 09/26] xen/riscv: introduce init interrupt controller operations

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> Introduce intc_hw_init_ops structure to avoid risky mix of init
> function and non-init function.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
> Changes in v2:
>  - New patch.
> ---
>  xen/arch/riscv/aplic.c            |  7 +++++--
>  xen/arch/riscv/include/asm/intc.h | 10 +++++++---
>  xen/arch/riscv/intc.c             | 10 ++++++++--
>  3 files changed, 20 insertions(+), 7 deletions(-)
> 
> diff --git a/xen/arch/riscv/aplic.c b/xen/arch/riscv/aplic.c
> index 739e8dab3498..97dc0ef731f0 100644
> --- a/xen/arch/riscv/aplic.c
> +++ b/xen/arch/riscv/aplic.c
> @@ -306,12 +306,15 @@ static const hw_irq_controller aplic_xen_irq_type = {
>  
>  static const struct intc_hw_operations aplic_ops = {
>      .info                = &aplic_info,
> -    .init                = aplic_init,
>      .host_irq_type       = &aplic_xen_irq_type,
>      .handle_interrupt    = aplic_handle_interrupt,
>      .set_irq_type        = aplic_set_irq_type,
>  };
>  
> +static const struct intc_hw_init_ops __initdata aplic_init_ops = {
> +    .init                = aplic_init,
> +};

const wants to pair with __initconst. Then:
Acked-by: Jan Beulich <jbeulich@suse.com>

However, I have another comment for consideration:

> --- a/xen/arch/riscv/intc.c
> +++ b/xen/arch/riscv/intc.c
> @@ -12,9 +12,13 @@
>  
>  static const struct intc_hw_operations *__ro_after_init intc_hw_ops;
>  
> -void __init register_intc_ops(const struct intc_hw_operations *ops)
> +static const struct intc_hw_init_ops *__initdata intc_hw_init_ops;
> +
> +void __init register_intc_ops(const struct intc_hw_operations *ops,
> +                              const struct intc_hw_init_ops *init_ops)
>  {
>      intc_hw_ops = ops;
> +    intc_hw_init_ops = init_ops;
>  }

Again following what we do e.g. in x86'es IOMMU code, instead of passing
two pointers to the function, have struct intc_hw_init_ops have a
const struct intc_hw_operations * member which then can be used to
set intc_hw_ops here? Both will always come in pairs anyway.

Jan

Re: [PATCH v2 10/26] xen/riscv: implement make_intc_domU_node()

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> @@ -46,6 +45,15 @@ struct intc_hw_init_ops {
>      int (*init)(void);
>  };
>  
> +struct vintc_init_ops {
> +    /* Create interrupt controller node for domain */
> +    int (*make_domu_dt_node)(struct kernel_info *kinfo);
> +};
> +
> +struct vintc {
> +    struct vintc_init_ops *init_ops;

Pointer-to-const?

> @@ -73,3 +74,13 @@ void intc_route_irq_to_xen(struct irq_desc *desc, unsigned int priority)
>      intc_set_irq_type(desc, desc->arch.type);
>      intc_set_irq_priority(desc, priority);
>  }
> +
> +int __init make_intc_domU_node(struct kernel_info *kinfo)
> +{
> +    struct vintc *vintc = kinfo->bd.d->arch.vintc;
> +
> +    if ( intc_hw_ops && vintc->init_ops && vintc->init_ops->make_domu_dt_node )

Why the intc_hw_ops check? The variable isn't otherwise used here.

As to the other two checks: Is it plausible for either of the two pointers
to be NULL? If not, merely ASSERT() or even build upon ...

> +        return vintc->init_ops->make_domu_dt_node(kinfo);

... NULL derefs to fault anyway?

Jan

Re: [PATCH v2 11/26] xen/riscv: introduce aia_init() and aia_usable()

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> aia_init() is going to contain all the logic related to AIA initialization.
> 
> At the moment, it only checks whether the SSAIA extension is available,
> and if so, sets is_aia_usable (which  indicates more than just the
> availability of the extension) to true; it also signifies that the necessary
> components (to be introduced in follow-up patches) have been initialized.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
albeit again with a comment for consideration:

> --- /dev/null
> +++ b/xen/arch/riscv/aia.c
> @@ -0,0 +1,23 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +
> +#include <xen/errno.h>
> +#include <xen/init.h>
> +#include <xen/sections.h>
> +#include <xen/types.h>
> +
> +#include <asm/cpufeature.h>
> +
> +static bool __ro_after_init is_aia_usable;

While in an if() the is_ prefix may make sense, in most other uses (all
of the ones below, imo) it is of questionable value. I'd suggest to drop
it, but of course with the then leading underscore retained, to ...

> +bool aia_usable(void)
> +{
> +    return is_aia_usable;
> +}

... not conflict with this function.

Jan

Re: [PATCH v2 12/26] xen/riscv: add basic VGEIN management for AIA guests

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> --- a/xen/arch/riscv/aia.c
> +++ b/xen/arch/riscv/aia.c
> @@ -1,11 +1,33 @@
>  /* SPDX-License-Identifier: GPL-2.0-only */
>  
> +#include <xen/bitmap.h>
> +#include <xen/cpu.h>
>  #include <xen/errno.h>
>  #include <xen/init.h>
>  #include <xen/sections.h>
> +#include <xen/sched.h>
> +#include <xen/spinlock.h>
>  #include <xen/types.h>
> +#include <xen/xvmalloc.h>
>  
> +#include <asm/aia.h>
>  #include <asm/cpufeature.h>
> +#include <asm/csr.h>
> +#include <asm/current.h>
> +
> +struct vgein_ctrl {
> +    unsigned long bmp;
> +    spinlock_t lock;
> +    struct vcpu **owners;
> +    /* The least-significant bits are implemented first, apart from bit 0 */
> +    unsigned int geilen;
> +};
> +
> +/*
> + * Bitmap for each physical cpus to detect which VS (guest)
> + * interrupt file id was used.
> + */
> +static DEFINE_PER_CPU(struct vgein_ctrl, vgein);

Why "Bitmap" in the comment?

> @@ -14,10 +36,132 @@ bool aia_usable(void)
>      return is_aia_usable;
>  }
>  
> +static int vgein_init(unsigned int cpu)
> +{
> +    struct vgein_ctrl *vgein = &per_cpu(vgein, cpu);
> +
> +    csr_write(CSR_HGEIE, -1UL);
> +    vgein->geilen = flsl(csr_read(CSR_HGEIE) >> 1);
> +    csr_write(CSR_HGEIE, 0);
> +
> +    printk("cpu%d.geilen=%u\n", cpu, vgein->geilen);

%u please with unsigned int.

> +    if ( !vgein->geilen )
> +        return -EOPNOTSUPP;
> +
> +    vgein->owners = xvzalloc_array(struct vcpu *, vgein->geilen);
> +    if ( !vgein->owners )
> +        return -ENOMEM;
> +
> +    spin_lock_init(&vgein->lock);
> +
> +    return 0;
> +}
> +
> +static int cf_check cpu_callback(struct notifier_block *nfb, unsigned long action,
> +                        void *hcpu)
> +{
> +    unsigned int cpu = (unsigned long)hcpu;
> +    int rc = 0;
> +
> +    switch ( action )
> +    {
> +    case CPU_STARTING:
> +        rc = vgein_init(cpu);
> +        if ( rc )
> +            printk("AIA: failed to init vgein for CPU%un", cpu);

Looks like there's \ missing ahead of the trailing 'n'.

> +        break;
> +    }
> +
> +    return notifier_from_errno(rc);
> +}
> +
> +static struct notifier_block cpu_nfb = {
> +    .notifier_call = cpu_callback,
> +};
> +
>  void __init aia_init(void)
>  {
> +    int rc;
> +
>      if ( !riscv_isa_extension_available(NULL, RISCV_ISA_EXT_ssaia) )
> +    {
> +        dprintk(XENLOG_WARNING, "SSAIA isn't present in riscv,isa\n");
> +        return;
> +    }
> +
> +    if ( (rc = vgein_init(0)) )
> +    {
> +        dprintk(XENLOG_ERR, "vgein_init() failed with rc(%d)\n", rc);

Messages like this one, provided they're really needed, should be yet more
terse imo: "vgein_init() failed: %d\n".

>          return;
> +    }
>  
>      is_aia_usable = true;
> +
> +    register_cpu_notifier(&cpu_nfb);
> +}
> +
> +unsigned int vgein_assign(struct vcpu *v)
> +{
> +    unsigned int vgein_id;
> +    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
> +    unsigned long *bmp = &vgein->bmp;
> +    unsigned long flags;
> +
> +    spin_lock_irqsave(&vgein->lock, flags);
> +    /*
> +     * The vgein_id shouldn't be zero, as it will indicate that no guest
> +     * external interrupt source is selected for VS-level external interrupts
> +     * according to RISC-V priviliged spec:
> +     *   Hypervisor Status Register (hstatus) in RISC-V priviliged spec:
> +     *
> +     *   The VGEIN (Virtual Guest External Interrupt Number) field selects
> +     *   a guest external interrupt source for VS-level external interrupts.
> +     *   VGEIN is a WLRL field that must be able to hold values between zero
> +     *   and the maximum guest external interrupt number (known as GEILEN),
> +     *   inclusive.
> +     *   When VGEIN=0, no guest external interrupt source is selected for
> +     *   VS-level external interrupts.
> +     *
> +     * So start to search from bit number 1.
> +     */
> +    vgein_id = find_next_zero_bit(bmp, vgein->geilen + 1, 1);
> +
> +    if ( vgein_id > vgein->geilen )
> +        vgein_id = 0;
> +    else
> +        __set_bit(vgein_id, bmp);
> +
> +    spin_unlock_irqrestore(&vgein->lock, flags);
> +
> +#ifdef VGEIN_DEBUG
> +    gprintk(XENLOG_DEBUG, "%s: %pv: vgein_id(%u), xen_cpu%d_bmp=%#lx\n",
> +           __func__, v, vgein_id, v->processor, *bmp);

%d vs unsigned int again (and then yet again further down).

> +#endif
> +
> +    vcpu_guest_cpu_user_regs(v)->hstatus &= ~HSTATUS_VGEIN;

Is this needed when vgein_release() also does it?

> +    vcpu_guest_cpu_user_regs(v)->hstatus |=
> +        MASK_INSR(vgein_id, HSTATUS_VGEIN);
> +
> +    return vgein_id;
> +}
> +
> +void vgein_release(struct vcpu *v, unsigned int vgen_id)
> +{
> +    unsigned long flags;
> +    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
> +
> +    if ( !vgen_id )
> +        return;
> +
> +    spin_lock_irqsave(&vgein->lock, flags);
> +     __clear_bit(vgen_id, &vgein->bmp);
> +    spin_unlock_irqrestore(&vgein->lock, flags);
> +
> +#ifdef VGEIN_DEBUG
> +    gprintk(XENLOG_DEBUG, "%s: vgein_id(%u), xen_cpu%d_bmp=%#lx\n",
> +           __func__, vgen_id, v->processor, vgein->bmp);
> +#endif
> +
> +    vcpu_guest_cpu_user_regs(v)->hstatus &= ~HSTATUS_VGEIN;
>  }

Overall: How is one to review these two functions, when it's entirely
unclear where they're going to be called from? Among other aspects it
doesn't become clear what the behavior is going to be when
vgein_assign() doesn't find an available ID. I've therefore only
commented on mechanical aspects I noticed.

Jan

Re: [PATCH v2 13/26] xen/riscv: introduce per-vCPU IMSIC state

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> --- a/xen/arch/riscv/imsic.c
> +++ b/xen/arch/riscv/imsic.c
> @@ -16,6 +16,7 @@
>  #include <xen/errno.h>
>  #include <xen/init.h>
>  #include <xen/macros.h>
> +#include <xen/sched.h>
>  #include <xen/smp.h>
>  #include <xen/spinlock.h>
>  #include <xen/xvmalloc.h>
> @@ -56,6 +57,16 @@ do {                            \
>      csr_clear(CSR_SIREG, v);    \
>  } while (0)
>  
> +unsigned int vcpu_guest_file_id(const struct vcpu *v)
> +{
> +    return ACCESS_ONCE(v->arch.vimsic_state->guest_file_id);
> +}
> +
> +void imsic_set_guest_file_id(const struct vcpu *v, unsigned int guest_file_id)

Some people will demand that "const" be omitted in cases like this one, as
it only works ...

> +{
> +    ACCESS_ONCE(v->arch.vimsic_state->guest_file_id) = guest_file_id;

... as long as vimsic_state is a pointer (and not a sub-structure).

> @@ -312,6 +323,30 @@ static int imsic_parse_node(const struct dt_device_node *node,
>      return 0;
>  }
>  
> +int vcpu_imsic_init(struct vcpu *v)
> +{
> +    struct vimsic_state *imsic_state;
> +
> +    /* Allocate IMSIC context */
> +    imsic_state = xvzalloc(struct vimsic_state);
> +    if ( !imsic_state )
> +        return -ENOMEM;
> +
> +    v->arch.vimsic_state = imsic_state;
> +
> +    /* Setup IMSIC context  */
> +    rwlock_init(&imsic_state->vsfile_lock);
> +
> +    imsic_state->vsfile_pcpu = NR_CPUS;
> +
> +    return 0;
> +}
> +
> +void vcpu_imsic_deinit(const struct vcpu *v)
> +{
> +    xvfree(v->arch.vimsic_state);

Better XVFREE(), for the function to be idempotent.

> --- a/xen/arch/riscv/include/asm/imsic.h
> +++ b/xen/arch/riscv/include/asm/imsic.h
> @@ -11,6 +11,7 @@
>  #ifndef ASM_RISCV_IMSIC_H
>  #define ASM_RISCV_IMSIC_H
>  
> +#include <xen/rwlock.h>
>  #include <xen/spinlock.h>
>  #include <xen/stdbool.h>
>  #include <xen/types.h>
> @@ -61,7 +62,24 @@ struct imsic_config {
>      spinlock_t lock;
>  };
>  
> +struct vimsic_state {
> +    /* IMSIC VS-file */
> +    rwlock_t vsfile_lock;
> +    /*
> +     * (guest_file_id == 0) -> s/w IMSIC SW-file
> +     * (guest_file_id > 0) -> h/w IMSIC VS-file
> +     */
> +    unsigned int guest_file_id;
> +    /*
> +     * (vsfile_pcpu >= 0) => h/w IMSIC VS-file
> +     * (vsfile_pcpu == NR_CPUS) => s/w IMSIC SW-file
> +     */
> +    unsigned int vsfile_pcpu;
> +};

In the comments, what does SW stand for? Not "software" I assume, as
that's already expressed by s/w.

Jan

Re: [PATCH v2 09/26] xen/riscv: introduce init interrupt controller operations

[Oleksii Kurochko]

On 5/21/26 3:25 PM, Jan Beulich wrote:
> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>> Introduce intc_hw_init_ops structure to avoid risky mix of init
>> function and non-init function.
>>
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>> ---
>> Changes in v2:
>>   - New patch.
>> ---
>>   xen/arch/riscv/aplic.c            |  7 +++++--
>>   xen/arch/riscv/include/asm/intc.h | 10 +++++++---
>>   xen/arch/riscv/intc.c             | 10 ++++++++--
>>   3 files changed, 20 insertions(+), 7 deletions(-)
>>
>> diff --git a/xen/arch/riscv/aplic.c b/xen/arch/riscv/aplic.c
>> index 739e8dab3498..97dc0ef731f0 100644
>> --- a/xen/arch/riscv/aplic.c
>> +++ b/xen/arch/riscv/aplic.c
>> @@ -306,12 +306,15 @@ static const hw_irq_controller aplic_xen_irq_type = {
>>   
>>   static const struct intc_hw_operations aplic_ops = {
>>       .info                = &aplic_info,
>> -    .init                = aplic_init,
>>       .host_irq_type       = &aplic_xen_irq_type,
>>       .handle_interrupt    = aplic_handle_interrupt,
>>       .set_irq_type        = aplic_set_irq_type,
>>   };
>>   
>> +static const struct intc_hw_init_ops __initdata aplic_init_ops = {
>> +    .init                = aplic_init,
>> +};
> 
> const wants to pair with __initconst. Then:
> Acked-by: Jan Beulich <jbeulich@suse.com>
> 
Thanks.

> However, I have another comment for consideration:
> 
>> --- a/xen/arch/riscv/intc.c
>> +++ b/xen/arch/riscv/intc.c
>> @@ -12,9 +12,13 @@
>>   
>>   static const struct intc_hw_operations *__ro_after_init intc_hw_ops;
>>   
>> -void __init register_intc_ops(const struct intc_hw_operations *ops)
>> +static const struct intc_hw_init_ops *__initdata intc_hw_init_ops;
>> +
>> +void __init register_intc_ops(const struct intc_hw_operations *ops,
>> +                              const struct intc_hw_init_ops *init_ops)
>>   {
>>       intc_hw_ops = ops;
>> +    intc_hw_init_ops = init_ops;
>>   }
> 
> Again following what we do e.g. in x86'es IOMMU code, instead of passing
> two pointers to the function, have struct intc_hw_init_ops have a
> const struct intc_hw_operations * member which then can be used to
> set intc_hw_ops here? Both will always come in pairs anyway.

It makes sense to me.

So I will do the following:

@@ -350,7 +351,7 @@ static int __init aplic_preinit(struct 
dt_device_node *node, const void *dat)

      dt_irq_xlate = aplic_irq_xlate;

-    register_intc_ops(&aplic_ops, &aplic_init_ops);
+    register_intc_ops(&aplic_init_ops);

      /* Enable supervisor external interrupt */
      csr_set(CSR_SIE, BIT(IRQ_S_EXT, UL));
diff --git a/xen/arch/riscv/include/asm/intc.h 
b/xen/arch/riscv/include/asm/intc.h
index 8b498e43b33f..3d84fcc51d1a 100644
--- a/xen/arch/riscv/include/asm/intc.h
+++ b/xen/arch/riscv/include/asm/intc.h
@@ -42,14 +42,14 @@ struct intc_hw_operations {
  };

  struct intc_hw_init_ops {
+    const struct intc_hw_operations *ops;
      /* Initialize the intc and the boot CPU */
      int (*init)(void);
  };

  void intc_preinit(void);

-void register_intc_ops(const struct intc_hw_operations *ops,
-                       const struct intc_hw_init_ops *init_ops);
+void register_intc_ops(const struct intc_hw_init_ops *init_ops);

  void intc_init(void);

diff --git a/xen/arch/riscv/intc.c b/xen/arch/riscv/intc.c
index 8649160403f7..3600d23bdb5b 100644
--- a/xen/arch/riscv/intc.c
+++ b/xen/arch/riscv/intc.c
@@ -14,10 +14,9 @@ static const struct intc_hw_operations 
*__ro_after_init intc_hw_ops;

  static const struct intc_hw_init_ops *__initdata intc_hw_init_ops;

-void __init register_intc_ops(const struct intc_hw_operations *ops,
-                              const struct intc_hw_init_ops *init_ops)
+void __init register_intc_ops(const struct intc_hw_init_ops *init_ops)
  {
-    intc_hw_ops = ops;
+    intc_hw_ops = init_ops->ops;
      intc_hw_init_ops = init_ops;
  }

Thanks.

~ Oleksii

Re: [PATCH v2 10/26] xen/riscv: implement make_intc_domU_node()

[Oleksii Kurochko]

On 5/21/26 3:30 PM, Jan Beulich wrote:
> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>> @@ -46,6 +45,15 @@ struct intc_hw_init_ops {
>>       int (*init)(void);
>>   };
>>   
>> +struct vintc_init_ops {
>> +    /* Create interrupt controller node for domain */
>> +    int (*make_domu_dt_node)(struct kernel_info *kinfo);
>> +};
>> +
>> +struct vintc {
>> +    struct vintc_init_ops *init_ops;
> 
> Pointer-to-const?
> 
>> @@ -73,3 +74,13 @@ void intc_route_irq_to_xen(struct irq_desc *desc, unsigned int priority)
>>       intc_set_irq_type(desc, desc->arch.type);
>>       intc_set_irq_priority(desc, priority);
>>   }
>> +
>> +int __init make_intc_domU_node(struct kernel_info *kinfo)
>> +{
>> +    struct vintc *vintc = kinfo->bd.d->arch.vintc;
>> +
>> +    if ( intc_hw_ops && vintc->init_ops && vintc->init_ops->make_domu_dt_node )
> 
> Why the intc_hw_ops check? The variable isn't otherwise used here.
> 
> As to the other two checks: Is it plausible for either of the two pointers
> to be NULL? If not, merely ASSERT() or even build upon ...
> 
>> +        return vintc->init_ops->make_domu_dt_node(kinfo);
> 
> ... NULL derefs to fault anyway?

Specifically this pointers can't be NULL.

Generally it would be okay to just have have NULL derefs in runtime but 
there isn't guarantee that some register which will be useful to debug 
where NULL deref happens will be properly filled. I faced that several 
times so I prefer to have here ASSERT().

Thanks.

~ Oleksii

Re: [PATCH v2 11/26] xen/riscv: introduce aia_init() and aia_usable()

[Oleksii Kurochko]

On 5/21/26 4:57 PM, Jan Beulich wrote:
> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>> aia_init() is going to contain all the logic related to AIA initialization.
>>
>> At the moment, it only checks whether the SSAIA extension is available,
>> and if so, sets is_aia_usable (which  indicates more than just the
>> availability of the extension) to true; it also signifies that the necessary
>> components (to be introduced in follow-up patches) have been initialized.
>>
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> 
> Acked-by: Jan Beulich <jbeulich@suse.com>

Thanks.

> albeit again with a comment for consideration:
> 
>> --- /dev/null
>> +++ b/xen/arch/riscv/aia.c
>> @@ -0,0 +1,23 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +
>> +#include <xen/errno.h>
>> +#include <xen/init.h>
>> +#include <xen/sections.h>
>> +#include <xen/types.h>
>> +
>> +#include <asm/cpufeature.h>
>> +
>> +static bool __ro_after_init is_aia_usable;
> 
> While in an if() the is_ prefix may make sense, in most other uses (all
> of the ones below, imo) it is of questionable value. I'd suggest to drop
> it, but of course with the then leading underscore retained, to ...
> 
>> +bool aia_usable(void)
>> +{
>> +    return is_aia_usable;
>> +}
> 
> ... not conflict with this function.
> 

IIUC then I am okay with the following changes:

-static bool __ro_after_init is_aia_usable;
+static bool __ro_after_init _aia_usable;

  bool aia_usable(void)
  {
-    return is_aia_usable;
+    return _aia_usable;
  }

  void __init aia_init(void)
@@ -19,5 +19,5 @@ void __init aia_init(void)
      if ( !riscv_isa_extension_available(NULL, RISCV_ISA_EXT_ssaia) )
          return;

-    is_aia_usable = true;
+    _aia_usable = true;
  }

~ Oleksii

Re: [PATCH v2 12/26] xen/riscv: add basic VGEIN management for AIA guests

[Oleksii Kurochko]

On 5/21/26 5:11 PM, Jan Beulich wrote:
> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>> --- a/xen/arch/riscv/aia.c
>> +++ b/xen/arch/riscv/aia.c
>> @@ -1,11 +1,33 @@
>>   /* SPDX-License-Identifier: GPL-2.0-only */
>>   
>> +#include <xen/bitmap.h>
>> +#include <xen/cpu.h>
>>   #include <xen/errno.h>
>>   #include <xen/init.h>
>>   #include <xen/sections.h>
>> +#include <xen/sched.h>
>> +#include <xen/spinlock.h>
>>   #include <xen/types.h>
>> +#include <xen/xvmalloc.h>
>>   
>> +#include <asm/aia.h>
>>   #include <asm/cpufeature.h>
>> +#include <asm/csr.h>
>> +#include <asm/current.h>
>> +
>> +struct vgein_ctrl {
>> +    unsigned long bmp;
>> +    spinlock_t lock;
>> +    struct vcpu **owners;
>> +    /* The least-significant bits are implemented first, apart from bit 0 */
>> +    unsigned int geilen;
>> +};
>> +
>> +/*
>> + * Bitmap for each physical cpus to detect which VS (guest)
>> + * interrupt file id was used.
>> + */
>> +static DEFINE_PER_CPU(struct vgein_ctrl, vgein);
> 
> Why "Bitmap" in the comment?

Hmm, good question. Looks like rudiment from initial implementation.

I will rephrase it to:
/*
  * VGEIN control structure for each physical CPU to track which VS (guest)
  * interrupt file IDs are in use.
  */

>> +
>> +unsigned int vgein_assign(struct vcpu *v)
>> +{
>> +    unsigned int vgein_id;
>> +    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
>> +    unsigned long *bmp = &vgein->bmp;
>> +    unsigned long flags;
>> +
>> +    spin_lock_irqsave(&vgein->lock, flags);
>> +    /*
>> +     * The vgein_id shouldn't be zero, as it will indicate that no guest
>> +     * external interrupt source is selected for VS-level external interrupts
>> +     * according to RISC-V priviliged spec:
>> +     *   Hypervisor Status Register (hstatus) in RISC-V priviliged spec:
>> +     *
>> +     *   The VGEIN (Virtual Guest External Interrupt Number) field selects
>> +     *   a guest external interrupt source for VS-level external interrupts.
>> +     *   VGEIN is a WLRL field that must be able to hold values between zero
>> +     *   and the maximum guest external interrupt number (known as GEILEN),
>> +     *   inclusive.
>> +     *   When VGEIN=0, no guest external interrupt source is selected for
>> +     *   VS-level external interrupts.
>> +     *
>> +     * So start to search from bit number 1.
>> +     */
>> +    vgein_id = find_next_zero_bit(bmp, vgein->geilen + 1, 1);
>> +
>> +    if ( vgein_id > vgein->geilen )
>> +        vgein_id = 0;
>> +    else
>> +        __set_bit(vgein_id, bmp);
>> +
>> +    spin_unlock_irqrestore(&vgein->lock, flags);
>> +
>> +#ifdef VGEIN_DEBUG
>> +    gprintk(XENLOG_DEBUG, "%s: %pv: vgein_id(%u), xen_cpu%d_bmp=%#lx\n",
>> +           __func__, v, vgein_id, v->processor, *bmp);
> 
> %d vs unsigned int again (and then yet again further down).
> 
>> +#endif
>> +
>> +    vcpu_guest_cpu_user_regs(v)->hstatus &= ~HSTATUS_VGEIN;
> 
> Is this needed when vgein_release() also does it?

Considering how ->hstatus is initialized I agree that we don't need that 
here.

> 
>> +    vcpu_guest_cpu_user_regs(v)->hstatus |=
>> +        MASK_INSR(vgein_id, HSTATUS_VGEIN);
>> +
>> +    return vgein_id;
>> +}
>> +
>> +void vgein_release(struct vcpu *v, unsigned int vgen_id)
>> +{
>> +    unsigned long flags;
>> +    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
>> +
>> +    if ( !vgen_id )
>> +        return;
>> +
>> +    spin_lock_irqsave(&vgein->lock, flags);
>> +     __clear_bit(vgen_id, &vgein->bmp);
>> +    spin_unlock_irqrestore(&vgein->lock, flags);
>> +
>> +#ifdef VGEIN_DEBUG
>> +    gprintk(XENLOG_DEBUG, "%s: vgein_id(%u), xen_cpu%d_bmp=%#lx\n",
>> +           __func__, vgen_id, v->processor, vgein->bmp);
>> +#endif
>> +
>> +    vcpu_guest_cpu_user_regs(v)->hstatus &= ~HSTATUS_VGEIN;
>>   }
> 
> Overall: How is one to review these two functions, when it's entirely
> unclear where they're going to be called from? Among other aspects it
> doesn't become clear what the behavior is going to be when
> vgein_assign() doesn't find an available ID. I've therefore only
> commented on mechanical aspects I noticed.

It is used in "[PATCH v2 14/26] xen/riscv: add very early virtual APLIC 
(vAPLIC) initialization support" from this patch series. I realize that 
was not obvious, sorry for the omission.

That said, I am planning to move all AIA-related logic out of 
vcpu_vaplic_init() and into continue_new_vcpu(), because some 
IMSIC-related operations require knowing which physical CPU the vCPU 
will be scheduled on, and moving the work there avoids redundant 
recalculation. As a result, vgein_assign() would no longer be called 
from vcpu_vaplic_init(), so I could drop this patch from the series 
entirely and reintroduce it when it is actually needed.

Alternatively, I can keep the patch and extend the commit message to 
explain the intended call site and how the return value will be handled.

Thanks.

~ Oleksii

Re: [PATCH v2 13/26] xen/riscv: introduce per-vCPU IMSIC state

[Oleksii Kurochko]

On 5/21/26 5:24 PM, Jan Beulich wrote:
> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>> --- a/xen/arch/riscv/imsic.c
>> +++ b/xen/arch/riscv/imsic.c
>> @@ -16,6 +16,7 @@
>>   #include <xen/errno.h>
>>   #include <xen/init.h>
>>   #include <xen/macros.h>
>> +#include <xen/sched.h>
>>   #include <xen/smp.h>
>>   #include <xen/spinlock.h>
>>   #include <xen/xvmalloc.h>
>> @@ -56,6 +57,16 @@ do {                            \
>>       csr_clear(CSR_SIREG, v);    \
>>   } while (0)
>>   
>> +unsigned int vcpu_guest_file_id(const struct vcpu *v)
>> +{
>> +    return ACCESS_ONCE(v->arch.vimsic_state->guest_file_id);
>> +}
>> +
>> +void imsic_set_guest_file_id(const struct vcpu *v, unsigned int guest_file_id)
> 
> Some people will demand that "const" be omitted in cases like this one, as
> it only works ...
> 
>> +{
>> +    ACCESS_ONCE(v->arch.vimsic_state->guest_file_id) = guest_file_id;
> 
> ... as long as vimsic_state is a pointer (and not a sub-structure).

Agree, it works only because of a pointer is used. Lets ommit const then 
here.

> 
>> @@ -312,6 +323,30 @@ static int imsic_parse_node(const struct dt_device_node *node,
>>       return 0;
>>   }
>>   
>> +int vcpu_imsic_init(struct vcpu *v)
>> +{
>> +    struct vimsic_state *imsic_state;
>> +
>> +    /* Allocate IMSIC context */
>> +    imsic_state = xvzalloc(struct vimsic_state);
>> +    if ( !imsic_state )
>> +        return -ENOMEM;
>> +
>> +    v->arch.vimsic_state = imsic_state;
>> +
>> +    /* Setup IMSIC context  */
>> +    rwlock_init(&imsic_state->vsfile_lock);
>> +
>> +    imsic_state->vsfile_pcpu = NR_CPUS;
>> +
>> +    return 0;
>> +}
>> +
>> +void vcpu_imsic_deinit(const struct vcpu *v)
>> +{
>> +    xvfree(v->arch.vimsic_state);
> 
> Better XVFREE(), for the function to be idempotent.

Agree it would be better.

> 
>> --- a/xen/arch/riscv/include/asm/imsic.h
>> +++ b/xen/arch/riscv/include/asm/imsic.h
>> @@ -11,6 +11,7 @@
>>   #ifndef ASM_RISCV_IMSIC_H
>>   #define ASM_RISCV_IMSIC_H
>>   
>> +#include <xen/rwlock.h>
>>   #include <xen/spinlock.h>
>>   #include <xen/stdbool.h>
>>   #include <xen/types.h>
>> @@ -61,7 +62,24 @@ struct imsic_config {
>>       spinlock_t lock;
>>   };
>>   
>> +struct vimsic_state {
>> +    /* IMSIC VS-file */
>> +    rwlock_t vsfile_lock;
>> +    /*
>> +     * (guest_file_id == 0) -> s/w IMSIC SW-file
>> +     * (guest_file_id > 0) -> h/w IMSIC VS-file
>> +     */
>> +    unsigned int guest_file_id;
>> +    /*
>> +     * (vsfile_pcpu >= 0) => h/w IMSIC VS-file
>> +     * (vsfile_pcpu == NR_CPUS) => s/w IMSIC SW-file
>> +     */
>> +    unsigned int vsfile_pcpu;
>> +};
> 
> In the comments, what does SW stand for? Not "software" I assume, as
> that's already expressed by s/w.

It is a typo and it should be VS-file.

Thanks.

~ Oleksii

Re: [PATCH v2 14/26] xen/riscv: add very early virtual APLIC (vAPLIC) initialization support

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> --- a/xen/arch/riscv/aplic.c
> +++ b/xen/arch/riscv/aplic.c
> @@ -295,6 +295,11 @@ static void cf_check aplic_set_irq_type(struct irq_desc *desc,
>      spin_unlock(&aplic.lock);
>  }
>  
> +static unsigned int cf_check aplic_irq_num(void)
> +{
> +    return aplic_info.num_irqs;
> +}
> +
>  static const hw_irq_controller aplic_xen_irq_type = {
>      .typename     = "aplic",
>      .startup      = aplic_irq_startup,
> @@ -309,6 +314,7 @@ static const struct intc_hw_operations aplic_ops = {
>      .host_irq_type       = &aplic_xen_irq_type,
>      .handle_interrupt    = aplic_handle_interrupt,
>      .set_irq_type        = aplic_set_irq_type,
> +    .irq_nums            = aplic_irq_num,

Hook handler names and respective field names would preferably match up. It's
unclear why the field uses some kind of plural(?), while the function uses
singular.

> --- a/xen/arch/riscv/domain.c
> +++ b/xen/arch/riscv/domain.c
> @@ -11,6 +11,7 @@
>  #include <asm/bitops.h>
>  #include <asm/cpufeature.h>
>  #include <asm/csr.h>
> +#include <asm/intc.h>
>  #include <asm/riscv_encoding.h>
>  #include <asm/vtimer.h>
>  
> @@ -155,14 +156,10 @@ int arch_vcpu_create(struct vcpu *v)
>      if ( (rc = vcpu_vtimer_init(v)) )
>          goto fail;
>  
> -    /*
> -     * As interrupt controller (IC) is not yet implemented,
> -     * return an error.
> -     *
> -     * TODO: Drop this once IC is implemented.
> -     */
> -    rc = -EOPNOTSUPP;
> -    goto fail;
> +    ASSERT(v->domain->arch.vintc->ops->vcpu_init);
> +
> +    if ( (rc = v->domain->arch.vintc->ops->vcpu_init(v)) )

I don't understand this model of the use of ASSERT(). As previously said
(more than once) - you'll crash anyway if any of the involved pointers is
NULL. If you really think an up-front check is better, then why would you
check only the leaf of the pointer chain, and not also vintc and ops?
(Once you do you'll then likely notice that there are more assertions
than actual code.)

> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/vaplic.h
> @@ -0,0 +1,34 @@
> +/* SPDX-License-Identifier: MIT */
> +/*
> + * xen/arch/riscv/vaplic.c
> + *
> + * Virtual RISC-V Advanced Platform-Level Interrupt Controller support
> + *
> + * Copyright (c) Microchip.
> + */
> +
> +#ifndef ASM__RISCV__VAPLIC_H
> +#define ASM__RISCV__VAPLIC_H
> +
> +#include <xen/kernel.h>
> +#include <xen/types.h>
> +
> +#include <asm/intc.h>
> +
> +struct domain;
> +
> +#define to_vaplic(d) container_of(d->arch.vintc, struct vaplic, vintc)

d wants parenthesizing here.

> --- a/xen/arch/riscv/intc.c
> +++ b/xen/arch/riscv/intc.c
> @@ -78,6 +78,13 @@ void intc_route_irq_to_xen(struct irq_desc *desc, unsigned int priority)
>      intc_set_irq_priority(desc, priority);
>  }
>  
> +unsigned int intc_irq_nums(void)
> +{
> +    ASSERT(intc_hw_ops && intc_hw_ops->irq_nums);
> +
> +    return intc_hw_ops->irq_nums();
> +}

You use this to set domains' properties. As indicated before, I view it as
wrong to do so for any domain, besides perhaps Dom0 / hwdom. If you want to
do so nevertheless, at the very least I'd expect something to be said about
such a decision in the description.

> --- /dev/null
> +++ b/xen/arch/riscv/vaplic.c
> @@ -0,0 +1,78 @@
> +/* SPDX-License-Identifier: MIT */
> +/*
> + * xen/arch/riscv/vaplic.c
> + *
> + * Virtual RISC-V Advanced Platform-Level Interrupt Controller support
> + *
> + * Copyright (c) Microchip.
> + * Copyright (c) Vates
> + */
> +
> +#include <xen/errno.h>
> +#include <xen/sched.h>
> +#include <xen/xvmalloc.h>
> +
> +#include <asm/aia.h>
> +#include <asm/imsic.h>
> +#include <asm/intc.h>
> +#include <asm/vaplic.h>
> +
> +#include "aplic-priv.h"
> +
> +#define VAPLIC_NUM_SOURCES 96
> +
> +static int cf_check vcpu_vaplic_init(struct vcpu *v)
> +{
> +    int rc = 0;

Nit: Pointless initializer.

> +    unsigned int vgein_id;
> +
> +    rc = vcpu_imsic_init(v);
> +    if ( rc )
> +        return rc;
> +
> +    if ( !(vgein_id = vgein_assign(v)) )
> +    {
> +        printk("Software interrupt files aren't supported\n");
> +        rc = -EOPNOTSUPP;
> +        goto fail;
> +    }
> +
> +    imsic_set_guest_file_id(v, vgein_id);
> +
> +    return rc;
> +
> + fail:
> +    vcpu_imsic_deinit(v);
> +
> +    return rc;
> +}
> +
> +static const struct vintc_ops vintc_ops = {
> +    .vcpu_init = vcpu_vaplic_init,
> +};
> +
> +int __init domain_vaplic_init(struct domain *d)

Why __init, and why is there no caller? Plus why is the vCPU-init a hook,
but the domain init is not? Either you mean to allow for other ICs, or
you you don't.

> +{
> +    struct vaplic *vaplic = xvzalloc(struct vaplic);
> +
> +    if ( !vaplic )
> +        return -ENOMEM;
> +
> +    d->arch.vintc = &vaplic->vintc;
> +    d->arch.vintc->ops = &vintc_ops;
> +
> +    vaplic->regs.domaincfg = APLIC_DOMAINCFG_IE | APLIC_DOMAINCFG_DM;
> +
> +    d->arch.vintc->irq_nums = min(intc_irq_nums(),
> +                                  VAPLIC_NUM_SOURCES + 0U);
> +
> +
> +    return 0;
> +}
> +
> +void __init domain_vaplic_deinit(struct domain *d)
> +{
> +    struct vaplic *vaplic = to_vaplic(d);
> +
> +    xvfree(vaplic);
> +}

And d->arch.vintc turns into a dangling pointer. The way you arrange data
types, you can't use XVFREE() here, but imo you really want to make sure
the function is idempotent.

Jan

Re: [PATCH v2 15/26] xen/riscv: introduce (de)initialization helpers for vINTC

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> --- a/xen/arch/riscv/intc.c
> +++ b/xen/arch/riscv/intc.c
> @@ -11,6 +11,7 @@
>  
>  #include <asm/aia.h>
>  #include <asm/intc.h>
> +#include <asm/vaplic.h>
>  
>  static const struct intc_hw_operations *__ro_after_init intc_hw_ops;
>  
> @@ -94,3 +95,38 @@ int __init make_intc_domU_node(struct kernel_info *kinfo)
>  
>      return -EOPNOTSUPP;
>  }
> +
> +int domain_vintc_init(struct domain *d)
> +{
> +    int ret = -EOPNOTSUPP;
> +    const enum intc_version ver = intc_hw_ops->info->hw_version;

Again - why would what the underlying hardware has control what all domains
get?

> +    switch ( ver )
> +    {
> +    case INTC_APLIC:
> +        ret = domain_vaplic_init(d);
> +        break;
> +
> +    default:
> +        printk("vintc (ver:%d) isn't implemented\n", ver);

If we take this path for whatever reason, ...

> +        break;
> +    }
> +
> +    return ret;
> +}
> +
> +void domain_vintc_deinit(struct domain *d)
> +{
> +    const enum intc_version ver = intc_hw_ops->info->hw_version;
> +
> +    switch ( ver )
> +    {
> +    case INTC_APLIC:
> +        domain_vaplic_deinit(d);
> +        break;
> +
> +    default:
> +        printk("vintc (ver:%d) isn't implemented\n", ver);

... we're also going to take this path (very quickly afterwards), just to
get the same message twice without it being clear why it appears twice.

Jan

Re: [PATCH v2 16/26] xen/riscv: create APLIC DT node for guest domains

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> --- a/xen/arch/riscv/imsic.c
> +++ b/xen/arch/riscv/imsic.c
> @@ -14,6 +14,7 @@
>  #include <xen/cpumask.h>
>  #include <xen/device_tree.h>
>  #include <xen/errno.h>
> +#include <xen/fdt-kernel.h>
>  #include <xen/init.h>
>  #include <xen/macros.h>
>  #include <xen/sched.h>
> @@ -522,3 +523,9 @@ int __init imsic_init(const struct dt_device_node *node)
>  
>      return rc;
>  }
> +
> +int __init vimsic_make_domu_dt_node(struct kernel_info *kinfo,
> +                                    unsigned int *phandle)
> +{
> +    return -EOPNOTSUPP;
> +}

This, I assume, is going to be filled properly by the next patch.

> --- a/xen/arch/riscv/vaplic.c
> +++ b/xen/arch/riscv/vaplic.c
> @@ -9,6 +9,8 @@
>   */
>  
>  #include <xen/errno.h>
> +#include <xen/fdt-kernel.h>
> +#include <xen/libfdt/libfdt.h>
>  #include <xen/sched.h>
>  #include <xen/xvmalloc.h>
>  
> @@ -19,8 +21,11 @@
>  
>  #include "aplic-priv.h"
>  
> +#define VAPLIC_COMPATIBLE "riscv,aplic"
>  #define VAPLIC_NUM_SOURCES 96
>  
> +#define FDT_VAPLIC_INT_CELLS 2
> +
>  static int cf_check vcpu_vaplic_init(struct vcpu *v)
>  {
>      int rc = 0;
> @@ -47,6 +52,73 @@ static int cf_check vcpu_vaplic_init(struct vcpu *v)
>      return rc;
>  }
>  
> +static int __init cf_check vaplic_make_domu_dt_node(struct kernel_info *kinfo)

Again - why __init here and ...

> +{
> +    int res = 0;
> +    void *fdt = kinfo->fdt;
> +    unsigned int msi_parent_phandle;
> +    char vaplic_name[128];
> +    paddr_t aplic_addr = GUEST_APLIC_S_BASE;
> +    paddr_t aplic_size = APLIC_SIZE(kinfo->bd.d->max_vcpus);
> +    const __be32 reg[] = {
> +        cpu_to_be32(aplic_addr >> 32),
> +        cpu_to_be32(aplic_addr),
> +        cpu_to_be32(aplic_size >> 32),
> +        cpu_to_be32(aplic_size),
> +    };
> +    struct vintc *vintc = kinfo->bd.d->arch.vintc;
> +
> +    res = snprintf(vaplic_name, sizeof(vaplic_name), "/soc/aplic@%x",
> +                   GUEST_APLIC_S_BASE);
> +    if ( res >= sizeof(vaplic_name) )
> +    {
> +        dprintk(XENLOG_DEBUG, "vaplic name is truncated\n");
> +        return -ENOBUFS;
> +    }
> +
> +    res = vimsic_make_domu_dt_node(kinfo, &msi_parent_phandle);
> +    if ( res )
> +        return res;
> +
> +    res = fdt_begin_node(fdt, vaplic_name);
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property_cell(fdt, "#interrupt-cells", FDT_VAPLIC_INT_CELLS);
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property(fdt, "reg", reg, sizeof(reg));
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property_cell(fdt, "riscv,num-sources", vintc->irq_nums);
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property(fdt, "interrupt-controller", NULL, 0);
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property_string(fdt, "compatible", VAPLIC_COMPATIBLE);
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property_cell(fdt, "msi-parent", msi_parent_phandle);
> +    if ( res )
> +        return res;
> +
> +    res = fdt_property_cell(fdt, "phandle", kinfo->phandle_intc);
> +    if ( res )
> +        return res;
> +
> +    return fdt_end_node(fdt);
> +}
> +
> +static const struct vintc_init_ops __initdata init_ops = {

... __initdata here? If you really want to have the option of moving
domain creation stuff to .init.* when only dom0less is configured,
then a proper abstraction is needed, along the lines of
init_or_livepatch.

> @@ -60,13 +132,14 @@ int __init domain_vaplic_init(struct domain *d)
>  
>      d->arch.vintc = &vaplic->vintc;
>      d->arch.vintc->ops = &vintc_ops;
> +    d->arch.vintc->init_ops = &init_ops;
>  
> -    vaplic->regs.domaincfg = APLIC_DOMAINCFG_IE | APLIC_DOMAINCFG_DM;
> +    vaplic->regs.domaincfg = APLIC_DOMAINCFG_IE | APLIC_DOMAINCFG_DM |
> +                             APLIC_DOMAINCFG_RO80;

This looks unrelated. I don't mind it being done here, but then it
wants mentioning in the description. Or maybe I simply don't understand
what this is about.

Jan

Re: [PATCH v2 17/26] xen/riscv: generate IMSIC DT node for guest domains

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> Guests using the IMSIC interrupt controller require a corresponding
> Device Tree description.
> 
> Add support for generating an IMSIC node when building the guest DT.
> This allows guests to discover and use the IMSIC interrupt controller.
> 
> Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
> Changes in v2:
>  - s/imsic_make_reg_property/guest_imsic_make_reg_property.
>  - s/imsic_set_interrupt_extended_prop/guest_imsic_set_interrupt_extended_prop.
>  - Use initalizer for regs[] array in imsic_make_reg_property().
>  - Move buf[] insde the for() loop.
>  - Correct check of returned phandle.
>  - Drop local variable len.
>  - /s/XVFREE/xvfree in imsic_set_interrupt_extended_prop().
>  - Drop initializer for local variable data.
>  - s/uint32_t/unsinged int for pos and cpu in imsic_set_interrupt_extended_prop().
>  - Drop next_phandle as it is now in common code.
>  - Introduce vcpu_imsic_deinit.
>  - Refactor vimsic_make_domu_dt_node() to avoid usage of host IMSIC dt node.
> ---
>  xen/arch/riscv/imsic.c                    | 127 +++++++++++++++++++++-
>  xen/arch/riscv/include/asm/guest-layout.h |   2 +
>  2 files changed, 128 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/riscv/imsic.c b/xen/arch/riscv/imsic.c
> index ceea6778d9dc..19cbacdf96e1 100644
> --- a/xen/arch/riscv/imsic.c
> +++ b/xen/arch/riscv/imsic.c
> @@ -13,9 +13,12 @@
>  #include <xen/const.h>
>  #include <xen/cpumask.h>
>  #include <xen/device_tree.h>
> +#include <xen/domain.h>
>  #include <xen/errno.h>
> +#include <xen/fdt-domain-build.h>
>  #include <xen/fdt-kernel.h>
>  #include <xen/init.h>
> +#include <xen/libfdt/libfdt.h>
>  #include <xen/macros.h>
>  #include <xen/sched.h>
>  #include <xen/smp.h>
> @@ -35,6 +38,11 @@ static struct imsic_config imsic_cfg = {
>      .lock = SPIN_LOCK_UNLOCKED,
>  };
>  
> +static unsigned int __ro_after_init guest_num_msis;

How come this is __ro_after_init, when it's ...

> @@ -291,6 +299,11 @@ static int imsic_parse_node(const struct dt_device_node *node,
>          return -ENOENT;
>      }
>  
> +    if ( dt_property_read_u32(node, "riscv,num-guest-ids", &tmp) )
> +        guest_num_msis = tmp;
> +    else
> +        guest_num_msis = imsic_cfg.nr_ids;

... written by a non-__init function? Plus are you again inheriting a host
property into guests without saying why?

> @@ -524,8 +537,120 @@ int __init imsic_init(const struct dt_device_node *node)
>      return rc;
>  }
>  
> +static int __init guest_imsic_make_reg_property(struct domain *d, void *fdt)

Same question again as to __init throughout here.

> +{
> +    paddr_t base_addr = GUEST_IMSIC_S_BASE;

So you make a local variable for a constant, ...

> +    __be32 regs[4] = {
> +        cpu_to_be32(base_addr >> 32),
> +        cpu_to_be32(base_addr),
> +        cpu_to_be32((IMSIC_MMIO_PAGE_SZ * d->max_vcpus) >> 32),
> +        cpu_to_be32(IMSIC_MMIO_PAGE_SZ * d->max_vcpus),

... but this non-constant expression is spelled out twice.

> +static int __init guest_imsic_set_interrupt_extended_prop(struct domain *d,
> +                                                          void *fdt)
> +{
> +    unsigned int cpu, pos = 0;
> +    uint32_t phandle;
> +    uint32_t *irq_ext;

Doesn't this want to be __be32, seeing ...

> +    int res;
> +
> +    irq_ext = xvzalloc_array(uint32_t, d->max_vcpus * 2);
> +    if ( !irq_ext )
> +        return -ENOMEM;
> +
> +    for ( cpu = 0; cpu < d->max_vcpus; cpu++ )
> +    {
> +        char buf[64];
> +
> +        snprintf(buf, sizeof(buf), "/cpus/cpu@%u/interrupt-controller", cpu);
> +        phandle = fdt_get_phandle(fdt, fdt_path_offset(fdt, buf));
> +
> +        if ( !phandle )
> +        {
> +            res = -ENODEV;
> +            goto out;
> +        }
> +
> +        irq_ext[pos++] = cpu_to_be32(phandle);
> +        irq_ext[pos++] = cpu_to_be32(IRQ_S_EXT);

... this?

Also, just like "buf", "phandle" can be local to this loop's body.

> --- a/xen/arch/riscv/include/asm/guest-layout.h
> +++ b/xen/arch/riscv/include/asm/guest-layout.h
> @@ -5,6 +5,8 @@
>  
>  #define GUEST_APLIC_S_BASE 0xd000000
>  
> +#define GUEST_IMSIC_S_BASE 0x28000000
> +
>  #define GUEST_RAM_BANKS   2

Is this going to become an unannotated collection of (seemingly) random
numbers?

Jan

Re: [PATCH v2 20/26] xen/riscv: add missing APLIC register offsets, masks to asm/aplic.h.

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> --- a/xen/arch/riscv/include/asm/aplic.h
> +++ b/xen/arch/riscv/include/asm/aplic.h
> @@ -15,6 +15,11 @@
>  
>  #include <asm/imsic.h>
>  
> +#define APLIC_REG_OFFSET_MASK   0x3fff

This I can see this as wanting to live separately. Yet ...

> +#define APLIC_TARGET_IPRIO_MASK 0xff
> +#define APLIC_TARGET_GUEST_IDX_SHIFT 12
> +#define APLIC_TARGET_EIID_MASK  0x7ff

... what (set of) register(s) do these apply to? Perhaps ...

> @@ -26,6 +31,36 @@
>  #define APLIC_SOURCECFG_SM_LEVEL_HIGH   0x6
>  #define APLIC_SOURCECFG_SM_LEVEL_LOW    0x7
>  
> +#define APLIC_DOMAINCFG         0x0000
> +#define APLIC_SOURCECFG_BASE    0x0004
> +#define APLIC_SOURCECFG_LAST    0x0ffc
> +
> +#define APLIC_SMSICFGADDR       0x1bc8
> +#define APLIC_SMSICFGADDRH      0x1bcc
> +
> +#define APLIC_SETIP_BASE        0x1c00
> +#define APLIC_SETIP_LAST        0x1c7c
> +#define APLIC_SETIPNUM          0x1cdc
> +
> +#define APLIC_CLRIP_BASE        0x1d00
> +#define APLIC_CLRIP_LAST        0x1d7c
> +#define APLIC_CLRIPNUM          0x1ddc
> +
> +#define APLIC_SETIE_BASE        0x1e00
> +#define APLIC_SETIE_LAST        0x1e7c
> +#define APLIC_SETIENUM          0x1edc
> +
> +#define APLIC_CLRIE_BASE        0x1f00
> +#define APLIC_CLRIE_LAST        0x1f7c
> +#define APLIC_CLRIENUM          0x1fdc
> +
> +#define APLIC_SETIPNUM_LE       0x2000
> +
> +#define APLIC_GENMSI            0x3000
> +
> +#define APLIC_TARGET_BASE       0x3004
> +#define APLIC_TARGET_LAST       0x3ffc

... these? And then is ...

>  #define APLIC_TARGET_HART_IDX_SHIFT 18

... this also covering some part of them? Can't they (a) live together and (b)
have some kind of connection to what they apply to?

And then why is there again a mix of *_SHIFT and *_MASK?

Jan

Re: [PATCH v2 19/26] xen/riscv: implement IRQ routing for device passthrough

[Jan Beulich]

On 08.05.2026 16:43, Oleksii Kurochko wrote:
> --- /dev/null
> +++ b/xen/arch/riscv/device.c
> @@ -0,0 +1,108 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +
> +#include <xen/device_tree.h>
> +#include <xen/errno.h>
> +#include <xen/iocap.h>
> +#include <xen/rangeset.h>
> +#include <xen/sched.h>
> +
> +#include <asm/intc.h>
> +
> +int map_irq_to_domain(struct domain *d, unsigned int irq,
> +                      bool need_mapping, const char *devname)
> +{
> +    int res;
> +
> +    res = irq_permit_access(d, irq);
> +    if ( res )
> +    {
> +        printk(XENLOG_ERR "Unable to permit to %pd access to IRQ %u\n", d, irq);

Nit: Drop the first "to"?

> +        return res;
> +    }
> +
> +    if ( need_mapping )
> +    {
> +        /*
> +         * Checking the return of vintc_reserve_virq is not
> +         * necessary. It should not fail except when we try to map
> +         * the IRQ twice. This can legitimately happen if the IRQ is shared.
> +         */
> +        vintc_reserve_virq(d, irq);
> +
> +        res = route_irq_to_guest(d, irq, irq, devname);
> +        if ( res < 0 )
> +        {
> +            printk(XENLOG_ERR "Unable to map IRQ%u to %pd\n", irq, d);
> +            return res;
> +        }
> +    }
> +
> +    dt_dprintk("  - IRQ: %u\n", irq);
> +
> +    return 0;
> +}
> +
> +/*
> + * map_device_irqs_to_domain retrieves the interrupts configuration from
> + * a device tree node and maps those interrupts to the target domain.
> + *
> + * Returns:
> + *   < 0 error
> + *   0   success
> + */
> +int map_device_irqs_to_domain(struct domain *d,
> +                              struct dt_device_node *dev,
> +                              bool need_mapping,
> +                              struct rangeset *irq_ranges)
> +{
> +    unsigned int i, nirq;
> +    int res, irq;
> +    struct dt_raw_irq rirq;

Move the latter three variables to the loop's scope and ...

> +    nirq = dt_number_of_irq(dev);

... make this the variable's initializer?

> +    /* Give permission and map IRQs */
> +    for ( i = 0; i < nirq; i++ )
> +    {
> +        res = dt_device_get_raw_irq(dev, i, &rirq);
> +        if ( res )
> +        {
> +            printk(XENLOG_ERR "Unable to retrieve irq %u for %s\n",
> +                   i, dt_node_full_name(dev));
> +            return res;
> +        }
> +
> +        /*
> +         * Don't map IRQ that have no physical meaning
> +         * ie: IRQ whose controller is not APLIC/IMSIC/PLIC.
> +         */
> +        if ( rirq.controller != dt_interrupt_controller )
> +        {
> +            dt_dprintk("irq %u not connected to primary controller."
> +                       "Connected to %s\n", i,
> +                       dt_node_full_name(rirq.controller));
> +            continue;
> +        }
> +
> +        irq = platform_get_irq(dev, i);
> +        if ( irq < 0 )
> +        {
> +            printk("Unable to get irq %u for %s\n", i, dt_node_full_name(dev));
> +            return irq;
> +        }
> +
> +        res = map_irq_to_domain(d, irq, need_mapping, dt_node_name(dev));
> +        if ( res )
> +            return res;
> +
> +
> +        /*
> +         * At the moment there is only one user of map_device_irqs_to_domain()
> +         * for RISC-V which calls it irq_ranges == NULL.
> +         */
> +        if ( irq_ranges )
> +            return -EOPNOTSUPP;

Why is this checked last, and inside the loop (when it's loop invariant)?

> --- a/xen/arch/riscv/include/asm/intc.h
> +++ b/xen/arch/riscv/include/asm/intc.h
> @@ -13,8 +13,11 @@ enum intc_version {
>  };
>  
>  struct cpu_user_regs;
> +struct domain;

I can spot why this is needed, but ...

> +struct dt_device_node;
>  struct irq_desc;
>  struct kernel_info;
> +struct rangeset;
>  struct vcpu;

... I'm at a loss to explain the need for these two additions.

> --- a/xen/arch/riscv/include/asm/setup.h
> +++ b/xen/arch/riscv/include/asm/setup.h
> @@ -5,6 +5,10 @@
>  
>  #include <xen/types.h>
>  
> +struct domain;
> +struct dt_device_node;
> +struct rangeset;

Same here - why would they be needed when you make no other changes
to this header?

> --- a/xen/arch/riscv/intc.c
> +++ b/xen/arch/riscv/intc.c
> @@ -7,7 +7,9 @@
>  #include <xen/init.h>
>  #include <xen/irq.h>
>  #include <xen/lib.h>
> +#include <xen/sched.h>
>  #include <xen/spinlock.h>
> +#include <xen/xvmalloc.h>
>  
>  #include <asm/aia.h>
>  #include <asm/intc.h>
> @@ -86,6 +88,22 @@ unsigned int intc_irq_nums(void)
>      return intc_hw_ops->irq_nums();
>  }
>  
> +int intc_route_irq_to_guest(struct irq_desc *desc,
> +                            unsigned int priority)
> +{
> +    ASSERT(spin_is_locked(&desc->lock));
> +
> +    ASSERT(intc_hw_ops->guest_irq_type);
> +
> +    desc->handler = intc_hw_ops->guest_irq_type;
> +    set_bit(_IRQ_GUEST, &desc->status);

Is desc->status accessed anywhere without holding desc->lock? If not,
__set_bit() or simply |= ?

> @@ -112,6 +130,14 @@ int domain_vintc_init(struct domain *d)
>          break;
>      }
>  
> +    if ( !ret )
> +    {
> +        d->arch.vintc->allocated_irqs =
> +            xvzalloc_array(unsigned long, BITS_TO_LONGS(d->arch.vintc->irq_nums));
> +        if ( !d->arch.vintc->allocated_irqs )
> +            ret = -ENOMEM;
> +    }
> +
>      return ret;
>  }
>  
> @@ -129,4 +155,14 @@ void domain_vintc_deinit(struct domain *d)
>          printk("vintc (ver:%d) isn't implemented\n", ver);
>          break;
>      }
> +
> +    xvfree(d->arch.vintc->allocated_irqs);
> +}

XVFREE()

> +bool vintc_reserve_virq(const struct domain *d, unsigned int virq)
> +{
> +    if ( virq >= d->arch.vintc->irq_nums )
> +        return false;
> +
> +    return !test_and_set_bit(virq, d->arch.vintc->allocated_irqs);
>  }

As to function / field naming: You don't look to be allocating IRQs. So
is there a reason the field name gives the impression of allocation?
Simply s/allocated/used/ or some such?

> @@ -221,3 +239,160 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq)
>      spin_unlock(&desc->lock);
>      irq_exit();
>  }
> +
> +static inline struct irq_guest *irq_get_guest_info(struct irq_desc *desc)
> +{
> +    ASSERT(spin_is_locked(&desc->lock));
> +    ASSERT(test_bit(_IRQ_GUEST, &desc->status));
> +    ASSERT(desc->action != NULL);
> +
> +    return desc->action->dev_id;
> +}
> +
> +static inline struct domain *irq_get_domain(struct irq_desc *desc)
> +{
> +    return irq_get_guest_info(desc)->d;
> +}
> +
> +void release_irq(unsigned int irq, const void *dev_id)
> +{
> +    struct irq_desc *desc;
> +    unsigned long flags;
> +    struct irqaction *action, **action_ptr;
> +
> +    desc = irq_to_desc(irq);
> +
> +    spin_lock_irqsave(&desc->lock,flags);
> +
> +    action_ptr = &desc->action;
> +#ifdef CONFIG_IRQ_HAS_MULTIPLE_ACTION
> +    for ( ;; )
> +    {
> +        action = *action_ptr;
> +        if ( !action )
> +        {
> +            printk(XENLOG_WARNING "Trying to free already-free IRQ %u\n", irq);
> +            spin_unlock_irqrestore(&desc->lock, flags);
> +            return;
> +        }
> +
> +        if ( action->dev_id == dev_id )
> +            break;
> +
> +        action_ptr = &action->next;
> +    }
> +
> +    /* Found it - remove it from the action list */
> +    *action_ptr = action->next;
> +#else
> +    action = *action_ptr;
> +#endif
> +
> +    /* If this was the last action, shut down the IRQ */
> +    if ( !desc->action )
> +    {
> +        desc->handler->shutdown(desc);
> +        clear_bit(_IRQ_GUEST, &desc->status);
> +    }
> +
> +    spin_unlock_irqrestore(&desc->lock,flags);
> +
> +    /* Wait to make sure it's not being used on another CPU */
> +    do { smp_mb(); } while ( test_bit(_IRQ_INPROGRESS, &desc->status) );
> +
> +    if ( action->free_on_release )
> +        xvfree(action);

When !IRQ_HAS_MULTIPLE_ACTION desc->action becomes a dangling pointer here.

> +/* Route an IRQ to a specific guest */
> +int route_irq_to_guest(struct domain *d, unsigned int virq,
> +                       unsigned int irq, const char *devname)
> +{
> +    struct irqaction *action;
> +    struct irq_guest *info;
> +    struct irq_desc *desc;
> +    unsigned long flags;
> +    int retval = 0;
> +
> +    desc = irq_to_desc(irq);
> +
> +    action = xvmalloc(struct irqaction);
> +    if ( !action )
> +        return -ENOMEM;

This is freed by release_irq(), but ...

> +    info = xvmalloc(struct irq_guest);
> +    if ( !info )

... where is the (non-error-path) freeing of this?

> +    {
> +        xvfree(action);
> +        return -ENOMEM;
> +    }
> +
> +    info->d = d;
> +    info->virq = virq;
> +
> +    action->dev_id = info;
> +    action->name = devname;
> +    action->free_on_release = 1;

true

> +    spin_lock_irqsave(&desc->lock, flags);
> +
> +    /*
> +     * If the IRQ is already used by someone
> +     *  - If it's the same domain -> Xen doesn't need to update the IRQ desc.
> +     *  For safety check if we are not trying to assign the IRQ to a
> +     *  different vIRQ.
> +     *  - Otherwise -> For now, don't allow the IRQ to be shared between
> +     *  Xen and domains.
> +     */
> +    if ( desc->action != NULL )
> +    {
> +        if ( test_bit(_IRQ_GUEST, &desc->status) )
> +        {
> +            struct domain *ad = irq_get_domain(desc);
> +
> +            if ( d != ad )
> +            {
> +                printk(XENLOG_G_ERR "IRQ %u is already used by domain %u\n",
> +                       irq, ad->domain_id);
> +                retval = -EBUSY;
> +            }
> +            else if ( irq_get_guest_info(desc)->virq != virq )
> +            {
> +                printk(XENLOG_G_ERR
> +                       "d%u: IRQ %u is already assigned to vIRQ %u\n",
> +                       d->domain_id, irq, irq_get_guest_info(desc)->virq);

Please can you get used to using %pd?

> +                retval = -EBUSY;
> +            }
> +        }
> +        else
> +        {
> +            printk(XENLOG_G_ERR "IRQ %u is already used by Xen\n", irq);
> +            retval = -EBUSY;
> +        }
> +        goto out;
> +    }
> +
> +    retval = _setup_irq(desc, 0, action);
> +    if ( retval )
> +        goto out;
> +
> +    retval = intc_route_irq_to_guest(desc, IRQ_NO_PRIORITY);
> +
> +    spin_unlock_irqrestore(&desc->lock, flags);
> +
> +    if ( retval )
> +    {
> +        release_irq(desc->irq, info);

Is de-referencing desc legitimate / race free with desc->lock not held?

Jan

Re: [PATCH v2 14/26] xen/riscv: add very early virtual APLIC (vAPLIC) initialization support

[Oleksii Kurochko]

On 6/3/26 4:54 PM, Jan Beulich wrote:
> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>> --- a/xen/arch/riscv/aplic.c
>> +++ b/xen/arch/riscv/aplic.c
>> @@ -295,6 +295,11 @@ static void cf_check aplic_set_irq_type(struct irq_desc *desc,
>>       spin_unlock(&aplic.lock);
>>   }
>>   
>> +static unsigned int cf_check aplic_irq_num(void)
>> +{
>> +    return aplic_info.num_irqs;
>> +}
>> +
>>   static const hw_irq_controller aplic_xen_irq_type = {
>>       .typename     = "aplic",
>>       .startup      = aplic_irq_startup,
>> @@ -309,6 +314,7 @@ static const struct intc_hw_operations aplic_ops = {
>>       .host_irq_type       = &aplic_xen_irq_type,
>>       .handle_interrupt    = aplic_handle_interrupt,
>>       .set_irq_type        = aplic_set_irq_type,
>> +    .irq_nums            = aplic_irq_num,
> 
> Hook handler names and respective field names would preferably match up. It's
> unclear why the field uses some kind of plural(?), while the function uses
> singular.

I will do s/aplic_irq_num/aplic_irq_nums.

> 
>> --- a/xen/arch/riscv/domain.c
>> +++ b/xen/arch/riscv/domain.c
>> @@ -11,6 +11,7 @@
>>   #include <asm/bitops.h>
>>   #include <asm/cpufeature.h>
>>   #include <asm/csr.h>
>> +#include <asm/intc.h>
>>   #include <asm/riscv_encoding.h>
>>   #include <asm/vtimer.h>
>>   
>> @@ -155,14 +156,10 @@ int arch_vcpu_create(struct vcpu *v)
>>       if ( (rc = vcpu_vtimer_init(v)) )
>>           goto fail;
>>   
>> -    /*
>> -     * As interrupt controller (IC) is not yet implemented,
>> -     * return an error.
>> -     *
>> -     * TODO: Drop this once IC is implemented.
>> -     */
>> -    rc = -EOPNOTSUPP;
>> -    goto fail;
>> +    ASSERT(v->domain->arch.vintc->ops->vcpu_init);
>> +
>> +    if ( (rc = v->domain->arch.vintc->ops->vcpu_init(v)) )
> 
> I don't understand this model of the use of ASSERT(). As previously said
> (more than once) - you'll crash anyway if any of the involved pointers is
> NULL. If you really think an up-front check is better, then why would you
> check only the leaf of the pointer chain, and not also vintc and ops?
> (Once you do you'll then likely notice that there are more assertions
> than actual code.)

Let me show an example. For the test I will comment ASSERT() here + make 
->vcpu_init = NULL. Then the following dump will be occurred because of 
NULL pointer dereference:

(XEN) scause    : 000000000000000c Unhandled exception[Instruction Page 
Fault]
(XEN) htval     : 0000000000000000 htinst    : 0000000000000000
(XEN) hedeleg   : 0000000000000000 hideleg   : 0000000000000000
(XEN) hstatus   : 0000000200000000 [ ]
(XEN) hgatp     : 0000000000000000
(XEN) hstateen0 : 0000000000000000
(XEN) stvec     : ffffffffc00397f0 vstvec    : 0000000000000000
(XEN) sepc      : 0000000000000000 vsepc     : 0000000000000000
(XEN) stval     : 0000000000000000 vstval    : 0000000000000000
(XEN) status    : 0000000200000120 vsstatus  : 0000000a00000000
(XEN) satp      : 80000000000806c5
(XEN) vscause   : 0000000000000000 [Instruction Address Misaligned]
(XEN) ra  : ffffffffc0038f92 sp  : ffffffffc00a6580
(XEN) gp  : 0000000000000000 tp  : ffffffffc005e9c0
(XEN) t0  : ffffffffc04c9f78 t1  : 0000000052464e43
(XEN) t2  : 0000000000000000 s0  : ffffffffc00a65a0
(XEN) s1  : 000000323feaa000 a0  : 000000323feaa000
(XEN) a1  : 000000321a0010a8 a2  : 0000000000000000
(XEN) a3  : ffffffffc005c978 a4  : 0000000000000000
(XEN) a5  : 0000000000000000 a6  : 0000000000000000
(XEN) a7  : 000000321a0010a0 s2  : 0000000000000000
(XEN) s3  : 0000000000000000 s4  : 0000000000000000
(XEN) s5  : 0000000000000000 s6  : ffffffffc00a6638
(XEN) s7  : ffffffffc00a6638 s8  : ffffffffc00a6658
(XEN) s9  : ffffffffffffe000 s10 : 00000032105f4328
(XEN) s11 : 0000000000000000 t3  : 0000000000000001
(XEN) t4  : 0000000000000000 t5  : 0000000000080000
(XEN) t6  : 0000000000000001

So sepc = 0x0 => the CPU attempted to fetch and execute an instruction 
at address NULL.

stval = 0x0 => Faulting address = 0x0 (confirms null fetch).

During writing of that I reliased that I have also ra register which 
will tell where the bad call came from and then using 
addr2line/nm/objdump/whatever it still could be recoverable a place 
where NULL dereference happened in this case:

$ addr2line -e xen/xen-syms 0xffffffffc0038f92
/build/xen/arch/riscv/domain.c:218

So I agree now that such ASSERT should be dropped and shouldn't be used 
in such cases. I will drop it here and in the similar places where I 
added ASSERT for the reason as it is hard to identify a place where 
fault happened.

>> +unsigned int intc_irq_nums(void)
>> +{
>> +    ASSERT(intc_hw_ops && intc_hw_ops->irq_nums);
>> +
>> +    return intc_hw_ops->irq_nums();
>> +}
> 
> You use this to set domains' properties. As indicated before, I view it as
> wrong to do so for any domain, besides perhaps Dom0 / hwdom. If you want to
> do so nevertheless, at the very least I'd expect something to be said about
> such a decision in the description.

If you could explain how this is expected to work for non-Dom0/hwdom 
domains, I would consider reworking it.

Basically, I don't understand how the following scenario is supposed to 
work. Let's say the host interrupt controller can manage 15 interrupts, 
while the guest interrupt controller supports only 7. If we want to pass 
through 8 devices to the guest, what should happen in that case? Should 
Xen simply report that the 8th device cannot be passed through because 
the guest supports only 7 IRQs?

Another concern is related to 1:1 IRQ mapping. Suppose I want to pass 
through a UART device whose IRQ number is typically greater than 10. In 
that case, it seems Xen would again have to report that the device 
cannot be passed through because its interrupt number exceeds the number 
of IRQs supported by the guest interrupt controller. This could be 
addressed by introducing a non-1:1 IRQ mapping between the host and 
guest, but the current dom0less codebase appears to assume a 1:1 IRQ 
mapping (unless I am mistaken).

Considering that virtual interrupt controllers use the maximum possible 
number of interrupts supported by the interrupt controller, the concerns 
mentioned above are unlikely to arise for a long time, if ever, unless 
support for features such as migration is introduced. Therefore, I think 
it would be reasonable to remove intc_irq_nums() and avoid using it to 
initialize virtual interrupts or domain properties.

>> +
>> +static const struct vintc_ops vintc_ops = {
>> +    .vcpu_init = vcpu_vaplic_init,
>> +};
>> +
>> +int __init domain_vaplic_init(struct domain *d)
> 
> Why __init, and why is there no caller? 

The caller is in follow-up patch. I will add that to commit message.

Considering that domain_vintc_init() isn't __init from where 
domain_vaplic_init() is called then __init should be dropped here. I 
will do that.

Plus why is the vCPU-init a hook,
> but the domain init is not? Either you mean to allow for other ICs, or
> you you don't.

IIUC your question domain_vaplic_init() ins't a hook because vaplic 
structure is allocated dynamically so vintc, vintc->ops and/or 
vintc->init_ops aren't initialized at the moment when vintc->{ops or 
init_ops}->domain_vaplic_init() is used in domain_vintc_init() (which is 
introduced in the follow up patch).

> 
>> +{
>> +    struct vaplic *vaplic = xvzalloc(struct vaplic);
>> +
>> +    if ( !vaplic )
>> +        return -ENOMEM;
>> +
>> +    d->arch.vintc = &vaplic->vintc;
>> +    d->arch.vintc->ops = &vintc_ops;
>> +
>> +    vaplic->regs.domaincfg = APLIC_DOMAINCFG_IE | APLIC_DOMAINCFG_DM;
>> +
>> +    d->arch.vintc->irq_nums = min(intc_irq_nums(),
>> +                                  VAPLIC_NUM_SOURCES + 0U);
>> +
>> +
>> +    return 0;
>> +}
>> +
>> +void __init domain_vaplic_deinit(struct domain *d)
>> +{
>> +    struct vaplic *vaplic = to_vaplic(d);
>> +
>> +    xvfree(vaplic);
>> +}
> 
> And d->arch.vintc turns into a dangling pointer. The way you arrange data
> types, you can't use XVFREE() here, but imo you really want to make sure
> the function is idempotent.

I will do then:

void domain_vaplic_deinit(struct domain *d)
{
     struct vaplic *vaplic;

     if ( !d->arch.vintc )
         return;

     vaplic = to_vaplic(d);
     d->arch.vintc = NULL;
     xvfree(vaplic);
}

Thanks.

~ Oleksii

Re: [PATCH v2 15/26] xen/riscv: introduce (de)initialization helpers for vINTC

[Oleksii Kurochko]

On 6/3/26 5:00 PM, Jan Beulich wrote:
> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>> --- a/xen/arch/riscv/intc.c
>> +++ b/xen/arch/riscv/intc.c
>> @@ -11,6 +11,7 @@
>>   
>>   #include <asm/aia.h>
>>   #include <asm/intc.h>
>> +#include <asm/vaplic.h>
>>   
>>   static const struct intc_hw_operations *__ro_after_init intc_hw_ops;
>>   
>> @@ -94,3 +95,38 @@ int __init make_intc_domU_node(struct kernel_info *kinfo)
>>   
>>       return -EOPNOTSUPP;
>>   }
>> +
>> +int domain_vintc_init(struct domain *d)
>> +{
>> +    int ret = -EOPNOTSUPP;
>> +    const enum intc_version ver = intc_hw_ops->info->hw_version;
> 
> Again - why would what the underlying hardware has control what all domains
> get?

If host uses AIA (APLIC/IMSIC) compatible controller then guest should 
use virtual AIA compatible controller, shouldn't it?

I don't think that it is a case when host uses PLIC interrupt controller 
but guests are going to use virtual APLIC.

> 
>> +    switch ( ver )
>> +    {
>> +    case INTC_APLIC:
>> +        ret = domain_vaplic_init(d);
>> +        break;
>> +
>> +    default:
>> +        printk("vintc (ver:%d) isn't implemented\n", ver);
> 
> If we take this path for whatever reason, ...
> 
>> +        break;
>> +    }
>> +
>> +    return ret;
>> +}
>> +
>> +void domain_vintc_deinit(struct domain *d)
>> +{
>> +    const enum intc_version ver = intc_hw_ops->info->hw_version;
>> +
>> +    switch ( ver )
>> +    {
>> +    case INTC_APLIC:
>> +        domain_vaplic_deinit(d);
>> +        break;
>> +
>> +    default:
>> +        printk("vintc (ver:%d) isn't implemented\n", ver);
> 
> ... we're also going to take this path (very quickly afterwards), just to
> get the same message twice without it being clear why it appears twice.

I missed to add printk("%s: ...", __func__, ...). I will update 
correspondingly if we will follow this way.

Thanks.

~ Oleksii

Re: [PATCH v2 16/26] xen/riscv: create APLIC DT node for guest domains

[Oleksii Kurochko]

On 6/3/26 5:10 PM, Jan Beulich wrote:
> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>> --- a/xen/arch/riscv/imsic.c
>> +++ b/xen/arch/riscv/imsic.c
>> @@ -14,6 +14,7 @@
>>   #include <xen/cpumask.h>
>>   #include <xen/device_tree.h>
>>   #include <xen/errno.h>
>> +#include <xen/fdt-kernel.h>
>>   #include <xen/init.h>
>>   #include <xen/macros.h>
>>   #include <xen/sched.h>
>> @@ -522,3 +523,9 @@ int __init imsic_init(const struct dt_device_node *node)
>>   
>>       return rc;
>>   }
>> +
>> +int __init vimsic_make_domu_dt_node(struct kernel_info *kinfo,
>> +                                    unsigned int *phandle)
>> +{
>> +    return -EOPNOTSUPP;
>> +}
> 
> This, I assume, is going to be filled properly by the next patch.

Yes as it was mentioned in the commit message. Actually here considering 
that this function isn't static I can just re-order patches and drop 
part of commit message in the current patch.

> 
>> --- a/xen/arch/riscv/vaplic.c
>> +++ b/xen/arch/riscv/vaplic.c
>> @@ -9,6 +9,8 @@
>>    */
>>   
>>   #include <xen/errno.h>
>> +#include <xen/fdt-kernel.h>
>> +#include <xen/libfdt/libfdt.h>
>>   #include <xen/sched.h>
>>   #include <xen/xvmalloc.h>
>>   
>> @@ -19,8 +21,11 @@
>>   
>>   #include "aplic-priv.h"
>>   
>> +#define VAPLIC_COMPATIBLE "riscv,aplic"
>>   #define VAPLIC_NUM_SOURCES 96
>>   
>> +#define FDT_VAPLIC_INT_CELLS 2
>> +
>>   static int cf_check vcpu_vaplic_init(struct vcpu *v)
>>   {
>>       int rc = 0;
>> @@ -47,6 +52,73 @@ static int cf_check vcpu_vaplic_init(struct vcpu *v)
>>       return rc;
>>   }
>>   
>> +static int __init cf_check vaplic_make_domu_dt_node(struct kernel_info *kinfo)
> 
> Again - why __init here and ...
> 
>> +{
>> +    int res = 0;
>> +    void *fdt = kinfo->fdt;
>> +    unsigned int msi_parent_phandle;
>> +    char vaplic_name[128];
>> +    paddr_t aplic_addr = GUEST_APLIC_S_BASE;
>> +    paddr_t aplic_size = APLIC_SIZE(kinfo->bd.d->max_vcpus);
>> +    const __be32 reg[] = {
>> +        cpu_to_be32(aplic_addr >> 32),
>> +        cpu_to_be32(aplic_addr),
>> +        cpu_to_be32(aplic_size >> 32),
>> +        cpu_to_be32(aplic_size),
>> +    };
>> +    struct vintc *vintc = kinfo->bd.d->arch.vintc;
>> +
>> +    res = snprintf(vaplic_name, sizeof(vaplic_name), "/soc/aplic@%x",
>> +                   GUEST_APLIC_S_BASE);
>> +    if ( res >= sizeof(vaplic_name) )
>> +    {
>> +        dprintk(XENLOG_DEBUG, "vaplic name is truncated\n");
>> +        return -ENOBUFS;
>> +    }
>> +
>> +    res = vimsic_make_domu_dt_node(kinfo, &msi_parent_phandle);
>> +    if ( res )
>> +        return res;
>> +
>> +    res = fdt_begin_node(fdt, vaplic_name);
>> +    if ( res )
>> +        return res;
>> +
>> +    res = fdt_property_cell(fdt, "#interrupt-cells", FDT_VAPLIC_INT_CELLS);
>> +    if ( res )
>> +        return res;
>> +
>> +    res = fdt_property(fdt, "reg", reg, sizeof(reg));
>> +    if ( res )
>> +        return res;
>> +
>> +    res = fdt_property_cell(fdt, "riscv,num-sources", vintc->irq_nums);
>> +    if ( res )
>> +        return res;
>> +
>> +    res = fdt_property(fdt, "interrupt-controller", NULL, 0);
>> +    if ( res )
>> +        return res;
>> +
>> +    res = fdt_property_string(fdt, "compatible", VAPLIC_COMPATIBLE);
>> +    if ( res )
>> +        return res;
>> +
>> +    res = fdt_property_cell(fdt, "msi-parent", msi_parent_phandle);
>> +    if ( res )
>> +        return res;
>> +
>> +    res = fdt_property_cell(fdt, "phandle", kinfo->phandle_intc);
>> +    if ( res )
>> +        return res;
>> +
>> +    return fdt_end_node(fdt);
>> +}
>> +
>> +static const struct vintc_init_ops __initdata init_ops = {
> 
> ... __initdata here? If you really want to have the option of moving
> domain creation stuff to .init.* when only dom0less is configured,
> then a proper abstraction is needed, along the lines of
> init_or_livepatch.

It was marked as __init and __initdata, respectively, because I had the 
dom0less use case in mind, where all domains are created during boot. In 
that scenario, vaplic_make_domu_dt_node() would not be reused, so the 
associated memory could be freed afterwards.

In the case of Dom0 with the xl toolstack, I assume the toolstack will 
create the node for the guest domain. For Dom0, however, I am not sure 
whether we can reuse the current implementation as-is. As it is written 
now, I assume that properties such as aplic_addr, reg, and compatible 
should be reused from the host, right?
If that is not a requirement for Dom0, then __init and __initdata could 
be removed (and possibly init_ops as well).

> 
>> @@ -60,13 +132,14 @@ int __init domain_vaplic_init(struct domain *d)
>>   
>>       d->arch.vintc = &vaplic->vintc;
>>       d->arch.vintc->ops = &vintc_ops;
>> +    d->arch.vintc->init_ops = &init_ops;
>>   
>> -    vaplic->regs.domaincfg = APLIC_DOMAINCFG_IE | APLIC_DOMAINCFG_DM;
>> +    vaplic->regs.domaincfg = APLIC_DOMAINCFG_IE | APLIC_DOMAINCFG_DM |
>> +                             APLIC_DOMAINCFG_RO80;
> 
> This looks unrelated. I don't mind it being done here, but then it
> wants mentioning in the description. Or maybe I simply don't understand
> what this is about.

Right, it shouldn't be here. It would be better to move to [PATCH v2 
14/26] xen/riscv: add very early virtual APLIC (vAPLIC) initialization 
support.

Thanks.

~ Oleksii

Re: [PATCH v2 17/26] xen/riscv: generate IMSIC DT node for guest domains

[Oleksii Kurochko]

On 6/3/26 5:21 PM, Jan Beulich wrote:
> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>> Guests using the IMSIC interrupt controller require a corresponding
>> Device Tree description.
>>
>> Add support for generating an IMSIC node when building the guest DT.
>> This allows guests to discover and use the IMSIC interrupt controller.
>>
>> Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>> ---
>> Changes in v2:
>>   - s/imsic_make_reg_property/guest_imsic_make_reg_property.
>>   - s/imsic_set_interrupt_extended_prop/guest_imsic_set_interrupt_extended_prop.
>>   - Use initalizer for regs[] array in imsic_make_reg_property().
>>   - Move buf[] insde the for() loop.
>>   - Correct check of returned phandle.
>>   - Drop local variable len.
>>   - /s/XVFREE/xvfree in imsic_set_interrupt_extended_prop().
>>   - Drop initializer for local variable data.
>>   - s/uint32_t/unsinged int for pos and cpu in imsic_set_interrupt_extended_prop().
>>   - Drop next_phandle as it is now in common code.
>>   - Introduce vcpu_imsic_deinit.
>>   - Refactor vimsic_make_domu_dt_node() to avoid usage of host IMSIC dt node.
>> ---
>>   xen/arch/riscv/imsic.c                    | 127 +++++++++++++++++++++-
>>   xen/arch/riscv/include/asm/guest-layout.h |   2 +
>>   2 files changed, 128 insertions(+), 1 deletion(-)
>>
>> diff --git a/xen/arch/riscv/imsic.c b/xen/arch/riscv/imsic.c
>> index ceea6778d9dc..19cbacdf96e1 100644
>> --- a/xen/arch/riscv/imsic.c
>> +++ b/xen/arch/riscv/imsic.c
>> @@ -13,9 +13,12 @@
>>   #include <xen/const.h>
>>   #include <xen/cpumask.h>
>>   #include <xen/device_tree.h>
>> +#include <xen/domain.h>
>>   #include <xen/errno.h>
>> +#include <xen/fdt-domain-build.h>
>>   #include <xen/fdt-kernel.h>
>>   #include <xen/init.h>
>> +#include <xen/libfdt/libfdt.h>
>>   #include <xen/macros.h>
>>   #include <xen/sched.h>
>>   #include <xen/smp.h>
>> @@ -35,6 +38,11 @@ static struct imsic_config imsic_cfg = {
>>       .lock = SPIN_LOCK_UNLOCKED,
>>   };
>>   
>> +static unsigned int __ro_after_init guest_num_msis;
> 
> How come this is __ro_after_init, when it's ...
> 
>> @@ -291,6 +299,11 @@ static int imsic_parse_node(const struct dt_device_node *node,
>>           return -ENOENT;
>>       }
>>   
>> +    if ( dt_property_read_u32(node, "riscv,num-guest-ids", &tmp) )
>> +        guest_num_msis = tmp;
>> +    else
>> +        guest_num_msis = imsic_cfg.nr_ids;
> 
> ... written by a non-__init function?

__ro_after_init should be dropped. I will use __read_mostly instead.

  Plus are you again inheriting a host
> property into guests without saying why?

Based on the discussion for similar place around vAPLIC code in the 
previous patch I will add here or the comment why or introduce 
VIMISC_NR_IDS and use it instead.

> 
>> @@ -524,8 +537,120 @@ int __init imsic_init(const struct dt_device_node *node)
>>       return rc;
>>   }
>>   
>> +static int __init guest_imsic_make_reg_property(struct domain *d, void *fdt)
> 
> Same question again as to __init throughout here.

 From Arm code I understood that libxl creates the interrupt controller 
node in userspace before handing the FDT to Xen so this and related 
functions won't be called by userspace.

And for dom0's it is a separate *_make_hdwom_dt_node() (took it from Arm 
code base) which can expose the real hardware MMIO regions to dom0 
(since dom0 may need direct hardware access - if it isn't strict 
requirement then the current one way to create node could be re-used).

So if for dom0/hwdom it should be created separate node then we will 
have similar separate function for IMSIC node here and then it looks 
fine to keep __init.

> 
>> +{
>> +    paddr_t base_addr = GUEST_IMSIC_S_BASE;
> 
> So you make a local variable for a constant, ...
> 
>> +    __be32 regs[4] = {
>> +        cpu_to_be32(base_addr >> 32),
>> +        cpu_to_be32(base_addr),
>> +        cpu_to_be32((IMSIC_MMIO_PAGE_SZ * d->max_vcpus) >> 32),
>> +        cpu_to_be32(IMSIC_MMIO_PAGE_SZ * d->max_vcpus),
> 
> ... but this non-constant expression is spelled out twice.

I will use instead of base_addr GUEST_IMSIC_S_BASE directly and 
introduce local variable for:
   paddr_t size = IMSIC_MMIO_PAGE_SZ * d->max_vcpus;

> 
>> +static int __init guest_imsic_set_interrupt_extended_prop(struct domain *d,
>> +                                                          void *fdt)
>> +{
>> +    unsigned int cpu, pos = 0;
>> +    uint32_t phandle;
>> +    uint32_t *irq_ext;
> 
> Doesn't this want to be __be32, seeing ...
> 
>> +    int res;
>> +
>> +    irq_ext = xvzalloc_array(uint32_t, d->max_vcpus * 2);
>> +    if ( !irq_ext )
>> +        return -ENOMEM;
>> +
>> +    for ( cpu = 0; cpu < d->max_vcpus; cpu++ )
>> +    {
>> +        char buf[64];
>> +
>> +        snprintf(buf, sizeof(buf), "/cpus/cpu@%u/interrupt-controller", cpu);
>> +        phandle = fdt_get_phandle(fdt, fdt_path_offset(fdt, buf));
>> +
>> +        if ( !phandle )
>> +        {
>> +            res = -ENODEV;
>> +            goto out;
>> +        }
>> +
>> +        irq_ext[pos++] = cpu_to_be32(phandle);
>> +        irq_ext[pos++] = cpu_to_be32(IRQ_S_EXT);
> 
> ... this?

Agree, __be32 should be added for definition of irq_ext.

> 
> Also, just like "buf", "phandle" can be local to this loop's body.
> 
>> --- a/xen/arch/riscv/include/asm/guest-layout.h
>> +++ b/xen/arch/riscv/include/asm/guest-layout.h
>> @@ -5,6 +5,8 @@
>>   
>>   #define GUEST_APLIC_S_BASE 0xd000000
>>   
>> +#define GUEST_IMSIC_S_BASE 0x28000000
>> +
>>   #define GUEST_RAM_BANKS   2
> 
> Is this going to become an unannotated collection of (seemingly) random
> numbers?

Could you please clarify what kind of annotation you would expect to see 
here? It's just an address, which is typically used for IMSIC by QEMU, 
so I chose the same notation for the guest address as well.

Thanks.

~ Oleksii

Re: [PATCH v2 19/26] xen/riscv: implement IRQ routing for device passthrough

[Oleksii Kurochko]

On 6/3/26 6:01 PM, Jan Beulich wrote:
> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>> --- /dev/null
>> +++ b/xen/arch/riscv/device.c
>> @@ -0,0 +1,108 @@
>> +/* SPDX-License-Identifier: GPL-2.0-or-later */
>> +
>> +#include <xen/device_tree.h>
>> +#include <xen/errno.h>
>> +#include <xen/iocap.h>
>> +#include <xen/rangeset.h>
>> +#include <xen/sched.h>
>> +
>> +#include <asm/intc.h>
>> +
>> +int map_irq_to_domain(struct domain *d, unsigned int irq,
>> +                      bool need_mapping, const char *devname)
>> +{
>> +    int res;
>> +
>> +    res = irq_permit_access(d, irq);
>> +    if ( res )
>> +    {
>> +        printk(XENLOG_ERR "Unable to permit to %pd access to IRQ %u\n", d, irq);
> 
> Nit: Drop the first "to"?

Sure, I will drop that.

> 
>> +        return res;
>> +    }
>> +
>> +    if ( need_mapping )
>> +    {
>> +        /*
>> +         * Checking the return of vintc_reserve_virq is not
>> +         * necessary. It should not fail except when we try to map
>> +         * the IRQ twice. This can legitimately happen if the IRQ is shared.
>> +         */
>> +        vintc_reserve_virq(d, irq);
>> +
>> +        res = route_irq_to_guest(d, irq, irq, devname);
>> +        if ( res < 0 )
>> +        {
>> +            printk(XENLOG_ERR "Unable to map IRQ%u to %pd\n", irq, d);
>> +            return res;
>> +        }
>> +    }
>> +
>> +    dt_dprintk("  - IRQ: %u\n", irq);
>> +
>> +    return 0;
>> +}
>> +
>> +/*
>> + * map_device_irqs_to_domain retrieves the interrupts configuration from
>> + * a device tree node and maps those interrupts to the target domain.
>> + *
>> + * Returns:
>> + *   < 0 error
>> + *   0   success
>> + */
>> +int map_device_irqs_to_domain(struct domain *d,
>> +                              struct dt_device_node *dev,
>> +                              bool need_mapping,
>> +                              struct rangeset *irq_ranges)
>> +{
>> +    unsigned int i, nirq;
>> +    int res, irq;
>> +    struct dt_raw_irq rirq;
> 
> Move the latter three variables to the loop's scope and ...
> 
>> +    nirq = dt_number_of_irq(dev);
> 
> ... make this the variable's initializer?

It makes sense. I will do that.

> 
>> +    /* Give permission and map IRQs */
>> +    for ( i = 0; i < nirq; i++ )
>> +    {
>> +        res = dt_device_get_raw_irq(dev, i, &rirq);
>> +        if ( res )
>> +        {
>> +            printk(XENLOG_ERR "Unable to retrieve irq %u for %s\n",
>> +                   i, dt_node_full_name(dev));
>> +            return res;
>> +        }
>> +
>> +        /*
>> +         * Don't map IRQ that have no physical meaning
>> +         * ie: IRQ whose controller is not APLIC/IMSIC/PLIC.
>> +         */
>> +        if ( rirq.controller != dt_interrupt_controller )
>> +        {
>> +            dt_dprintk("irq %u not connected to primary controller."
>> +                       "Connected to %s\n", i,
>> +                       dt_node_full_name(rirq.controller));
>> +            continue;
>> +        }
>> +
>> +        irq = platform_get_irq(dev, i);
>> +        if ( irq < 0 )
>> +        {
>> +            printk("Unable to get irq %u for %s\n", i, dt_node_full_name(dev));
>> +            return irq;
>> +        }
>> +
>> +        res = map_irq_to_domain(d, irq, need_mapping, dt_node_name(dev));
>> +        if ( res )
>> +            return res;
>> +
>> +
>> +        /*
>> +         * At the moment there is only one user of map_device_irqs_to_domain()
>> +         * for RISC-V which calls it irq_ranges == NULL.
>> +         */
>> +        if ( irq_ranges )
>> +            return -EOPNOTSUPP;
> 
> Why is this checked last, and inside the loop (when it's loop invariant)?

Just to show the place where irq_ranges will be handled. But currently I 
agree it would be better just move outside the loop.

> 
>> --- a/xen/arch/riscv/include/asm/intc.h
>> +++ b/xen/arch/riscv/include/asm/intc.h
>> @@ -13,8 +13,11 @@ enum intc_version {
>>   };
>>   
>>   struct cpu_user_regs;
>> +struct domain;
> 
> I can spot why this is needed, but ...
> 
>> +struct dt_device_node;
>>   struct irq_desc;
>>   struct kernel_info;
>> +struct rangeset;
>>   struct vcpu;
> 
> ... I'm at a loss to explain the need for these two additions.

Rudements from previous version of this patch series. I will drop them.
The same below ...

> 
>> --- a/xen/arch/riscv/include/asm/setup.h
>> +++ b/xen/arch/riscv/include/asm/setup.h
>> @@ -5,6 +5,10 @@
>>   
>>   #include <xen/types.h>
>>   
>> +struct domain;
>> +struct dt_device_node;
>> +struct rangeset;
> 
> Same here - why would they be needed when you make no other changes
> to this header?

... here.

> 
>> --- a/xen/arch/riscv/intc.c
>> +++ b/xen/arch/riscv/intc.c
>> @@ -7,7 +7,9 @@
>>   #include <xen/init.h>
>>   #include <xen/irq.h>
>>   #include <xen/lib.h>
>> +#include <xen/sched.h>
>>   #include <xen/spinlock.h>
>> +#include <xen/xvmalloc.h>
>>   
>>   #include <asm/aia.h>
>>   #include <asm/intc.h>
>> @@ -86,6 +88,22 @@ unsigned int intc_irq_nums(void)
>>       return intc_hw_ops->irq_nums();
>>   }
>>   
>> +int intc_route_irq_to_guest(struct irq_desc *desc,
>> +                            unsigned int priority)
>> +{
>> +    ASSERT(spin_is_locked(&desc->lock));
>> +
>> +    ASSERT(intc_hw_ops->guest_irq_type);
>> +
>> +    desc->handler = intc_hw_ops->guest_irq_type;
>> +    set_bit(_IRQ_GUEST, &desc->status);
> 
> Is desc->status accessed anywhere without holding desc->lock? If not,
> __set_bit() or simply |= ?

In release_irq() it could be used without lock:
...
     /* Wait to make sure it's not being used on another CPU */
     do { smp_mb(); } while ( test_bit(_IRQ_INPROGRESS, &desc->status) );

> 
>> @@ -112,6 +130,14 @@ int domain_vintc_init(struct domain *d)
>>           break;
>>       }
>>   
>> +    if ( !ret )
>> +    {
>> +        d->arch.vintc->allocated_irqs =
>> +            xvzalloc_array(unsigned long, BITS_TO_LONGS(d->arch.vintc->irq_nums));
>> +        if ( !d->arch.vintc->allocated_irqs )
>> +            ret = -ENOMEM;
>> +    }
>> +
>>       return ret;
>>   }
>>   
>> @@ -129,4 +155,14 @@ void domain_vintc_deinit(struct domain *d)
>>           printk("vintc (ver:%d) isn't implemented\n", ver);
>>           break;
>>       }
>> +
>> +    xvfree(d->arch.vintc->allocated_irqs);
>> +}
> 
> XVFREE()
> 
>> +bool vintc_reserve_virq(const struct domain *d, unsigned int virq)
>> +{
>> +    if ( virq >= d->arch.vintc->irq_nums )
>> +        return false;
>> +
>> +    return !test_and_set_bit(virq, d->arch.vintc->allocated_irqs);
>>   }
> 
> As to function / field naming: You don't look to be allocating IRQs. So
> is there a reason the field name gives the impression of allocation?
> Simply s/allocated/used/ or some such?

I am okay to rename to 'used' instead of 'allocated'.

> 
>> @@ -221,3 +239,160 @@ void do_IRQ(struct cpu_user_regs *regs, unsigned int irq)
>>       spin_unlock(&desc->lock);
>>       irq_exit();
>>   }
>> +
>> +static inline struct irq_guest *irq_get_guest_info(struct irq_desc *desc)
>> +{
>> +    ASSERT(spin_is_locked(&desc->lock));
>> +    ASSERT(test_bit(_IRQ_GUEST, &desc->status));
>> +    ASSERT(desc->action != NULL);
>> +
>> +    return desc->action->dev_id;
>> +}
>> +
>> +static inline struct domain *irq_get_domain(struct irq_desc *desc)
>> +{
>> +    return irq_get_guest_info(desc)->d;
>> +}
>> +
>> +void release_irq(unsigned int irq, const void *dev_id)
>> +{
>> +    struct irq_desc *desc;
>> +    unsigned long flags;
>> +    struct irqaction *action, **action_ptr;
>> +
>> +    desc = irq_to_desc(irq);
>> +
>> +    spin_lock_irqsave(&desc->lock,flags);
>> +
>> +    action_ptr = &desc->action;
>> +#ifdef CONFIG_IRQ_HAS_MULTIPLE_ACTION
>> +    for ( ;; )
>> +    {
>> +        action = *action_ptr;
>> +        if ( !action )
>> +        {
>> +            printk(XENLOG_WARNING "Trying to free already-free IRQ %u\n", irq);
>> +            spin_unlock_irqrestore(&desc->lock, flags);
>> +            return;
>> +        }
>> +
>> +        if ( action->dev_id == dev_id )
>> +            break;
>> +
>> +        action_ptr = &action->next;
>> +    }
>> +
>> +    /* Found it - remove it from the action list */
>> +    *action_ptr = action->next;
>> +#else
>> +    action = *action_ptr;

It is needed to add *action_ptr = NULL here to deal with ...

>> +#endif
>> +
>> +    /* If this was the last action, shut down the IRQ */
>> +    if ( !desc->action )
>> +    {
>> +        desc->handler->shutdown(desc);
>> +        clear_bit(_IRQ_GUEST, &desc->status);
>> +    }
>> +
>> +    spin_unlock_irqrestore(&desc->lock,flags);
>> +
>> +    /* Wait to make sure it's not being used on another CPU */
>> +    do { smp_mb(); } while ( test_bit(_IRQ_INPROGRESS, &desc->status) );
>> +
>> +    if ( action->free_on_release )
>> +        xvfree(action);
> 
> When !IRQ_HAS_MULTIPLE_ACTION desc->action becomes a dangling pointer here.

... it could be xvfree here as action is local variable.

> 
>> +/* Route an IRQ to a specific guest */
>> +int route_irq_to_guest(struct domain *d, unsigned int virq,
>> +                       unsigned int irq, const char *devname)
>> +{
>> +    struct irqaction *action;
>> +    struct irq_guest *info;
>> +    struct irq_desc *desc;
>> +    unsigned long flags;
>> +    int retval = 0;
>> +
>> +    desc = irq_to_desc(irq);
>> +
>> +    action = xvmalloc(struct irqaction);
>> +    if ( !action )
>> +        return -ENOMEM;
> 
> This is freed by release_irq(), but ...
> 
>> +    info = xvmalloc(struct irq_guest);
>> +    if ( !info )
> 
> ... where is the (non-error-path) freeing of this?
> 

Agree it should be freed.

I am thing about just to update release_irq():

if ( action->free_on_release )
{
     xvfree(action->dev_id);
     xvfree(action);
}

But I think it is conceptually is incorrect as owner of ->dev_id in this 
case is guest so it would be better if guest will do that. So I think it 
would be better to intoduce now release_guest_irq():

int release_guest_irq(struct domain *d, unsigned int virq)
{
     struct irq_desc *desc;
     struct irq_guest *info;
     unsigned long flags;

     desc = irq_to_desc(virq);

     spin_lock_irqsave(&desc->lock, flags);

     if ( !test_bit(_IRQ_GUEST, &desc->status) )
         goto unlock_err;

     info = irq_get_guest_info(desc);
     if ( d != info->d )
         goto unlock_err;

     spin_unlock_irqrestore(&desc->lock, flags);

     release_irq(desc->irq, info);
     xvfree(info);

     return 0;

  unlock_err:
     spin_unlock_irqrestore(&desc->lock, flags);
     return -EINVAL;
}

and then call it in domain_vintc_deinit() for all virqs of a domain.

>> +    {
>> +        xvfree(action);
>> +        return -ENOMEM;
>> +    }
>> +
>> +    info->d = d;
>> +    info->virq = virq;
>> +
>> +    action->dev_id = info;
>> +    action->name = devname;
>> +    action->free_on_release = 1;
> 
> true
> 
>> +    spin_lock_irqsave(&desc->lock, flags);
>> +
>> +    /*
>> +     * If the IRQ is already used by someone
>> +     *  - If it's the same domain -> Xen doesn't need to update the IRQ desc.
>> +     *  For safety check if we are not trying to assign the IRQ to a
>> +     *  different vIRQ.
>> +     *  - Otherwise -> For now, don't allow the IRQ to be shared between
>> +     *  Xen and domains.
>> +     */
>> +    if ( desc->action != NULL )
>> +    {
>> +        if ( test_bit(_IRQ_GUEST, &desc->status) )
>> +        {
>> +            struct domain *ad = irq_get_domain(desc);
>> +
>> +            if ( d != ad )
>> +            {
>> +                printk(XENLOG_G_ERR "IRQ %u is already used by domain %u\n",
>> +                       irq, ad->domain_id);
>> +                retval = -EBUSY;
>> +            }
>> +            else if ( irq_get_guest_info(desc)->virq != virq )
>> +            {
>> +                printk(XENLOG_G_ERR
>> +                       "d%u: IRQ %u is already assigned to vIRQ %u\n",
>> +                       d->domain_id, irq, irq_get_guest_info(desc)->virq);
> 
> Please can you get used to using %pd?

I'll do my best. Thanks for consistently pointing that out. I appreciate it.


> 
>> +                retval = -EBUSY;
>> +            }
>> +        }
>> +        else
>> +        {
>> +            printk(XENLOG_G_ERR "IRQ %u is already used by Xen\n", irq);
>> +            retval = -EBUSY;
>> +        }
>> +        goto out;
>> +    }
>> +
>> +    retval = _setup_irq(desc, 0, action);
>> +    if ( retval )
>> +        goto out;
>> +
>> +    retval = intc_route_irq_to_guest(desc, IRQ_NO_PRIORITY);
>> +
>> +    spin_unlock_irqrestore(&desc->lock, flags);
>> +
>> +    if ( retval )
>> +    {
>> +        release_irq(desc->irq, info);
> 
> Is de-referencing desc legitimate / race free with desc->lock not held?

desc itself cannot be freed, it's a pointer into the statically 
allocated array irq_desc[NR_IRQS] (riscv/irq.c:29). The descriptor 
object lives for the lifetime of the system.

desc->irq is write-once, it's set during init_irq_data() at boot 
(riscv/irq.c:172) and never modified again. It's effectively an 
immutable field after initialization, so reading it without the lock 
held is safe.

Thanks.

~ Oleksii

Re: [PATCH v2 14/26] xen/riscv: add very early virtual APLIC (vAPLIC) initialization support

[Jan Beulich]

On 04.06.2026 13:29, Oleksii Kurochko wrote:
> On 6/3/26 4:54 PM, Jan Beulich wrote:
>> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>>> --- a/xen/arch/riscv/aplic.c
>>> +++ b/xen/arch/riscv/aplic.c
>>> @@ -295,6 +295,11 @@ static void cf_check aplic_set_irq_type(struct irq_desc *desc,
>>>       spin_unlock(&aplic.lock);
>>>   }
>>>   
>>> +static unsigned int cf_check aplic_irq_num(void)
>>> +{
>>> +    return aplic_info.num_irqs;
>>> +}
>>> +
>>>   static const hw_irq_controller aplic_xen_irq_type = {
>>>       .typename     = "aplic",
>>>       .startup      = aplic_irq_startup,
>>> @@ -309,6 +314,7 @@ static const struct intc_hw_operations aplic_ops = {
>>>       .host_irq_type       = &aplic_xen_irq_type,
>>>       .handle_interrupt    = aplic_handle_interrupt,
>>>       .set_irq_type        = aplic_set_irq_type,
>>> +    .irq_nums            = aplic_irq_num,
>>
>> Hook handler names and respective field names would preferably match up. It's
>> unclear why the field uses some kind of plural(?), while the function uses
>> singular.
> 
> I will do s/aplic_irq_num/aplic_irq_nums.

Well. What "nums"? Isn't it "number of IRQs" that the hook returns? That would
call for .nr_irqs and aplic_nr_irqs(), for example.

>>> +unsigned int intc_irq_nums(void)
>>> +{
>>> +    ASSERT(intc_hw_ops && intc_hw_ops->irq_nums);
>>> +
>>> +    return intc_hw_ops->irq_nums();
>>> +}
>>
>> You use this to set domains' properties. As indicated before, I view it as
>> wrong to do so for any domain, besides perhaps Dom0 / hwdom. If you want to
>> do so nevertheless, at the very least I'd expect something to be said about
>> such a decision in the description.
> 
> If you could explain how this is expected to work for non-Dom0/hwdom 
> domains, I would consider reworking it.
> 
> Basically, I don't understand how the following scenario is supposed to 
> work. Let's say the host interrupt controller can manage 15 interrupts, 
> while the guest interrupt controller supports only 7. If we want to pass 
> through 8 devices to the guest, what should happen in that case? Should 
> Xen simply report that the 8th device cannot be passed through because 
> the guest supports only 7 IRQs?

If IRQs cannot be shared - yes.

> Another concern is related to 1:1 IRQ mapping. Suppose I want to pass 
> through a UART device whose IRQ number is typically greater than 10. In 
> that case, it seems Xen would again have to report that the device 
> cannot be passed through because its interrupt number exceeds the number 
> of IRQs supported by the guest interrupt controller. This could be 
> addressed by introducing a non-1:1 IRQ mapping between the host and 
> guest, but the current dom0less codebase appears to assume a 1:1 IRQ 
> mapping (unless I am mistaken).

Sounds like a (perhaps significant) shortcoming to me.

> Considering that virtual interrupt controllers use the maximum possible 
> number of interrupts supported by the interrupt controller, the concerns 
> mentioned above are unlikely to arise for a long time, if ever, unless 
> support for features such as migration is introduced. Therefore, I think 
> it would be reasonable to remove intc_irq_nums() and avoid using it to 
> initialize virtual interrupts or domain properties.

Yes.

Jan

Re: [PATCH v2 15/26] xen/riscv: introduce (de)initialization helpers for vINTC

[Jan Beulich]

On 04.06.2026 13:33, Oleksii Kurochko wrote:
> On 6/3/26 5:00 PM, Jan Beulich wrote:
>> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>>> --- a/xen/arch/riscv/intc.c
>>> +++ b/xen/arch/riscv/intc.c
>>> @@ -11,6 +11,7 @@
>>>   
>>>   #include <asm/aia.h>
>>>   #include <asm/intc.h>
>>> +#include <asm/vaplic.h>
>>>   
>>>   static const struct intc_hw_operations *__ro_after_init intc_hw_ops;
>>>   
>>> @@ -94,3 +95,38 @@ int __init make_intc_domU_node(struct kernel_info *kinfo)
>>>   
>>>       return -EOPNOTSUPP;
>>>   }
>>> +
>>> +int domain_vintc_init(struct domain *d)
>>> +{
>>> +    int ret = -EOPNOTSUPP;
>>> +    const enum intc_version ver = intc_hw_ops->info->hw_version;
>>
>> Again - why would what the underlying hardware has control what all domains
>> get?
> 
> If host uses AIA (APLIC/IMSIC) compatible controller then guest should 
> use virtual AIA compatible controller, shouldn't it?
> 
> I don't think that it is a case when host uses PLIC interrupt controller 
> but guests are going to use virtual APLIC.

Well, this may be a present restriction, but once you want to support migration
(and different IC models), I guess you won't get around such.

>>> +    switch ( ver )
>>> +    {
>>> +    case INTC_APLIC:
>>> +        ret = domain_vaplic_init(d);
>>> +        break;
>>> +
>>> +    default:
>>> +        printk("vintc (ver:%d) isn't implemented\n", ver);
>>
>> If we take this path for whatever reason, ...
>>
>>> +        break;
>>> +    }
>>> +
>>> +    return ret;
>>> +}
>>> +
>>> +void domain_vintc_deinit(struct domain *d)
>>> +{
>>> +    const enum intc_version ver = intc_hw_ops->info->hw_version;
>>> +
>>> +    switch ( ver )
>>> +    {
>>> +    case INTC_APLIC:
>>> +        domain_vaplic_deinit(d);
>>> +        break;
>>> +
>>> +    default:
>>> +        printk("vintc (ver:%d) isn't implemented\n", ver);
>>
>> ... we're also going to take this path (very quickly afterwards), just to
>> get the same message twice without it being clear why it appears twice.
> 
> I missed to add printk("%s: ...", __func__, ...). I will update 
> correspondingly if we will follow this way.

That's not quite what I was after, though. I question the need for this latter
message.

Jan

Re: [PATCH v2 17/26] xen/riscv: generate IMSIC DT node for guest domains

[Jan Beulich]

On 04.06.2026 16:21, Oleksii Kurochko wrote:
> 
> 
> On 6/3/26 5:21 PM, Jan Beulich wrote:
>> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>>> Guests using the IMSIC interrupt controller require a corresponding
>>> Device Tree description.
>>>
>>> Add support for generating an IMSIC node when building the guest DT.
>>> This allows guests to discover and use the IMSIC interrupt controller.
>>>
>>> Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>> ---
>>> Changes in v2:
>>>   - s/imsic_make_reg_property/guest_imsic_make_reg_property.
>>>   - s/imsic_set_interrupt_extended_prop/guest_imsic_set_interrupt_extended_prop.
>>>   - Use initalizer for regs[] array in imsic_make_reg_property().
>>>   - Move buf[] insde the for() loop.
>>>   - Correct check of returned phandle.
>>>   - Drop local variable len.
>>>   - /s/XVFREE/xvfree in imsic_set_interrupt_extended_prop().
>>>   - Drop initializer for local variable data.
>>>   - s/uint32_t/unsinged int for pos and cpu in imsic_set_interrupt_extended_prop().
>>>   - Drop next_phandle as it is now in common code.
>>>   - Introduce vcpu_imsic_deinit.
>>>   - Refactor vimsic_make_domu_dt_node() to avoid usage of host IMSIC dt node.
>>> ---
>>>   xen/arch/riscv/imsic.c                    | 127 +++++++++++++++++++++-
>>>   xen/arch/riscv/include/asm/guest-layout.h |   2 +
>>>   2 files changed, 128 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/xen/arch/riscv/imsic.c b/xen/arch/riscv/imsic.c
>>> index ceea6778d9dc..19cbacdf96e1 100644
>>> --- a/xen/arch/riscv/imsic.c
>>> +++ b/xen/arch/riscv/imsic.c
>>> @@ -13,9 +13,12 @@
>>>   #include <xen/const.h>
>>>   #include <xen/cpumask.h>
>>>   #include <xen/device_tree.h>
>>> +#include <xen/domain.h>
>>>   #include <xen/errno.h>
>>> +#include <xen/fdt-domain-build.h>
>>>   #include <xen/fdt-kernel.h>
>>>   #include <xen/init.h>
>>> +#include <xen/libfdt/libfdt.h>
>>>   #include <xen/macros.h>
>>>   #include <xen/sched.h>
>>>   #include <xen/smp.h>
>>> @@ -35,6 +38,11 @@ static struct imsic_config imsic_cfg = {
>>>       .lock = SPIN_LOCK_UNLOCKED,
>>>   };
>>>   
>>> +static unsigned int __ro_after_init guest_num_msis;
>>
>> How come this is __ro_after_init, when it's ...
>>
>>> @@ -291,6 +299,11 @@ static int imsic_parse_node(const struct dt_device_node *node,
>>>           return -ENOENT;
>>>       }
>>>   
>>> +    if ( dt_property_read_u32(node, "riscv,num-guest-ids", &tmp) )
>>> +        guest_num_msis = tmp;
>>> +    else
>>> +        guest_num_msis = imsic_cfg.nr_ids;
>>
>> ... written by a non-__init function?
> 
> __ro_after_init should be dropped. I will use __read_mostly instead.
> 
>   Plus are you again inheriting a host
>> property into guests without saying why?
> 
> Based on the discussion for similar place around vAPLIC code in the 
> previous patch I will add here or the comment why or introduce 
> VIMISC_NR_IDS and use it instead.
> 
>>
>>> @@ -524,8 +537,120 @@ int __init imsic_init(const struct dt_device_node *node)
>>>       return rc;
>>>   }
>>>   
>>> +static int __init guest_imsic_make_reg_property(struct domain *d, void *fdt)
>>
>> Same question again as to __init throughout here.
> 
>  From Arm code I understood that libxl creates the interrupt controller 
> node in userspace before handing the FDT to Xen so this and related 
> functions won't be called by userspace.

Hmm, okay, on the v3 submission I'll have to keep that in mind, and call
paths will need checking accordingly.

>>> --- a/xen/arch/riscv/include/asm/guest-layout.h
>>> +++ b/xen/arch/riscv/include/asm/guest-layout.h
>>> @@ -5,6 +5,8 @@
>>>   
>>>   #define GUEST_APLIC_S_BASE 0xd000000
>>>   
>>> +#define GUEST_IMSIC_S_BASE 0x28000000
>>> +
>>>   #define GUEST_RAM_BANKS   2
>>
>> Is this going to become an unannotated collection of (seemingly) random
>> numbers?
> 
> Could you please clarify what kind of annotation you would expect to see 
> here? It's just an address, which is typically used for IMSIC by QEMU, 
> so I chose the same notation for the guest address as well.

Well, how could I have known? The description says nothing, and there are
also no comments. The numbers here look entirely arbitrary, and if someone
wanted to change them, it wouldn't be clear at all what constraints need
taking into consideration.

Jan

Re: [PATCH v2 19/26] xen/riscv: implement IRQ routing for device passthrough

[Jan Beulich]

On 04.06.2026 17:35, Oleksii Kurochko wrote:
> On 6/3/26 6:01 PM, Jan Beulich wrote:
>> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>>> @@ -86,6 +88,22 @@ unsigned int intc_irq_nums(void)
>>>       return intc_hw_ops->irq_nums();
>>>   }
>>>   
>>> +int intc_route_irq_to_guest(struct irq_desc *desc,
>>> +                            unsigned int priority)
>>> +{
>>> +    ASSERT(spin_is_locked(&desc->lock));
>>> +
>>> +    ASSERT(intc_hw_ops->guest_irq_type);
>>> +
>>> +    desc->handler = intc_hw_ops->guest_irq_type;
>>> +    set_bit(_IRQ_GUEST, &desc->status);
>>
>> Is desc->status accessed anywhere without holding desc->lock? If not,
>> __set_bit() or simply |= ?
> 
> In release_irq() it could be used without lock:
> ...
>      /* Wait to make sure it's not being used on another CPU */
>      do { smp_mb(); } while ( test_bit(_IRQ_INPROGRESS, &desc->status) );

That's fine with the update being non-atomic. I probably should have said
"modified" instead of "accessed".

Jan

Re: [PATCH v2 20/26] xen/riscv: add missing APLIC register offsets, masks to asm/aplic.h.

[Oleksii Kurochko]

On 6/3/26 5:36 PM, Jan Beulich wrote:
> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>> --- a/xen/arch/riscv/include/asm/aplic.h
>> +++ b/xen/arch/riscv/include/asm/aplic.h
>> @@ -15,6 +15,11 @@
>>   
>>   #include <asm/imsic.h>
>>   
>> +#define APLIC_REG_OFFSET_MASK   0x3fff
> 
> This I can see this as wanting to live separately. Yet ...
> 
>> +#define APLIC_TARGET_IPRIO_MASK 0xff

This could be dropped as I don't use this mask anymore.

>> +#define APLIC_TARGET_GUEST_IDX_SHIFT 12
>> +#define APLIC_TARGET_EIID_MASK  0x7ff
> 
> ... what (set of) register(s) do these apply to? Perhaps ...
> 
>> @@ -26,6 +31,36 @@
>>   #define APLIC_SOURCECFG_SM_LEVEL_HIGH   0x6
>>   #define APLIC_SOURCECFG_SM_LEVEL_LOW    0x7
>>   
>> +#define APLIC_DOMAINCFG         0x0000
>> +#define APLIC_SOURCECFG_BASE    0x0004
>> +#define APLIC_SOURCECFG_LAST    0x0ffc
>> +
>> +#define APLIC_SMSICFGADDR       0x1bc8
>> +#define APLIC_SMSICFGADDRH      0x1bcc
>> +
>> +#define APLIC_SETIP_BASE        0x1c00
>> +#define APLIC_SETIP_LAST        0x1c7c
>> +#define APLIC_SETIPNUM          0x1cdc
>> +
>> +#define APLIC_CLRIP_BASE        0x1d00
>> +#define APLIC_CLRIP_LAST        0x1d7c
>> +#define APLIC_CLRIPNUM          0x1ddc
>> +
>> +#define APLIC_SETIE_BASE        0x1e00
>> +#define APLIC_SETIE_LAST        0x1e7c
>> +#define APLIC_SETIENUM          0x1edc
>> +
>> +#define APLIC_CLRIE_BASE        0x1f00
>> +#define APLIC_CLRIE_LAST        0x1f7c
>> +#define APLIC_CLRIENUM          0x1fdc
>> +
>> +#define APLIC_SETIPNUM_LE       0x2000
>> +
>> +#define APLIC_GENMSI            0x3000
>> +
>> +#define APLIC_TARGET_BASE       0x3004
>> +#define APLIC_TARGET_LAST       0x3ffc
> 
> ... these? And then is ...

Yes, it is applied to target register.

> 
>>   #define APLIC_TARGET_HART_IDX_SHIFT 18
> 
> ... this also covering some part of them? Can't they (a) live together and (b)
> have some kind of connection to what they apply to?

Agreed, they could live together. The intent was simply to keep all MMIO 
definitions in sequence, but I'm okay with grouping macros related to a 
specific register together.

> 
> And then why is there again a mix of *_SHIFT and *_MASK?

This APLIC_TARGET_GUEST_IDX_SHIFT - could be dropped I've already 
reworked that in different branch and use *_MASK instead so I will 
re-apply that changes in this patch. But APLIC_TARGET_HART_IDX_SHIFT is 
used in to not open-code 12 in calculation of shift value for group_index:

static void cf_check aplic_set_irq_affinity(struct irq_desc *desc, const 
cpumask_t *mask)
{
...
     value |= cpu << APLIC_TARGET_HART_IDX_SHIFT;
     value |= group_index << (lhxw + APLIC_TARGET_HART_IDX_SHIFT);

Thanks.

~ Oleksii

Re: [PATCH v2 20/26] xen/riscv: add missing APLIC register offsets, masks to asm/aplic.h.

[Jan Beulich]

On 05.06.2026 10:48, Oleksii Kurochko wrote:
> On 6/3/26 5:36 PM, Jan Beulich wrote:
>> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>>> --- a/xen/arch/riscv/include/asm/aplic.h
>>> +++ b/xen/arch/riscv/include/asm/aplic.h
>>> @@ -15,6 +15,11 @@
>>>   
>>>   #include <asm/imsic.h>
>>>   
>>> +#define APLIC_REG_OFFSET_MASK   0x3fff
>>
>> This I can see this as wanting to live separately. Yet ...
>>
>>> +#define APLIC_TARGET_IPRIO_MASK 0xff
> 
> This could be dropped as I don't use this mask anymore.
> 
>>> +#define APLIC_TARGET_GUEST_IDX_SHIFT 12
>>> +#define APLIC_TARGET_EIID_MASK  0x7ff
>>
>> ... what (set of) register(s) do these apply to? Perhaps ...
>>
>>> @@ -26,6 +31,36 @@
>>>   #define APLIC_SOURCECFG_SM_LEVEL_HIGH   0x6
>>>   #define APLIC_SOURCECFG_SM_LEVEL_LOW    0x7
>>>   
>>> +#define APLIC_DOMAINCFG         0x0000
>>> +#define APLIC_SOURCECFG_BASE    0x0004
>>> +#define APLIC_SOURCECFG_LAST    0x0ffc
>>> +
>>> +#define APLIC_SMSICFGADDR       0x1bc8
>>> +#define APLIC_SMSICFGADDRH      0x1bcc
>>> +
>>> +#define APLIC_SETIP_BASE        0x1c00
>>> +#define APLIC_SETIP_LAST        0x1c7c
>>> +#define APLIC_SETIPNUM          0x1cdc
>>> +
>>> +#define APLIC_CLRIP_BASE        0x1d00
>>> +#define APLIC_CLRIP_LAST        0x1d7c
>>> +#define APLIC_CLRIPNUM          0x1ddc
>>> +
>>> +#define APLIC_SETIE_BASE        0x1e00
>>> +#define APLIC_SETIE_LAST        0x1e7c
>>> +#define APLIC_SETIENUM          0x1edc
>>> +
>>> +#define APLIC_CLRIE_BASE        0x1f00
>>> +#define APLIC_CLRIE_LAST        0x1f7c
>>> +#define APLIC_CLRIENUM          0x1fdc
>>> +
>>> +#define APLIC_SETIPNUM_LE       0x2000
>>> +
>>> +#define APLIC_GENMSI            0x3000
>>> +
>>> +#define APLIC_TARGET_BASE       0x3004
>>> +#define APLIC_TARGET_LAST       0x3ffc
>>
>> ... these? And then is ...
> 
> Yes, it is applied to target register.
> 
>>
>>>   #define APLIC_TARGET_HART_IDX_SHIFT 18
>>
>> ... this also covering some part of them? Can't they (a) live together and (b)
>> have some kind of connection to what they apply to?
> 
> Agreed, they could live together. The intent was simply to keep all MMIO 
> definitions in sequence,

That would also be fine, but then please add a comment to the register field
definition groups.

Jan

> but I'm okay with grouping macros related to a 
> specific register together.

Re: [PATCH v2 15/26] xen/riscv: introduce (de)initialization helpers for vINTC

[Oleksii Kurochko]

On 6/5/26 9:26 AM, Jan Beulich wrote:
>>>> +    switch ( ver )
>>>> +    {
>>>> +    case INTC_APLIC:
>>>> +        ret = domain_vaplic_init(d);
>>>> +        break;
>>>> +
>>>> +    default:
>>>> +        printk("vintc (ver:%d) isn't implemented\n", ver);
>>> If we take this path for whatever reason, ...
>>>
>>>> +        break;
>>>> +    }
>>>> +
>>>> +    return ret;
>>>> +}
>>>> +
>>>> +void domain_vintc_deinit(struct domain *d)
>>>> +{
>>>> +    const enum intc_version ver = intc_hw_ops->info->hw_version;
>>>> +
>>>> +    switch ( ver )
>>>> +    {
>>>> +    case INTC_APLIC:
>>>> +        domain_vaplic_deinit(d);
>>>> +        break;
>>>> +
>>>> +    default:
>>>> +        printk("vintc (ver:%d) isn't implemented\n", ver);
>>> ... we're also going to take this path (very quickly afterwards), just to
>>> get the same message twice without it being clear why it appears twice.
>> I missed to add printk("%s: ...", __func__, ...). I will update
>> correspondingly if we will follow this way.
> That's not quite what I was after, though. I question the need for this latter
> message.

I think I got you, and I agree that when someone decides to add support 
for a new IC to domain_vintc_init(), it is expected that 
domain_vintc_deinit() will be updated as well. So there is no real need 
to have the same or a very similar message in domain_vintc_deinit().

I will remove the message from domain_vintc_deinit().

Thanks.

~ Oleksii

Re: [PATCH v2 17/26] xen/riscv: generate IMSIC DT node for guest domains

[Oleksii Kurochko]

On 6/5/26 9:31 AM, Jan Beulich wrote:
> On 04.06.2026 16:21, Oleksii Kurochko wrote:
>>
>>
>> On 6/3/26 5:21 PM, Jan Beulich wrote:
>>> On 08.05.2026 16:43, Oleksii Kurochko wrote:
>>>> Guests using the IMSIC interrupt controller require a corresponding
>>>> Device Tree description.
>>>>
>>>> Add support for generating an IMSIC node when building the guest DT.
>>>> This allows guests to discover and use the IMSIC interrupt controller.
>>>>
>>>> Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
>>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>>> ---
>>>> Changes in v2:
>>>>    - s/imsic_make_reg_property/guest_imsic_make_reg_property.
>>>>    - s/imsic_set_interrupt_extended_prop/guest_imsic_set_interrupt_extended_prop.
>>>>    - Use initalizer for regs[] array in imsic_make_reg_property().
>>>>    - Move buf[] insde the for() loop.
>>>>    - Correct check of returned phandle.
>>>>    - Drop local variable len.
>>>>    - /s/XVFREE/xvfree in imsic_set_interrupt_extended_prop().
>>>>    - Drop initializer for local variable data.
>>>>    - s/uint32_t/unsinged int for pos and cpu in imsic_set_interrupt_extended_prop().
>>>>    - Drop next_phandle as it is now in common code.
>>>>    - Introduce vcpu_imsic_deinit.
>>>>    - Refactor vimsic_make_domu_dt_node() to avoid usage of host IMSIC dt node.
>>>> ---
>>>>    xen/arch/riscv/imsic.c                    | 127 +++++++++++++++++++++-
>>>>    xen/arch/riscv/include/asm/guest-layout.h |   2 +
>>>>    2 files changed, 128 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/xen/arch/riscv/imsic.c b/xen/arch/riscv/imsic.c
>>>> index ceea6778d9dc..19cbacdf96e1 100644
>>>> --- a/xen/arch/riscv/imsic.c
>>>> +++ b/xen/arch/riscv/imsic.c
>>>> @@ -13,9 +13,12 @@
>>>>    #include <xen/const.h>
>>>>    #include <xen/cpumask.h>
>>>>    #include <xen/device_tree.h>
>>>> +#include <xen/domain.h>
>>>>    #include <xen/errno.h>
>>>> +#include <xen/fdt-domain-build.h>
>>>>    #include <xen/fdt-kernel.h>
>>>>    #include <xen/init.h>
>>>> +#include <xen/libfdt/libfdt.h>
>>>>    #include <xen/macros.h>
>>>>    #include <xen/sched.h>
>>>>    #include <xen/smp.h>
>>>> @@ -35,6 +38,11 @@ static struct imsic_config imsic_cfg = {
>>>>        .lock = SPIN_LOCK_UNLOCKED,
>>>>    };
>>>>    
>>>> +static unsigned int __ro_after_init guest_num_msis;
>>>
>>> How come this is __ro_after_init, when it's ...
>>>
>>>> @@ -291,6 +299,11 @@ static int imsic_parse_node(const struct dt_device_node *node,
>>>>            return -ENOENT;
>>>>        }
>>>>    
>>>> +    if ( dt_property_read_u32(node, "riscv,num-guest-ids", &tmp) )
>>>> +        guest_num_msis = tmp;
>>>> +    else
>>>> +        guest_num_msis = imsic_cfg.nr_ids;
>>>
>>> ... written by a non-__init function?
>>
>> __ro_after_init should be dropped. I will use __read_mostly instead.
>>
>>    Plus are you again inheriting a host
>>> property into guests without saying why?
>>
>> Based on the discussion for similar place around vAPLIC code in the
>> previous patch I will add here or the comment why or introduce
>> VIMISC_NR_IDS and use it instead.
>>
>>>
>>>> @@ -524,8 +537,120 @@ int __init imsic_init(const struct dt_device_node *node)
>>>>        return rc;
>>>>    }
>>>>    
>>>> +static int __init guest_imsic_make_reg_property(struct domain *d, void *fdt)
>>>
>>> Same question again as to __init throughout here.
>>
>>   From Arm code I understood that libxl creates the interrupt controller
>> node in userspace before handing the FDT to Xen so this and related
>> functions won't be called by userspace.
> 
> Hmm, okay, on the v3 submission I'll have to keep that in mind, and call
> paths will need checking accordingly.
> 

I will update the commit message with clarification why __init is used 
here and connected functions.

>>>> --- a/xen/arch/riscv/include/asm/guest-layout.h
>>>> +++ b/xen/arch/riscv/include/asm/guest-layout.h
>>>> @@ -5,6 +5,8 @@
>>>>    
>>>>    #define GUEST_APLIC_S_BASE 0xd000000
>>>>    
>>>> +#define GUEST_IMSIC_S_BASE 0x28000000
>>>> +
>>>>    #define GUEST_RAM_BANKS   2
>>>
>>> Is this going to become an unannotated collection of (seemingly) random
>>> numbers?
>>
>> Could you please clarify what kind of annotation you would expect to see
>> here? It's just an address, which is typically used for IMSIC by QEMU,
>> so I chose the same notation for the guest address as well.
> 
> Well, how could I have known? The description says nothing, and there are
> also no comments. The numbers here look entirely arbitrary, and if someone
> wanted to change them, it wouldn't be clear at all what constraints need
> taking into consideration.

I will add the comment or update the commit message with information why 
this numbers were choosen.

Thanks.

~ Oleksii

Re: [PATCH v2 14/26] xen/riscv: add very early virtual APLIC (vAPLIC) initialization support

[Oleksii Kurochko]

On 6/4/26 1:29 PM, Oleksii Kurochko wrote:
>>> +
>>> +static const struct vintc_ops vintc_ops = {
>>> +    .vcpu_init = vcpu_vaplic_init,
>>> +};
>>> +
>>> +int __init domain_vaplic_init(struct domain *d)
>>
>> Why __init, and why is there no caller? 
> 
> The caller is in follow-up patch. I will add that to commit message.
> 
> Considering that domain_vintc_init() isn't __init from where 
> domain_vaplic_init() is called then __init should be dropped here. I 
> will do that.
> 
> Plus why is the vCPU-init a hook,
>> but the domain init is not? Either you mean to allow for other ICs, or
>> you you don't.
> 
> IIUC your question domain_vaplic_init() ins't a hook because vaplic 
> structure is allocated dynamically so vintc, vintc->ops and/or vintc- 
>  >init_ops aren't initialized at the moment when vintc->{ops or 
> init_ops}->domain_vaplic_init() is used in domain_vintc_init() (which is 
> introduced in the follow up patch).

As an alternative it could be that vintc->init_ops are moved as a 
variable to int.c and then domain_vaplic_(de)init will be just a hook. 
Something like:

$ git diff
diff --git a/xen/arch/riscv/aplic.c b/xen/arch/riscv/aplic.c
index 86d0a247ef3d..4c4f777f7e1b 100644
--- a/xen/arch/riscv/aplic.c
+++ b/xen/arch/riscv/aplic.c
@@ -28,6 +28,7 @@
  #include <asm/imsic.h>
  #include <asm/intc.h>
  #include <asm/io.h>
+#include <asm/vaplic.h>
  #include <asm/riscv_encoding.h>

  #define APLIC_DEFAULT_PRIORITY  1
@@ -394,6 +395,7 @@ static int __init aplic_preinit(struct 
dt_device_node *node, const void *dat)
      dt_irq_xlate = aplic_irq_xlate;

      register_intc_ops(&aplic_init_ops);
+    register_vintc_init_ops(&vaplic_vintc_init_ops);

      /* Enable supervisor external interrupt */
      csr_set(CSR_SIE, BIT(IRQ_S_EXT, UL));
diff --git a/xen/arch/riscv/include/asm/intc.h 
b/xen/arch/riscv/include/asm/intc.h
index 4842ca08aa54..84b97d6c56f9 100644
--- a/xen/arch/riscv/include/asm/intc.h
+++ b/xen/arch/riscv/include/asm/intc.h
@@ -56,6 +56,8 @@ struct intc_hw_init_ops {
  };

  struct vintc_init_ops {
+    int (*init)(struct domain *d);
+    void (*deinit)(struct domain *d);
      /* Create interrupt controller node for domain */
      int (*make_domu_dt_node)(struct kernel_info *kinfo);
  };
@@ -79,13 +81,13 @@ struct vintc_ops {
  struct vintc {
      unsigned int irq_nums;
      unsigned long *allocated_irqs;
-    const struct vintc_init_ops *init_ops;
      const struct vintc_ops *ops;
  };

  void intc_preinit(void);

  void register_intc_ops(const struct intc_hw_init_ops *init_ops);
+void register_vintc_init_ops(const struct vintc_init_ops *ops);

  void intc_init(void);

diff --git a/xen/arch/riscv/include/asm/vaplic.h 
b/xen/arch/riscv/include/asm/vaplic.h
index 0fa690fcb2d7..7720e6556fcb 100644
--- a/xen/arch/riscv/include/asm/vaplic.h
+++ b/xen/arch/riscv/include/asm/vaplic.h
@@ -15,8 +15,6 @@

  #include <asm/intc.h>

-struct domain;
-
  #define to_vaplic(d) container_of(d->arch.vintc, struct vaplic, vintc)

  struct vaplic_regs {
@@ -31,7 +29,6 @@ struct vaplic {
      paddr_t regs_size;
  };

-int domain_vaplic_init(struct domain *d);
-void domain_vaplic_deinit(struct domain *d);
+extern const struct vintc_init_ops vaplic_vintc_init_ops;

  #endif /* ASM__RISCV__VAPLIC_H */
diff --git a/xen/arch/riscv/intc.c b/xen/arch/riscv/intc.c
index 0934bce3f2f8..304d4def7a4b 100644
--- a/xen/arch/riscv/intc.c
+++ b/xen/arch/riscv/intc.c
@@ -13,18 +13,24 @@

  #include <asm/aia.h>
  #include <asm/intc.h>
-#include <asm/vaplic.h>

  static const struct intc_hw_operations *__ro_after_init intc_hw_ops;

  static const struct intc_hw_init_ops *__initdata intc_hw_init_ops;

+static const struct vintc_init_ops *__ro_after_init vintc_init_ops;
+
  void __init register_intc_ops(const struct intc_hw_init_ops *init_ops)
  {
      intc_hw_ops = init_ops->ops;
      intc_hw_init_ops = init_ops;
  }

+void __init register_vintc_init_ops(const struct vintc_init_ops *ops)
+{
+    vintc_init_ops = ops;
+}
+
  void __init intc_preinit(void)
  {
      if ( acpi_disabled )
@@ -105,29 +111,18 @@ int intc_route_irq_to_guest(struct irq_desc *desc,

  int __init make_intc_domU_node(struct kernel_info *kinfo)
  {
-    struct vintc *vintc = kinfo->bd.d->arch.vintc;
-
-    ASSERT(vintc->init_ops && vintc->init_ops->make_domu_dt_node);

-    return vintc->init_ops->make_domu_dt_node(kinfo);
+    return vintc_init_ops->make_domu_dt_node(kinfo);
  }

  int domain_vintc_init(struct domain *d)
  {
-    int ret = -EOPNOTSUPP;
-    const enum intc_version ver = intc_hw_ops->info->hw_version;
-
-    switch ( ver )
-    {
-    case INTC_APLIC:
-        ret = domain_vaplic_init(d);
-        break;
+    int ret;

-    default:
-        printk("vintc (ver:%d) isn't implemented\n", ver);
-        break;
-    }

+    ret = vintc_init_ops->init(d);
      if ( !ret )
      {
          d->arch.vintc->allocated_irqs =
@@ -141,19 +136,9 @@ int domain_vintc_init(struct domain *d)

  void domain_vintc_deinit(struct domain *d)
  {
-    const enum intc_version ver = intc_hw_ops->info->hw_version;
-
-    switch ( ver )
-    {
-    case INTC_APLIC:
-        domain_vaplic_deinit(d);
-        break;
-
-    default:
-        printk("vintc (ver:%d) isn't implemented\n", ver);
-        break;
-    }
+    ASSERT(vintc_init_ops && vintc_init_ops->deinit);

+    vintc_init_ops->deinit(d);
      xvfree(d->arch.vintc->allocated_irqs);
  }

diff --git a/xen/arch/riscv/vaplic.c b/xen/arch/riscv/vaplic.c
index 57c3433ba03b..301078b8639e 100644
--- a/xen/arch/riscv/vaplic.c
+++ b/xen/arch/riscv/vaplic.c
@@ -397,7 +397,12 @@ static int __init cf_check 
vaplic_make_domu_dt_node(struct kernel_info *kinfo)
      return fdt_end_node(fdt);
  }

-static const struct vintc_init_ops __initdata init_ops = {
+static int domain_vaplic_init(struct domain *d);
+static void domain_vaplic_deinit(struct domain *d);
+
+const struct vintc_init_ops vaplic_vintc_init_ops = {
+    .init              = domain_vaplic_init,
+    .deinit            = domain_vaplic_deinit,
      .make_domu_dt_node = vaplic_make_domu_dt_node,
  };

@@ -408,7 +413,7 @@ static const struct vintc_ops vintc_ops = {
      .emulate_load = vaplic_emulate_load,
  };

-int __init domain_vaplic_init(struct domain *d)
+static int domain_vaplic_init(struct domain *d)
  {
      struct vaplic *vaplic = xvzalloc(struct vaplic);

@@ -417,7 +422,6 @@ int __init domain_vaplic_init(struct domain *d)

      d->arch.vintc = &vaplic->vintc;
      d->arch.vintc->ops = &vintc_ops;
-    d->arch.vintc->init_ops = &init_ops;

      vaplic->regs.domaincfg = APLIC_DOMAINCFG_IE | APLIC_DOMAINCFG_DM |
                               APLIC_DOMAINCFG_RO80;
@@ -428,7 +432,7 @@ int __init domain_vaplic_init(struct domain *d)
      return 0;
  }

-void __init domain_vaplic_deinit(struct domain *d)
+static void domain_vaplic_deinit(struct domain *d)
  {
      struct vaplic *vaplic = to_vaplic(d);


The downside is that we still need register_vintc_init_ops() which will 
called from real interrupt controller code.

Would it be better solution?

~ Oleksii

Re: [PATCH v2 14/26] xen/riscv: add very early virtual APLIC (vAPLIC) initialization support

[Jan Beulich]

On 05.06.2026 13:59, Oleksii Kurochko wrote:
> On 6/4/26 1:29 PM, Oleksii Kurochko wrote:
> @@ -105,29 +111,18 @@ int intc_route_irq_to_guest(struct irq_desc *desc,
> 
>   int __init make_intc_domU_node(struct kernel_info *kinfo)
>   {
> -    struct vintc *vintc = kinfo->bd.d->arch.vintc;
> -
> -    ASSERT(vintc->init_ops && vintc->init_ops->make_domu_dt_node);
> 
> -    return vintc->init_ops->make_domu_dt_node(kinfo);
> +    return vintc_init_ops->make_domu_dt_node(kinfo);
>   }
> 
>   int domain_vintc_init(struct domain *d)
>   {
> -    int ret = -EOPNOTSUPP;
> -    const enum intc_version ver = intc_hw_ops->info->hw_version;
> -
> -    switch ( ver )
> -    {
> -    case INTC_APLIC:
> -        ret = domain_vaplic_init(d);
> -        break;
> +    int ret;
> 
> -    default:
> -        printk("vintc (ver:%d) isn't implemented\n", ver);
> -        break;
> -    }
> 
> +    ret = vintc_init_ops->init(d);
>       if ( !ret )
>       {
>           d->arch.vintc->allocated_irqs =

Thinking about it some more, in particular the longer term case of domains
not inheriting HW properties may end up better without this change, i.e.
keeping the switch() that you had (and a direct call).

Jan