[PATCH v1 00/17] [RISC-V] virtual interrupt controller (vAPLIC/vIMSIC) support

[PATCH v1 00/17] [RISC-V] virtual interrupt controller (vAPLIC/vIMSIC) support

[Oleksii Kurochko]

Hi all,

This series adds the initial virtual interrupt controller (vINTC) support
for RISC-V guests in Xen, based on the Advanced Interrupt Architecture
(AIA): a virtual APLIC (vAPLIC) in MSI mode backed by a virtual IMSIC
(vIMSIC) using hardware guest interrupt files.

Rather than emulating APLIC in direct-delivery mode (which requires
trap-and-emulate for every interrupt and is costly), the series targets
IMSIC from the start. AIA lets a hart implement several "guest interrupt
files" (up to GEILEN), so external interrupts can be delivered to a vCPU
directly by hardware via the VGEIN field of hstatus, without a hypervisor
round-trip. Xen only has to emulate the APLIC MMIO programming interface
and route the guest's intent onto the physical MSI topology; interrupt
delivery itself stays in hardware.

The work breaks down into a few logical blocks:

Physical APLIC/AIA groundwork (patches 1-3)
  - Correctly track IRQ_DISABLED across runtime enable/disable and order
    the ->status update against the IMSIC CSR write.
  - Per-pCPU VGEIN (guest interrupt file) allocator: assign/release a
    hardware guest file to a vCPU and program hstatus.VGEIN.
  - Add the missing APLIC register offsets/masks needed by both the
    physical and virtual APLIC code (no functional change).

Device-agnostic MMIO dispatch + vAPLIC emulation (patches 4-5)
  - A per-domain MMIO handler table modelled on Arm's framework, so
    emulated devices self-register their GPA ranges and the fault path
    stays agnostic via a single try_handle_mmio() entry point.
  - vAPLIC MMIO read/write emulation. Writes are gated by the domain's
    authorised-IRQ bitmap so a guest cannot touch interrupts it does not
    own, and TARGET writes are translated from virtual to physical
    hart/guest-file indices. Delegation (SOURCECFG.D) is not yet
    supported.

vIMSIC guest interrupt files and state (patches 6-9)
  - Stage-2 map a vCPU's physical guest interrupt file to the fixed
    per-vCPU GPA page the guest expects at offset 0.
  - vcpu_aia_init(): assign a VGEIN, map its guest file and record the
    IMSIC state as a consistent unit.
  - IMSIC state save/restore (currently tracking which pCPU owns the
    guest file, needed because the pCPU id is part of the MSI address).
  - has_msi_support() helper to decide whether IMSIC state has to be
    saved/restored.

vINTC state save/restore plumbing (patches 10-11)
  - vintc_state_{save,restore}() wrappers over new store/restore hooks in
    struct vintc_ops, and the vAPLIC implementation of those hooks. No
    callers are wired up yet; the context-switch calls arrive with vCPU
    context switch support.

Trap and instruction emulation infrastructure (patches 12-17)
  - Extend the exception-table format with type/data fields and add
    EX_TYPE_TRAP_INFO so fixups can capture sepc/scause/stval.
  - riscv_vcpu_unpriv_read() (HLV/HLVX) to read guest memory/instructions
    safely, and riscv_vcpu_trap_redirect() to forward a synchronous trap
    back into the guest's VS-mode handler.
  - A guest page-fault handler that decodes the trapped load/store
    instruction (HTINST, or an unprivileged fetch as a fallback) and
    dispatches the MMIO access through try_handle_mmio().

Note on versioning: the series is posted as v1 as a freshly split-out
series, but several patches carry v2/v3 changelogs because they were
previously circulated as part of a larger another patches [1] from which the
current depends.

CI tests: https://gitlab.com/xen-project/people/olkur/xen/-/pipelines/2690874319

[1] https://lore.kernel.org/xen-devel/cover.1784559209.git.oleksii.kurochko@gmail.com/T/#t

Oleksii Kurochko (17):
  xen/riscv: manage IRQ_DISABLED flag in APLIC irq enable/disable
    callbacks
  xen/riscv: add basic VGEIN management for AIA guests
  xen/riscv: add missing APLIC register offsets, masks to asm/aplic.h
  xen/riscv: introduce device-agnostic MMIO emulation dispatch
  xen/riscv: implement virtual APLIC MMIO emulation
  xen/riscv: map IMSIC interrupt file for vCPUs
  xen/riscv: introduce vCPU AIA initialization
  xen/riscv: add IMSIC state save/restore
  xen/riscv: add helper to check APLIC MSI mode
  xen/riscv: introduce vintc_state_{save,restore}()
  xen/riscv: add vAPLIC state save/restore hooks
  xen/riscv: extend exception tables with type and data fields
  xen/riscv: add unprivileged guest memory read helper
  xen/riscv: add guest page fault handling stub
  xen/riscv: implement trap redirection to a guest
  xen/riscv: add guest load emulation for trapped MMIO accesses
  xen/riscv: add guest store emulation for trapped MMIO accesses

 xen/arch/riscv/Makefile                   |   1 +
 xen/arch/riscv/aia.c                      | 175 ++++++++++++
 xen/arch/riscv/aplic-priv.h               |   2 +
 xen/arch/riscv/aplic.c                    |  83 +++++-
 xen/arch/riscv/domain.c                   |   4 +
 xen/arch/riscv/extable.c                  |  60 +++-
 xen/arch/riscv/guestcopy.c                | 145 ++++++++++
 xen/arch/riscv/imsic.c                    |  99 +++++++
 xen/arch/riscv/include/asm/aia.h          |  10 +
 xen/arch/riscv/include/asm/aplic.h        |  61 +++++
 xen/arch/riscv/include/asm/domain.h       |   3 +
 xen/arch/riscv/include/asm/extable.h      |  62 +++--
 xen/arch/riscv/include/asm/gpr-num.h      |  33 +++
 xen/arch/riscv/include/asm/guest_access.h |   8 +
 xen/arch/riscv/include/asm/imsic.h        |  17 ++
 xen/arch/riscv/include/asm/intc.h         |   9 +
 xen/arch/riscv/include/asm/mmio.h         |  63 +++++
 xen/arch/riscv/include/asm/processor.h    |   2 +
 xen/arch/riscv/include/asm/traps.h        |  12 +
 xen/arch/riscv/include/asm/vaplic.h       |   6 +
 xen/arch/riscv/intc.c                     |  14 +
 xen/arch/riscv/mmio.c                     | 145 ++++++++++
 xen/arch/riscv/traps.c                    | 279 +++++++++++++++++++
 xen/arch/riscv/vaplic.c                   | 319 ++++++++++++++++++++++
 24 files changed, 1590 insertions(+), 22 deletions(-)
 create mode 100644 xen/arch/riscv/include/asm/gpr-num.h
 create mode 100644 xen/arch/riscv/include/asm/mmio.h
 create mode 100644 xen/arch/riscv/mmio.c

-- 
2.54.0

[PATCH v1 01/17] xen/riscv: manage IRQ_DISABLED flag in APLIC irq enable/disable callbacks

[Oleksii Kurochko]

desc->status is only set once during setup_irq(), but interrupts can be
enabled/disabled at runtime, so update it in the corresponding callbacks.

For the purposes of the FENCE instruction, CSR read accesses are
classified as device input (I) and CSR write accesses as device output
(O), while the barriers used by spin locks (fence rw,rw) only order
normal memory accesses. An explicit wmb() (fence ow,ow) is therefore
added in aplic_irq_{enable,disable}() to order the desc->status update
with respect to the IMSIC CSR write.

Fixes: d4676a1398bc5 ("xen/riscv: implementation of aplic and imsic operations")
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v3:
 - Add the comment above wmb() in aplic_irq_enable() and add wmb() also in
   *_disable().
 - Update the commit message: drop info about wmb() and add information why
   it is safe to drop update of desc->status from setup_irq().
---
Changes in v2:
 - New patch
---
---
 xen/arch/riscv/aplic.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/xen/arch/riscv/aplic.c b/xen/arch/riscv/aplic.c
index fe319041cece..3681f0669efb 100644
--- a/xen/arch/riscv/aplic.c
+++ b/xen/arch/riscv/aplic.c
@@ -136,6 +136,15 @@ static void cf_check aplic_irq_enable(struct irq_desc *desc)
 
     spin_lock(&aplic.lock);
 
+    desc->status &= ~IRQ_DISABLED;
+    /*
+     * wmb() (fence ow,ow) orders the ->status memory write (w) before the
+     * CSR write inside imsic_irq_enable() (device output, o on RISC-V).
+     * arch_lock_release_barrier() uses fence rw,rw which does not cover
+     * device output (o), so wmb() is required to close that gap.
+     */
+    wmb();
+
     /* Enable interrupt in IMSIC */
     imsic_irq_enable(desc->irq);
 
@@ -163,6 +172,16 @@ static void cf_check aplic_irq_disable(struct irq_desc *desc)
 
     /* Disable interrupt in IMSIC */
     imsic_irq_disable(desc->irq);
+    /*
+     * wmb() (fence ow,ow) ensures the CSR write (device output, o) inside
+     * imsic_irq_disable() is globally visible before ->status is marked
+     * IRQ_DISABLED. imsic_irq_disable()'s spin_unlock uses fence rw,rw
+     * which does not order device output (o) writes before subsequent
+     * memory writes (w), so an explicit wmb() is needed here.
+     */
+    wmb();
+
+    desc->status |= IRQ_DISABLED;
 
     spin_unlock(&aplic.lock);
 }
-- 
2.54.0

Re: [PATCH v1 01/17] xen/riscv: manage IRQ_DISABLED flag in APLIC irq enable/disable callbacks

[Jan Beulich]

On 20.07.2026 18:01, Oleksii Kurochko wrote:
> desc->status is only set once during setup_irq(), but interrupts can be
> enabled/disabled at runtime, so update it in the corresponding callbacks.
> 
> For the purposes of the FENCE instruction, CSR read accesses are
> classified as device input (I) and CSR write accesses as device output
> (O), while the barriers used by spin locks (fence rw,rw) only order
> normal memory accesses. An explicit wmb() (fence ow,ow) is therefore
> added in aplic_irq_{enable,disable}() to order the desc->status update
> with respect to the IMSIC CSR write.
> 
> Fixes: d4676a1398bc5 ("xen/riscv: implementation of aplic and imsic operations")
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Acked-by: Jan Beulich <jbeulich@suse.com>

Re: [PATCH v1 01/17] xen/riscv: manage IRQ_DISABLED flag in APLIC irq enable/disable callbacks

[Baptiste Le Duc]

[-- Attachment #1: Type: text/plain, Size: 888 bytes --]

On Mon, 20 Jul 2026 18:01:59 +0200, Oleksii Kurochko <oleksii.kurochko@gmail.com> wrote:
> desc->status is only set once during setup_irq(), but interrupts can be
> enabled/disabled at runtime, so update it in the corresponding callbacks.
> 
> For the purposes of the FENCE instruction, CSR read accesses are
> classified as device input (I) and CSR write accesses as device output
> (O), while the barriers used by spin locks (fence rw,rw) only order
> normal memory accesses. An explicit wmb() (fence ow,ow) is therefore
> added in aplic_irq_{enable,disable}() to order the desc->status update
> with respect to the IMSIC CSR write.
> 
> [...]

Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech>

-- 
Baptiste Le Duc <baptiste.le-duc@vates.tech>


--
Baptiste Le Duc | Vates XCP-ng Intern

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech

[PATCH v1 02/17] xen/riscv: add basic VGEIN management for AIA guests

[Oleksii Kurochko]

It was decided to add support for IMSIC from the start instead of having APLIC
operate in direct delivery mode, as it requires a trap-and-emulation approach,
which is not optimal from a performance standpoint.

AIA provides a hardware-accelerated mechanism for delivering external
interrupts to domains via "guest interrupt files" located in IMSIC.
A single physical hart can implement multiple such files (up to GEILEN),
allowing several virtual harts to receive interrupts directly from hardware.

Introduce per-CPU tracking of guest interrupt file identifiers (VGEIN)
for systems implementing AIA specification. Each CPU maintains
a bitmap describing which guest interrupt files are currently in use.

Add helpers to initialize the bitmap based on the number of available
guest interrupt files (GEILEN), assign a VGEIN to a vCPU, and release it
when no longer needed. When assigning a VGEIN, the corresponding value
is written to the VGEIN field of the guest hstatus register so that
VS-level external interrupts are delivered from the selected interrupt
file.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v3:
 - Rephrase per-CPU comment from "Bitmap for each physical cpus..." to
   "VGEIN control structure for each physical CPU...".
 - Use %u for cpu in printk("cpu%u.geilen=%u\n").
 - Fix missing \n in "AIA: failed to init vgein for CPU%u\n".
 - Shorten dprintk message to "vgein_init() failed: %d\n".
 - Use %u instead of %d for v->processor in gprintk() calls.
 - Drop update of ->hstatus in vgein_{assign,release} and let the caller
   to do that. For example, it could be useful in the case of migration
   vCPU from one pCPU to another and so vcpu->hstatus will be updated as a
   last step of this process.
 - Fill ->owners in vgein_{assign,release} just from the start.
---
Changes in v2:
 - add static for defintion of vgein_bmp;
 - Drop declarartion of vgein_bmp from aia.h.
 - Move declaration of 'struct vgein_bmp' from aia.h to aia.c as all the
   management is inside aia.c.
 - Instead of decrement of vgein->geilen just update the wait how it is
   initialized.
 - Return -EOPNOTSUPP in vgein_init() instead of BUG_ON().
 - Use %u to print unsigned int.
 - make bmp field in vgein_bmp not a pointer.
 - allocate owners dynamically.
 - Drop unnessary blank lines.
 - use find_first_zero_bit() instead of bitmap_weight() to find a free slot
   for vgein number.
 - Drop the section number for the comment.
 - Start to search from bitnum 1 for free vgein_id, as bitnum 0 is reserved to
   tell that no guest extrenal interrupt number is used. Thereby drop vgein_id++
   at the end of vgein_assign().
 - s/bitmap_set/__set_bit.
 - s/bitmap_clear/__clear_bit.
 - as vgein_init() is needed to be invoked once per CPU being brought up, drop
   __init for it.
 - Return vgein_id == 0 if vgein_id is higher then maximun supported by h/w
   VGEIN.
 - Add check in vgein_relase() that vgein is 0 and if it is there is nothing
   is needed to do.
 - Use gdprintk instead of printk() in vgein_{assign,release}.
 - Add the claryfing comment above geilen field.
 - Drop ASSERT in vgein_assign() and return just vgein_id = 0 in the case when
   there is no aviablable h/w VGEINs.
 - Make vgein_init() static.
---
---
 xen/arch/riscv/aia.c             | 145 +++++++++++++++++++++++++++++++
 xen/arch/riscv/include/asm/aia.h |   8 ++
 2 files changed, 153 insertions(+)

diff --git a/xen/arch/riscv/aia.c b/xen/arch/riscv/aia.c
index e31c9c2d24b6..4f7f46f58f0b 100644
--- a/xen/arch/riscv/aia.c
+++ b/xen/arch/riscv/aia.c
@@ -1,11 +1,33 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 
+#include <xen/bitmap.h>
+#include <xen/cpu.h>
 #include <xen/errno.h>
 #include <xen/init.h>
 #include <xen/sections.h>
+#include <xen/sched.h>
+#include <xen/spinlock.h>
 #include <xen/types.h>
+#include <xen/xvmalloc.h>
 
+#include <asm/aia.h>
 #include <asm/cpufeature.h>
+#include <asm/csr.h>
+#include <asm/current.h>
+
+struct vgein_ctrl {
+    unsigned long bmp;
+    spinlock_t lock;
+    struct vcpu **owners;
+    /* The least-significant bits are implemented first, apart from bit 0 */
+    unsigned int geilen;
+};
+
+/*
+ * VGEIN control structure for each physical CPU to track which VS (guest)
+ * interrupt file IDs are in use.
+ */
+static DEFINE_PER_CPU(struct vgein_ctrl, vgein);
 
 static bool __ro_after_init _aia_usable;
 
@@ -14,10 +36,133 @@ bool aia_usable(void)
     return _aia_usable;
 }
 
+static int vgein_init(unsigned int cpu)
+{
+    struct vgein_ctrl *vgein = &per_cpu(vgein, cpu);
+
+    csr_write(CSR_HGEIE, -1UL);
+    vgein->geilen = flsl(csr_read(CSR_HGEIE) >> 1);
+    csr_write(CSR_HGEIE, 0);
+
+    printk("cpu%u.geilen=%u\n", cpu, vgein->geilen);
+
+    if ( !vgein->geilen )
+        return -EOPNOTSUPP;
+
+    vgein->owners = xvzalloc_array(struct vcpu *, vgein->geilen);
+    if ( !vgein->owners )
+        return -ENOMEM;
+
+    spin_lock_init(&vgein->lock);
+
+    return 0;
+}
+
+static int cf_check cpu_callback(struct notifier_block *nfb, unsigned long action,
+                        void *hcpu)
+{
+    unsigned int cpu = (unsigned long)hcpu;
+    int rc = 0;
+
+    switch ( action )
+    {
+    case CPU_STARTING:
+        rc = vgein_init(cpu);
+        if ( rc )
+            printk("AIA: failed to init vgein for CPU%u\n", cpu);
+        break;
+    }
+
+    return notifier_from_errno(rc);
+}
+
+static struct notifier_block cpu_nfb = {
+    .notifier_call = cpu_callback,
+};
+
 void __init aia_init(void)
 {
+    int rc;
+
     if ( !riscv_isa_extension_available(NULL, RISCV_ISA_EXT_ssaia) )
+    {
+        dprintk(XENLOG_WARNING, "SSAIA isn't present in riscv,isa\n");
         return;
+    }
+
+    if ( (rc = vgein_init(0)) )
+    {
+        dprintk(XENLOG_ERR, "vgein_init() failed: %d\n", rc);
+        return;
+    }
 
     _aia_usable = true;
+
+    register_cpu_notifier(&cpu_nfb);
+}
+
+unsigned int vgein_assign(struct vcpu *v)
+{
+    unsigned int vgein_id;
+    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
+    unsigned long *bmp = &vgein->bmp;
+    unsigned long flags;
+
+    if ( !vgein->geilen )
+        return 0;
+
+    spin_lock_irqsave(&vgein->lock, flags);
+    /*
+     * The vgein_id shouldn't be zero, as it will indicate that no guest
+     * external interrupt source is selected for VS-level external interrupts
+     * according to RISC-V privileged spec:
+     *   Hypervisor Status Register (hstatus) in RISC-V privileged spec:
+     *
+     *   The VGEIN (Virtual Guest External Interrupt Number) field selects
+     *   a guest external interrupt source for VS-level external interrupts.
+     *   VGEIN is a WLRL field that must be able to hold values between zero
+     *   and the maximum guest external interrupt number (known as GEILEN),
+     *   inclusive.
+     *   When VGEIN=0, no guest external interrupt source is selected for
+     *   VS-level external interrupts.
+     *
+     * So start to search from bit number 1.
+     */
+    vgein_id = find_next_zero_bit(bmp, vgein->geilen + 1, 1);
+
+    if ( vgein_id > vgein->geilen )
+        vgein_id = 0;
+    else
+    {
+        __set_bit(vgein_id, bmp);
+        vgein->owners[vgein_id] = v;
+    }
+
+    spin_unlock_irqrestore(&vgein->lock, flags);
+
+#ifdef VGEIN_DEBUG
+    gprintk(XENLOG_DEBUG, "%s: %pv: vgein_id(%u), xen_cpu%u_bmp=%#lx\n",
+           __func__, v, vgein_id, v->processor, *bmp);
+#endif
+
+    return vgein_id;
+}
+
+void vgein_release(struct vcpu *v, unsigned int vgein_id)
+{
+    unsigned long flags;
+    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
+
+    if ( !vgein_id )
+        return;
+
+    spin_lock_irqsave(&vgein->lock, flags);
+    __clear_bit(vgein_id, &vgein->bmp);
+    vgein->owners[vgein_id] = NULL;
+    spin_unlock_irqrestore(&vgein->lock, flags);
+
+#ifdef VGEIN_DEBUG
+    gprintk(XENLOG_DEBUG, "%s: vgein_id(%u), xen_cpu%u_bmp=%#lx\n",
+           __func__, vgein_id, v->processor, vgein->bmp);
+#endif
 }
diff --git a/xen/arch/riscv/include/asm/aia.h b/xen/arch/riscv/include/asm/aia.h
index aaa4bf91fc75..c67be0069a1d 100644
--- a/xen/arch/riscv/include/asm/aia.h
+++ b/xen/arch/riscv/include/asm/aia.h
@@ -3,8 +3,16 @@
 #ifndef RISCV_AIA_H
 #define RISCV_AIA_H
 
+#include <xen/percpu.h>
+#include <xen/spinlock.h>
+
+struct vcpu;
+
 bool aia_usable(void);
 
 void aia_init(void);
 
+unsigned int vgein_assign(struct vcpu *v);
+void vgein_release(struct vcpu *v, unsigned int vgein_id);
+
 #endif /* RISCV_AIA_H */
-- 
2.54.0

Re: [PATCH v1 02/17] xen/riscv: add basic VGEIN management for AIA guests

[Jan Beulich]

On 20.07.2026 18:02, Oleksii Kurochko wrote:
> It was decided to add support for IMSIC from the start instead of having APLIC
> operate in direct delivery mode, as it requires a trap-and-emulation approach,
> which is not optimal from a performance standpoint.
> 
> AIA provides a hardware-accelerated mechanism for delivering external
> interrupts to domains via "guest interrupt files" located in IMSIC.
> A single physical hart can implement multiple such files (up to GEILEN),
> allowing several virtual harts to receive interrupts directly from hardware.
> 
> Introduce per-CPU tracking of guest interrupt file identifiers (VGEIN)
> for systems implementing AIA specification. Each CPU maintains
> a bitmap describing which guest interrupt files are currently in use.
> 
> Add helpers to initialize the bitmap based on the number of available
> guest interrupt files (GEILEN), assign a VGEIN to a vCPU, and release it
> when no longer needed. When assigning a VGEIN, the corresponding value
> is written to the VGEIN field of the guest hstatus register so that
> VS-level external interrupts are delivered from the selected interrupt
> file.

And when exactly is this "assignment" intended to occur? vgein_assign() and
vgein_release() have no callers here, so this remains entirely unclear.

> @@ -14,10 +36,133 @@ bool aia_usable(void)
>      return _aia_usable;
>  }
>  
> +static int vgein_init(unsigned int cpu)
> +{
> +    struct vgein_ctrl *vgein = &per_cpu(vgein, cpu);
> +
> +    csr_write(CSR_HGEIE, -1UL);
> +    vgein->geilen = flsl(csr_read(CSR_HGEIE) >> 1);
> +    csr_write(CSR_HGEIE, 0);
> +
> +    printk("cpu%u.geilen=%u\n", cpu, vgein->geilen);

At most dprintk(), I'd say. Better drop altogether.

> +    if ( !vgein->geilen )
> +        return -EOPNOTSUPP;
> +
> +    vgein->owners = xvzalloc_array(struct vcpu *, vgein->geilen);
> +    if ( !vgein->owners )
> +        return -ENOMEM;
> +
> +    spin_lock_init(&vgein->lock);
> +
> +    return 0;
> +}
> +
> +static int cf_check cpu_callback(struct notifier_block *nfb, unsigned long action,

Nit: Line length.

> +                        void *hcpu)

Nit: Indentation.

> +{
> +    unsigned int cpu = (unsigned long)hcpu;
> +    int rc = 0;
> +
> +    switch ( action )
> +    {
> +    case CPU_STARTING:
> +        rc = vgein_init(cpu);
> +        if ( rc )
> +            printk("AIA: failed to init vgein for CPU%u\n", cpu);
> +        break;
> +    }
> +
> +    return notifier_from_errno(rc);
> +}

Where's the freeing of the allocation vgein_init(), when CPU bringup fails
or a CPU was brought down?

> +static struct notifier_block cpu_nfb = {
> +    .notifier_call = cpu_callback,
> +};
> +
>  void __init aia_init(void)
>  {
> +    int rc;
> +
>      if ( !riscv_isa_extension_available(NULL, RISCV_ISA_EXT_ssaia) )
> +    {
> +        dprintk(XENLOG_WARNING, "SSAIA isn't present in riscv,isa\n");
>          return;
> +    }
> +
> +    if ( (rc = vgein_init(0)) )
> +    {
> +        dprintk(XENLOG_ERR, "vgein_init() failed: %d\n", rc);
> +        return;
> +    }
>  
>      _aia_usable = true;
> +
> +    register_cpu_notifier(&cpu_nfb);
> +}
> +
> +unsigned int vgein_assign(struct vcpu *v)
> +{
> +    unsigned int vgein_id;
> +    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
> +    unsigned long *bmp = &vgein->bmp;
> +    unsigned long flags;
> +
> +    if ( !vgein->geilen )
> +        return 0;
> +
> +    spin_lock_irqsave(&vgein->lock, flags);

Because it's unclear where this is to be called from, it's also unclear whether
a lock is needed here (and if so whether a plain spin lock is appropriate).

> +    /*
> +     * The vgein_id shouldn't be zero, as it will indicate that no guest
> +     * external interrupt source is selected for VS-level external interrupts
> +     * according to RISC-V privileged spec:
> +     *   Hypervisor Status Register (hstatus) in RISC-V privileged spec:
> +     *
> +     *   The VGEIN (Virtual Guest External Interrupt Number) field selects
> +     *   a guest external interrupt source for VS-level external interrupts.
> +     *   VGEIN is a WLRL field that must be able to hold values between zero
> +     *   and the maximum guest external interrupt number (known as GEILEN),
> +     *   inclusive.
> +     *   When VGEIN=0, no guest external interrupt source is selected for
> +     *   VS-level external interrupts.
> +     *
> +     * So start to search from bit number 1.
> +     */
> +    vgein_id = find_next_zero_bit(bmp, vgein->geilen + 1, 1);
> +
> +    if ( vgein_id > vgein->geilen )
> +        vgein_id = 0;
> +    else
> +    {
> +        __set_bit(vgein_id, bmp);
> +        vgein->owners[vgein_id] = v;

Again somewhat related to is being unclear how the function is going to be used,
it also remains unclear what ->owners[] is going to be needed for. Right now the
array is only ever written to.

> +    }
> +
> +    spin_unlock_irqrestore(&vgein->lock, flags);
> +
> +#ifdef VGEIN_DEBUG
> +    gprintk(XENLOG_DEBUG, "%s: %pv: vgein_id(%u), xen_cpu%u_bmp=%#lx\n",
> +           __func__, v, vgein_id, v->processor, *bmp);
> +#endif
> +
> +    return vgein_id;
> +}
> +
> +void vgein_release(struct vcpu *v, unsigned int vgein_id)
> +{
> +    unsigned long flags;
> +    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
> +
> +    if ( !vgein_id )
> +        return;
> +
> +    spin_lock_irqsave(&vgein->lock, flags);
> +    __clear_bit(vgein_id, &vgein->bmp);
> +    vgein->owners[vgein_id] = NULL;

If already you track the vCPU, also assert that prior to clearing the array
slot it has the expected value? For the bit being cleared, maybe also

    if ( !__test_and_clear_bit(vgein_id, &vgein->bmp) )
        ASSERT_UNREACHABLE();

? Yet as said - much remains unclear without knowing how all of this is
meant to be used.

Jan

Re: [PATCH v1 02/17] xen/riscv: add basic VGEIN management for AIA guests

[Oleksii Kurochko]

On 7/27/26 5:41 PM, Jan Beulich wrote:
> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>> It was decided to add support for IMSIC from the start instead of having APLIC
>> operate in direct delivery mode, as it requires a trap-and-emulation approach,
>> which is not optimal from a performance standpoint.
>>
>> AIA provides a hardware-accelerated mechanism for delivering external
>> interrupts to domains via "guest interrupt files" located in IMSIC.
>> A single physical hart can implement multiple such files (up to GEILEN),
>> allowing several virtual harts to receive interrupts directly from hardware.
>>
>> Introduce per-CPU tracking of guest interrupt file identifiers (VGEIN)
>> for systems implementing AIA specification. Each CPU maintains
>> a bitmap describing which guest interrupt files are currently in use.
>>
>> Add helpers to initialize the bitmap based on the number of available
>> guest interrupt files (GEILEN), assign a VGEIN to a vCPU, and release it
>> when no longer needed. When assigning a VGEIN, the corresponding value
>> is written to the VGEIN field of the guest hstatus register so that
>> VS-level external interrupts are delivered from the selected interrupt
>> file.
> 
> And when exactly is this "assignment" intended to occur? vgein_assign() and
> vgein_release() have no callers here, so this remains entirely unclear.

[A] Agreed, I should have added that information to the commit message:

VGEIN is assigned (via vgein_assign()) before jumping to the new vCPU 
execution context (in continue_new_vcpu()) and is re-assigned during 
vCPU migration from one pCPU to another.

VGEIN is released (via vgein_release()) on the old pCPU during migration.


> 
>> @@ -14,10 +36,133 @@ bool aia_usable(void)
>>       return _aia_usable;
>>   }
>>   
>> +static int vgein_init(unsigned int cpu)
>> +{
>> +    struct vgein_ctrl *vgein = &per_cpu(vgein, cpu);
>> +
>> +    csr_write(CSR_HGEIE, -1UL);
>> +    vgein->geilen = flsl(csr_read(CSR_HGEIE) >> 1);
>> +    csr_write(CSR_HGEIE, 0);
>> +
>> +    printk("cpu%u.geilen=%u\n", cpu, vgein->geilen);
> 
> At most dprintk(), I'd say. Better drop altogether.

I will drop it.

> 
>> +    if ( !vgein->geilen )
>> +        return -EOPNOTSUPP;
>> +
>> +    vgein->owners = xvzalloc_array(struct vcpu *, vgein->geilen);
>> +    if ( !vgein->owners )
>> +        return -ENOMEM;
>> +
>> +    spin_lock_init(&vgein->lock);
>> +
>> +    return 0;
>> +}
>> +
>> +static int cf_check cpu_callback(struct notifier_block *nfb, unsigned long action,
> 
> Nit: Line length.
> 
>> +                        void *hcpu)
> 
> Nit: Indentation.
> 
>> +{
>> +    unsigned int cpu = (unsigned long)hcpu;
>> +    int rc = 0;
>> +
>> +    switch ( action )
>> +    {
>> +    case CPU_STARTING:
>> +        rc = vgein_init(cpu);
>> +        if ( rc )
>> +            printk("AIA: failed to init vgein for CPU%u\n", cpu);
>> +        break;
>> +    }
>> +
>> +    return notifier_from_errno(rc);
>> +}
> 
> Where's the freeing of the allocation vgein_init(), when CPU bringup fails
> or a CPU was brought down?
I'll add the following:

case CPU_UP_CANCELED:
case CPU_DEAD:
     vgein_free(cpu);
     break;

and:

static void vgein_free(unsigned int cpu)
{
     struct vgein_ctrl *vgein = &per_cpu(vgein, cpu);

     ASSERT(!vgein->bmp);

     vgein->geilen = 0;
     XVFREE(vgein->owners);
}

I'm also wondering whether vgein_init() should be moved to 
CPU_UP_PREPARE. If vgein_init() fails in CPU_STARTING, the hypervisor 
will stop instead of simply ignoring the CPU.

However, in CPU_UP_PREPARE we don't yet know the value of GEILEN, which 
is needed to allocate vgein->owners. As I understand it, CPU_UP_PREPARE 
is not executed on the CPU that is being brought up.


> 
>> +static struct notifier_block cpu_nfb = {
>> +    .notifier_call = cpu_callback,
>> +};
>> +
>>   void __init aia_init(void)
>>   {
>> +    int rc;
>> +
>>       if ( !riscv_isa_extension_available(NULL, RISCV_ISA_EXT_ssaia) )
>> +    {
>> +        dprintk(XENLOG_WARNING, "SSAIA isn't present in riscv,isa\n");
>>           return;
>> +    }
>> +
>> +    if ( (rc = vgein_init(0)) )
>> +    {
>> +        dprintk(XENLOG_ERR, "vgein_init() failed: %d\n", rc);
>> +        return;
>> +    }
>>   
>>       _aia_usable = true;
>> +
>> +    register_cpu_notifier(&cpu_nfb);
>> +}
>> +
>> +unsigned int vgein_assign(struct vcpu *v)
>> +{
>> +    unsigned int vgein_id;
>> +    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
>> +    unsigned long *bmp = &vgein->bmp;
>> +    unsigned long flags;
>> +
>> +    if ( !vgein->geilen )
>> +        return 0;
>> +
>> +    spin_lock_irqsave(&vgein->lock, flags);
> 
> Because it's unclear where this is to be called from, it's also unclear whether
> a lock is needed here (and if so whether a plain spin lock is appropriate).

Based on what I wrote in [A] above a lock is defintely needed as it 
could be that vgein_release() is called for old pCPU during migration 
and at the same time old pCPU could call vgein_assign() so we want to 
keep vgein bitmap consistent.

Regarding why _irqsave() it is mostly connected to ...

> 
>> +    /*
>> +     * The vgein_id shouldn't be zero, as it will indicate that no guest
>> +     * external interrupt source is selected for VS-level external interrupts
>> +     * according to RISC-V privileged spec:
>> +     *   Hypervisor Status Register (hstatus) in RISC-V privileged spec:
>> +     *
>> +     *   The VGEIN (Virtual Guest External Interrupt Number) field selects
>> +     *   a guest external interrupt source for VS-level external interrupts.
>> +     *   VGEIN is a WLRL field that must be able to hold values between zero
>> +     *   and the maximum guest external interrupt number (known as GEILEN),
>> +     *   inclusive.
>> +     *   When VGEIN=0, no guest external interrupt source is selected for
>> +     *   VS-level external interrupts.
>> +     *
>> +     * So start to search from bit number 1.
>> +     */
>> +    vgein_id = find_next_zero_bit(bmp, vgein->geilen + 1, 1);
>> +
>> +    if ( vgein_id > vgein->geilen )
>> +        vgein_id = 0;
>> +    else
>> +    {
>> +        __set_bit(vgein_id, bmp);
>> +        vgein->owners[vgein_id] = v;
> 
> Again somewhat related to is being unclear how the function is going to be used,
> it also remains unclear what ->owners[] is going to be needed for. Right now the
> array is only ever written to.

->owners[] is used in IRQ context to wake up a vCPU. For example, if a 
vCPU has been descheduled, we need to set the corresponding CSR_HGEIE[] 
bit so that when an interrupt associated with that vCPU occurs, it traps 
into the hgei_interrupt() handler, which then wakes the vCPU. (all of 
that isn't introduced now but I thought it would be useful to track 
->owners[] just from the start).

Since ->owners[] is accessed from both IRQ-safe (hgei_interrupt()) and 
IRQ-unsafe (vCPU migration) contexts, we specifically need the 
_irqsave() variant of the lock.

To make this clearer, I'll add the following to the commit message (if 
that helps):
```
Along with the bitmap, track which vCPU owns each guest interrupt file
id. Nothing consumes this yet, but it is filled in from the start as
the owner is what a guest external interrupt handler needs: a guest
interrupt file stays enabled in hgeie while its vCPU is descheduled, so
an interrupt targeting that file traps to Xen, which then has to find
the vCPU it belongs to in order to wake it up.

While the tracking is per-CPU data, it isn't accessed only locally: a
guest interrupt file belongs to the pCPU a vCPU is going to run on, so
it is allocated and released by whichever CPU is handling the vCPU at
the time: the release side is even passed the target CPU explicitly.
Hence a lock is needed. It has to be the IRQ-safe variant, as the
tracking is also going to be read from interrupt context on the CPU
owning it.
```
> 
>> +    }
>> +
>> +    spin_unlock_irqrestore(&vgein->lock, flags);
>> +
>> +#ifdef VGEIN_DEBUG
>> +    gprintk(XENLOG_DEBUG, "%s: %pv: vgein_id(%u), xen_cpu%u_bmp=%#lx\n",
>> +           __func__, v, vgein_id, v->processor, *bmp);
>> +#endif
>> +
>> +    return vgein_id;
>> +}
>> +
>> +void vgein_release(struct vcpu *v, unsigned int vgein_id)
>> +{
>> +    unsigned long flags;
>> +    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
>> +
>> +    if ( !vgein_id )
>> +        return;
>> +
>> +    spin_lock_irqsave(&vgein->lock, flags);
>> +    __clear_bit(vgein_id, &vgein->bmp);
>> +    vgein->owners[vgein_id] = NULL;
> 
> If already you track the vCPU, also assert that prior to clearing the array
> slot it has the expected value? For the bit being cleared, maybe also
> 
>      if ( !__test_and_clear_bit(vgein_id, &vgein->bmp) )
>          ASSERT_UNREACHABLE();
> 
> ? Yet as said - much remains unclear without knowing how all of this is
> meant to be used.

It makes sense. I will add that.

Thanks.

~ Oleksii

Re: [PATCH v1 02/17] xen/riscv: add basic VGEIN management for AIA guests

[Jan Beulich]

On 29.07.2026 16:55, Oleksii Kurochko wrote:
> On 7/27/26 5:41 PM, Jan Beulich wrote:
>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>> It was decided to add support for IMSIC from the start instead of having APLIC
>>> operate in direct delivery mode, as it requires a trap-and-emulation approach,
>>> which is not optimal from a performance standpoint.
>>>
>>> AIA provides a hardware-accelerated mechanism for delivering external
>>> interrupts to domains via "guest interrupt files" located in IMSIC.
>>> A single physical hart can implement multiple such files (up to GEILEN),
>>> allowing several virtual harts to receive interrupts directly from hardware.
>>>
>>> Introduce per-CPU tracking of guest interrupt file identifiers (VGEIN)
>>> for systems implementing AIA specification. Each CPU maintains
>>> a bitmap describing which guest interrupt files are currently in use.
>>>
>>> Add helpers to initialize the bitmap based on the number of available
>>> guest interrupt files (GEILEN), assign a VGEIN to a vCPU, and release it
>>> when no longer needed. When assigning a VGEIN, the corresponding value
>>> is written to the VGEIN field of the guest hstatus register so that
>>> VS-level external interrupts are delivered from the selected interrupt
>>> file.
>>
>> And when exactly is this "assignment" intended to occur? vgein_assign() and
>> vgein_release() have no callers here, so this remains entirely unclear.
> 
> [A] Agreed, I should have added that information to the commit message:
> 
> VGEIN is assigned (via vgein_assign()) before jumping to the new vCPU 
> execution context (in continue_new_vcpu()) and is re-assigned during 
> vCPU migration from one pCPU to another.
> 
> VGEIN is released (via vgein_release()) on the old pCPU during migration.

That is, state of that vCPU is held in hardware for perhaps an extended
period of time after the vCPU was last de-scheduled. That's a fair
optimization (we do something similar on x86, albeit that has been
increasingly under question lately). However, doesn't this then require
sync_local_execstate() to become non-empty?

Furthermore, rather than having vgein_assign() fail when
find_next_zero_bit() fails to find an available ID, shouldn't you release
some other vCPU's ID, making it available for re-use?

>>> +static int cf_check cpu_callback(struct notifier_block *nfb, unsigned long action,
>>> +                        void *hcpu)
>>> +{
>>> +    unsigned int cpu = (unsigned long)hcpu;
>>> +    int rc = 0;
>>> +
>>> +    switch ( action )
>>> +    {
>>> +    case CPU_STARTING:
>>> +        rc = vgein_init(cpu);
>>> +        if ( rc )
>>> +            printk("AIA: failed to init vgein for CPU%u\n", cpu);
>>> +        break;
>>> +    }
>>> +
>>> +    return notifier_from_errno(rc);
>>> +}
>>
>> Where's the freeing of the allocation vgein_init(), when CPU bringup fails
>> or a CPU was brought down?
> I'll add the following:
> 
> case CPU_UP_CANCELED:
> case CPU_DEAD:
>      vgein_free(cpu);
>      break;
> 
> and:
> 
> static void vgein_free(unsigned int cpu)
> {
>      struct vgein_ctrl *vgein = &per_cpu(vgein, cpu);
> 
>      ASSERT(!vgein->bmp);

Does this hold in all cases? What migrates vCPU-s off of a pCPU going down?
IOW aren't you introducing an ordering problem between your notifier handler
and the scheduler's?

>      vgein->geilen = 0;
>      XVFREE(vgein->owners);
> }
> 
> I'm also wondering whether vgein_init() should be moved to 
> CPU_UP_PREPARE. If vgein_init() fails in CPU_STARTING, the hypervisor 
> will stop instead of simply ignoring the CPU.
> 
> However, in CPU_UP_PREPARE we don't yet know the value of GEILEN, which 
> is needed to allocate vgein->owners. As I understand it, CPU_UP_PREPARE 
> is not executed on the CPU that is being brought up.

But there's an upper bound, isn't there? Use that for preliminary allocation,
and re-alloc (best effort) from CPU_ONLINE?

Yet then I continue to question the presence of this array in the first place.
Something similar isn't needed elsewhere (afaik), and its intended use (as
said) doesn't become obvious here.

>>> +unsigned int vgein_assign(struct vcpu *v)
>>> +{
>>> +    unsigned int vgein_id;
>>> +    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
>>> +    unsigned long *bmp = &vgein->bmp;
>>> +    unsigned long flags;
>>> +
>>> +    if ( !vgein->geilen )
>>> +        return 0;
>>> +
>>> +    spin_lock_irqsave(&vgein->lock, flags);
>>
>> Because it's unclear where this is to be called from, it's also unclear whether
>> a lock is needed here (and if so whether a plain spin lock is appropriate).
> 
> Based on what I wrote in [A] above a lock is defintely needed as it 
> could be that vgein_release() is called for old pCPU during migration 
> and at the same time old pCPU could call vgein_assign() so we want to 
> keep vgein bitmap consistent.

Can this really happen? It almost sounds as if you were suspecting
context-switch-in could race with context-switch-out. Yet again - none of
this can sensibly be discussed without seeing how / where the functions are
to be used.

> Regarding why _irqsave() it is mostly connected to ...

Why the mention of _irqsave? My use of "plain spinlock" was meant to contrast
to the possible use of an r/w lock.

Jan

Re: [PATCH v1 02/17] xen/riscv: add basic VGEIN management for AIA guests

[Oleksii Kurochko]

On 7/30/26 9:42 AM, Jan Beulich wrote:
> On 29.07.2026 16:55, Oleksii Kurochko wrote:
>> On 7/27/26 5:41 PM, Jan Beulich wrote:
>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>> It was decided to add support for IMSIC from the start instead of having APLIC
>>>> operate in direct delivery mode, as it requires a trap-and-emulation approach,
>>>> which is not optimal from a performance standpoint.
>>>>
>>>> AIA provides a hardware-accelerated mechanism for delivering external
>>>> interrupts to domains via "guest interrupt files" located in IMSIC.
>>>> A single physical hart can implement multiple such files (up to GEILEN),
>>>> allowing several virtual harts to receive interrupts directly from hardware.
>>>>
>>>> Introduce per-CPU tracking of guest interrupt file identifiers (VGEIN)
>>>> for systems implementing AIA specification. Each CPU maintains
>>>> a bitmap describing which guest interrupt files are currently in use.
>>>>
>>>> Add helpers to initialize the bitmap based on the number of available
>>>> guest interrupt files (GEILEN), assign a VGEIN to a vCPU, and release it
>>>> when no longer needed. When assigning a VGEIN, the corresponding value
>>>> is written to the VGEIN field of the guest hstatus register so that
>>>> VS-level external interrupts are delivered from the selected interrupt
>>>> file.
>>>
>>> And when exactly is this "assignment" intended to occur? vgein_assign() and
>>> vgein_release() have no callers here, so this remains entirely unclear.
>>
>> [A] Agreed, I should have added that information to the commit message:
>>
>> VGEIN is assigned (via vgein_assign()) before jumping to the new vCPU
>> execution context (in continue_new_vcpu()) and is re-assigned during
>> vCPU migration from one pCPU to another.
>>
>> VGEIN is released (via vgein_release()) on the old pCPU during migration.
> 
> That is, state of that vCPU is held in hardware for perhaps an extended
> period of time after the vCPU was last de-scheduled. That's a fair
> optimization (we do something similar on x86, albeit that has been
> increasingly under question lately). However, doesn't this then require
> sync_local_execstate() to become non-empty?

IIUC, sync_local_execstate() is needed for the lazy context switch case 
when switching from vCPUA to the idle vCPU. The idea is that if, after 
running the idle vCPU, the next scheduled vCPU is again vCPUA, then 
nothing needs to be done because no real context switch has occurred yet.

IMO, this is not the case for VGEIN. It is perfectly fine for a vCPU to 
keep its previously assigned VGEIN even if the next scheduled vCPU is 
different. In fact, it is necessary to preserve VGEIN because it will be 
needed later, for example, to wake up the vCPU if an interrupt for that 
vCPU occurs.

The idea is that if a vCPU uses a hardware interrupt file, then when the 
vCPU is descheduled, the corresponding CSR_HGEIE bit, where the bit 
number corresponds to the VGEIN value, is set. If an IRQ_S_GEXT trap 
then occurs, the hgei_interrupt() handler can determine which vCPU 
should be woken up:

void hgei_interrupt(void)
{
     unsigned long hgei_mask, flags;
     struct vgein_ctrl *vgein_ = &this_cpu(vgein);

     hgei_mask = csr_read(CSR_HGEIP) & csr_read(CSR_HGEIE);
     csr_clear(CSR_HGEIE, hgei_mask);

     spin_lock_irqsave(&vgein_->lock, flags);

     for_each_set_bit ( vs_guest_file_id, hgei_mask )
     {
        ...
        /* do some logic to call vcpu_kick */
        ...
     }

     spin_unlock_irqrestore(&vgein_->lock, flags);
}

> 
> Furthermore, rather than having vgein_assign() fail when
> find_next_zero_bit() fails to find an available ID, shouldn't you release
> some other vCPU's ID, making it available for re-use?

This is a good question, and it requires a separate investigation to 
determine whether such an approach would actually be beneficial. It 
would require not only changing the VGEIN field in vcpu->hstatus, but 
also synchronizing at least the pending interrupts from one IMSIC 
interrupt file to another, which would also consume time and further 
complicate the logic.

If find_next_zero_bit() fails, the vCPU will simply receive VGEIN=0, 
which means that the software interrupt file will be used. Therefore, 
everything should continue to work correctly, although it will be slower 
than using a hardware interrupt file.

Considering that the maximum value of GEILEN is 31 for RV32 and 63 for 
RV64 (although there is no guarantee that an implementation will support 
the maximum value), let's assume a GEILEN value of 31 for RV64 as well. 
In that case, a system with 4 CPUs would cover the maximum number of 
vCPUs supported by Xen (IIURC, it is 128). Therefore, a software 
interrupt file would not be needed at all, assuming the scheduler 
distributes vCPUs reasonably well.

Even if GEILEN is smaller, I expect that the scheduler will migrate 
vCPUs between pCPUs from time to time. This will free a hardware IMSIC 
interrupt file slot on the previous pCPU, allowing another vCPU to 
obtain a hardware IMSIC interrupt file slot.

For now, I would prefer to keep the current VGEIN allocation strategy as 
it is definitely easier for implementation at least and consider your 
suggestion of releasing another vCPU's VGEIN as a potential 
optimization. I think this optimization should first be evaluated 
through measurements and experiments.

> 
>>>> +static int cf_check cpu_callback(struct notifier_block *nfb, unsigned long action,
>>>> +                        void *hcpu)
>>>> +{
>>>> +    unsigned int cpu = (unsigned long)hcpu;
>>>> +    int rc = 0;
>>>> +
>>>> +    switch ( action )
>>>> +    {
>>>> +    case CPU_STARTING:
>>>> +        rc = vgein_init(cpu);
>>>> +        if ( rc )
>>>> +            printk("AIA: failed to init vgein for CPU%u\n", cpu);
>>>> +        break;
>>>> +    }
>>>> +
>>>> +    return notifier_from_errno(rc);
>>>> +}
>>>
>>> Where's the freeing of the allocation vgein_init(), when CPU bringup fails
>>> or a CPU was brought down?
>> I'll add the following:
>>
>> case CPU_UP_CANCELED:
>> case CPU_DEAD:
>>       vgein_free(cpu);
>>       break;
>>
>> and:
>>
>> static void vgein_free(unsigned int cpu)
>> {
>>       struct vgein_ctrl *vgein = &per_cpu(vgein, cpu);
>>
>>       ASSERT(!vgein->bmp);
> 
> Does this hold in all cases? What migrates vCPU-s off of a pCPU going down?
> IOW aren't you introducing an ordering problem between your notifier handler
> and the scheduler's?

The migration is done by the scheduler/cpupool notifiers at 
CPU_DOWN_PREPARE (cpupool_cpu_remove_prologue() -> 
cpu_disable_scheduler()) and CPU_DYING (cpupool_cpu_remove()), i.e. in 
actions strictly preceding CPU_DEAD; each unit migration goes through 
sched_move_irqs() -> arch_move_irqs() -> imsic_migrate_vcpu(), which 
releases the VGEIN on the old pCPU. So there's no ordering dependency on 
notifier priority within CPU_DEAD.

However you're right that the assertion doesn't hold in all cases. On 
the suspend path both notifiers bail out for system_state > 
SYS_STATE_active, so no vCPU is migrated off at all and the bits are 
still set at CPU_DEAD — and freeing owners[] there would be actively 
wrong, since the vCPUs still reference that pCPU's VS-file. 
Independently, arch_vcpu_destroy() never releases the VGEIN today, so a 
destroyed vCPU leaks its bit for good.

Given GEILEN <= XLEN-1, I'll drop the allocation altogether and use a 
fixed struct vcpu *owners[BITS_PER_LONG] in the per-CPU structure; that 
removes vgein_free() and the question with it. I'll fix the missing 
release in vcpu teardown separately.

> 
>>       vgein->geilen = 0;
>>       XVFREE(vgein->owners);
>> }
>>
>> I'm also wondering whether vgein_init() should be moved to
>> CPU_UP_PREPARE. If vgein_init() fails in CPU_STARTING, the hypervisor
>> will stop instead of simply ignoring the CPU.
>>
>> However, in CPU_UP_PREPARE we don't yet know the value of GEILEN, which
>> is needed to allocate vgein->owners. As I understand it, CPU_UP_PREPARE
>> is not executed on the CPU that is being brought up.
> 
> But there's an upper bound, isn't there? Use that for preliminary allocation,
> and re-alloc (best effort) from CPU_ONLINE?

Considering that upper bound isn't to big then we could just live with 
that without having re-alloc. Look at what I wrote above.

> 
> Yet then I continue to question the presence of this array in the first place.
> Something similar isn't needed elsewhere (afaik), and its intended use (as
> said) doesn't become obvious here.

I can drop it for now and reintroduce it later when it is actually 
needed. In short, it is intended to be used in hgei_interrupt(), as I 
described above, in the following way (inside the for-loop):

...
for_each_set_bit(vs_guest_file_id, hgei_mask)
{
unsigned int owners_index = vs_guest_file_id /* - 1 */;

```
if ( vgein_->owners[owners_index] )
{
     dprintk("kick ->%pv, hgei_mask(%#lx)\n",
             vgein_->owners[owners_index], hgei_mask);

     vcpu_kick(vgein_->owners[owners_index]);
}
```

}
...

Alternatively, I could introduce this in this series, since it will be 
necessary to set the HGEIE bit in imsic_state_save() anyway to allow the 
vCPU to be woken up. It also seems like the best option, as it addresses 
at least some of the comments you raised.

> 
>>>> +unsigned int vgein_assign(struct vcpu *v)
>>>> +{
>>>> +    unsigned int vgein_id;
>>>> +    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
>>>> +    unsigned long *bmp = &vgein->bmp;
>>>> +    unsigned long flags;
>>>> +
>>>> +    if ( !vgein->geilen )
>>>> +        return 0;
>>>> +
>>>> +    spin_lock_irqsave(&vgein->lock, flags);
>>>
>>> Because it's unclear where this is to be called from, it's also unclear whether
>>> a lock is needed here (and if so whether a plain spin lock is appropriate).
>>
>> Based on what I wrote in [A] above a lock is defintely needed as it
>> could be that vgein_release() is called for old pCPU during migration
>> and at the same time old pCPU could call vgein_assign() so we want to
>> keep vgein bitmap consistent.
> 
> Can this really happen? It almost sounds as if you were suspecting
> context-switch-in could race with context-switch-out. Yet again - none of
> this can sensibly be discussed without seeing how / where the functions are
> to be used.

Maybe I didn't explain it clearly, but during migration (which, 
according to my understanding of vcpu_move_irqs(), is executed on 
pCPU1), when vCPU0 is migrated from pCPU0 to pCPU1, its old VGEIN on 
pCPU0 needs to be released. I don't see any reason why, at the same 
time, pCPU0 could not try to assign that VGEIN to another vCPU. Without 
proper protection, this could lead to race conditions.

Also, setting a bit in the VGEIN bitmap and updating the owner array 
should be an atomic operation, at least to correctly handle the 
hgei_interrupt() case mentioned above and vCPU migration, which calls 
vgein_release(...,old_pcpu,...).

I agree that it would probably be easier if the migration patches were 
included in this patch series as well. I can either post those patches 
to this thread now or include them in the v2 series when it is ready. 
What do you think?

> 
>> Regarding why _irqsave() it is mostly connected to ...
> 
> Why the mention of _irqsave? My use of "plain spinlock" was meant to contrast
> to the possible use of an r/w lock.

Oh, okay... I thought your question was why the _irqsave() variant is 
used specifically.

I think it is hard to predict whether read operations will be much more 
frequent than write operations in this case. It depends on how often 
vCPU migration occurs and how often hgei_interrupt() is called. At the 
moment, I believe hgei_interrupt() is the only reader.

~ Oleksii

Re: [PATCH v1 02/17] xen/riscv: add basic VGEIN management for AIA guests

[Jan Beulich]

On 30.07.2026 17:46, Oleksii Kurochko wrote:
> On 7/30/26 9:42 AM, Jan Beulich wrote:
>> On 29.07.2026 16:55, Oleksii Kurochko wrote:
>>> On 7/27/26 5:41 PM, Jan Beulich wrote:
>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>> It was decided to add support for IMSIC from the start instead of having APLIC
>>>>> operate in direct delivery mode, as it requires a trap-and-emulation approach,
>>>>> which is not optimal from a performance standpoint.
>>>>>
>>>>> AIA provides a hardware-accelerated mechanism for delivering external
>>>>> interrupts to domains via "guest interrupt files" located in IMSIC.
>>>>> A single physical hart can implement multiple such files (up to GEILEN),
>>>>> allowing several virtual harts to receive interrupts directly from hardware.
>>>>>
>>>>> Introduce per-CPU tracking of guest interrupt file identifiers (VGEIN)
>>>>> for systems implementing AIA specification. Each CPU maintains
>>>>> a bitmap describing which guest interrupt files are currently in use.
>>>>>
>>>>> Add helpers to initialize the bitmap based on the number of available
>>>>> guest interrupt files (GEILEN), assign a VGEIN to a vCPU, and release it
>>>>> when no longer needed. When assigning a VGEIN, the corresponding value
>>>>> is written to the VGEIN field of the guest hstatus register so that
>>>>> VS-level external interrupts are delivered from the selected interrupt
>>>>> file.
>>>>
>>>> And when exactly is this "assignment" intended to occur? vgein_assign() and
>>>> vgein_release() have no callers here, so this remains entirely unclear.
>>>
>>> [A] Agreed, I should have added that information to the commit message:
>>>
>>> VGEIN is assigned (via vgein_assign()) before jumping to the new vCPU
>>> execution context (in continue_new_vcpu()) and is re-assigned during
>>> vCPU migration from one pCPU to another.
>>>
>>> VGEIN is released (via vgein_release()) on the old pCPU during migration.
>>
>> That is, state of that vCPU is held in hardware for perhaps an extended
>> period of time after the vCPU was last de-scheduled. That's a fair
>> optimization (we do something similar on x86, albeit that has been
>> increasingly under question lately). However, doesn't this then require
>> sync_local_execstate() to become non-empty?
> 
> IIUC, sync_local_execstate() is needed for the lazy context switch case 
> when switching from vCPUA to the idle vCPU.

Or when full state is to be obtained for a vCPU, for example.

> The idea is that if, after 
> running the idle vCPU, the next scheduled vCPU is again vCPUA, then 
> nothing needs to be done because no real context switch has occurred yet.
> 
> IMO, this is not the case for VGEIN. It is perfectly fine for a vCPU to 
> keep its previously assigned VGEIN even if the next scheduled vCPU is 
> different. In fact, it is necessary to preserve VGEIN because it will be 
> needed later, for example, to wake up the vCPU if an interrupt for that 
> vCPU occurs.
> 
> The idea is that if a vCPU uses a hardware interrupt file, then when the 
> vCPU is descheduled, the corresponding CSR_HGEIE bit, where the bit 
> number corresponds to the VGEIN value, is set. If an IRQ_S_GEXT trap 
> then occurs, the hgei_interrupt() handler can determine which vCPU 
> should be woken up:
> 
> void hgei_interrupt(void)
> {
>      unsigned long hgei_mask, flags;
>      struct vgein_ctrl *vgein_ = &this_cpu(vgein);
> 
>      hgei_mask = csr_read(CSR_HGEIP) & csr_read(CSR_HGEIE);
>      csr_clear(CSR_HGEIE, hgei_mask);
> 
>      spin_lock_irqsave(&vgein_->lock, flags);
> 
>      for_each_set_bit ( vs_guest_file_id, hgei_mask )
>      {
>         ...
>         /* do some logic to call vcpu_kick */
>         ...
>      }
> 
>      spin_unlock_irqrestore(&vgein_->lock, flags);
> }

Ah, that's pretty helpful extra information.

>> Furthermore, rather than having vgein_assign() fail when
>> find_next_zero_bit() fails to find an available ID, shouldn't you release
>> some other vCPU's ID, making it available for re-use?
> 
> This is a good question, and it requires a separate investigation to 
> determine whether such an approach would actually be beneficial. It 
> would require not only changing the VGEIN field in vcpu->hstatus, but 
> also synchronizing at least the pending interrupts from one IMSIC 
> interrupt file to another, which would also consume time and further 
> complicate the logic.
> 
> If find_next_zero_bit() fails, the vCPU will simply receive VGEIN=0, 
> which means that the software interrupt file will be used. Therefore, 
> everything should continue to work correctly, although it will be slower 
> than using a hardware interrupt file.

Plus there may end up being subtly different behavior. Imo you want to
let the hardware do what it can do for you.

> Considering that the maximum value of GEILEN is 31 for RV32 and 63 for 
> RV64 (although there is no guarantee that an implementation will support 
> the maximum value), let's assume a GEILEN value of 31 for RV64 as well. 
> In that case, a system with 4 CPUs would cover the maximum number of 
> vCPUs supported by Xen (IIURC, it is 128).

That's a single domain. There can be many domains, totaling to far more
than 128 vCPU-s.

> Therefore, a software 
> interrupt file would not be needed at all, assuming the scheduler 
> distributes vCPUs reasonably well.
> 
> Even if GEILEN is smaller, I expect that the scheduler will migrate 
> vCPUs between pCPUs from time to time. This will free a hardware IMSIC 
> interrupt file slot on the previous pCPU, allowing another vCPU to 
> obtain a hardware IMSIC interrupt file slot.
> 
> For now, I would prefer to keep the current VGEIN allocation strategy as 
> it is definitely easier for implementation at least and consider your 
> suggestion of releasing another vCPU's VGEIN as a potential 
> optimization. I think this optimization should first be evaluated 
> through measurements and experiments.

Well, I'm not going to insist, but I expect this will need re-doing rather
sooner than later then.

>> Yet then I continue to question the presence of this array in the first place.
>> Something similar isn't needed elsewhere (afaik), and its intended use (as
>> said) doesn't become obvious here.
> 
> I can drop it for now and reintroduce it later when it is actually 
> needed. In short, it is intended to be used in hgei_interrupt(), as I 
> described above, in the following way (inside the for-loop):
> 
> ...
> for_each_set_bit(vs_guest_file_id, hgei_mask)
> {
> unsigned int owners_index = vs_guest_file_id /* - 1 */;
> 
> ```
> if ( vgein_->owners[owners_index] )
> {
>      dprintk("kick ->%pv, hgei_mask(%#lx)\n",
>              vgein_->owners[owners_index], hgei_mask);
> 
>      vcpu_kick(vgein_->owners[owners_index]);
> }
> ```
> 
> }
> ...
> 
> Alternatively, I could introduce this in this series, since it will be 
> necessary to set the HGEIE bit in imsic_state_save() anyway to allow the 
> vCPU to be woken up. It also seems like the best option, as it addresses 
> at least some of the comments you raised.

Right, and then preferably in an order where one won't need to peek ahead
in the series to actually understand what's going on.

>>>>> +unsigned int vgein_assign(struct vcpu *v)
>>>>> +{
>>>>> +    unsigned int vgein_id;
>>>>> +    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
>>>>> +    unsigned long *bmp = &vgein->bmp;
>>>>> +    unsigned long flags;
>>>>> +
>>>>> +    if ( !vgein->geilen )
>>>>> +        return 0;
>>>>> +
>>>>> +    spin_lock_irqsave(&vgein->lock, flags);
>>>>
>>>> Because it's unclear where this is to be called from, it's also unclear whether
>>>> a lock is needed here (and if so whether a plain spin lock is appropriate).
>>>
>>> Based on what I wrote in [A] above a lock is defintely needed as it
>>> could be that vgein_release() is called for old pCPU during migration
>>> and at the same time old pCPU could call vgein_assign() so we want to
>>> keep vgein bitmap consistent.
>>
>> Can this really happen? It almost sounds as if you were suspecting
>> context-switch-in could race with context-switch-out. Yet again - none of
>> this can sensibly be discussed without seeing how / where the functions are
>> to be used.
> 
> Maybe I didn't explain it clearly, but during migration (which, 
> according to my understanding of vcpu_move_irqs(), is executed on 
> pCPU1), when vCPU0 is migrated from pCPU0 to pCPU1, its old VGEIN on 
> pCPU0 needs to be released. I don't see any reason why, at the same 
> time, pCPU0 could not try to assign that VGEIN to another vCPU. Without 
> proper protection, this could lead to race conditions.

Doesn't migration of vCPU-s between pCPU-s happen under suitable scheduler
locks?

> Also, setting a bit in the VGEIN bitmap and updating the owner array 
> should be an atomic operation, at least to correctly handle the 
> hgei_interrupt() case mentioned above and vCPU migration, which calls 
> vgein_release(...,old_pcpu,...).
> 
> I agree that it would probably be easier if the migration patches were 
> included in this patch series as well. I can either post those patches 
> to this thread now or include them in the v2 series when it is ready. 
> What do you think?

Including in v2 may be helpful, again to eliminate gaps in the understanding
a reader like me needs to have.

Jan

Re: [PATCH v1 02/17] xen/riscv: add basic VGEIN management for AIA guests

[Oleksii Kurochko]

On 7/30/26 6:03 PM, Jan Beulich wrote:
> On 30.07.2026 17:46, Oleksii Kurochko wrote:
>> On 7/30/26 9:42 AM, Jan Beulich wrote:
>>> On 29.07.2026 16:55, Oleksii Kurochko wrote:
>>>> On 7/27/26 5:41 PM, Jan Beulich wrote:
>>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>>> It was decided to add support for IMSIC from the start instead of having APLIC
>>>>>> operate in direct delivery mode, as it requires a trap-and-emulation approach,
>>>>>> which is not optimal from a performance standpoint.
>>>>>>
>>>>>> AIA provides a hardware-accelerated mechanism for delivering external
>>>>>> interrupts to domains via "guest interrupt files" located in IMSIC.
>>>>>> A single physical hart can implement multiple such files (up to GEILEN),
>>>>>> allowing several virtual harts to receive interrupts directly from hardware.
>>>>>>
>>>>>> Introduce per-CPU tracking of guest interrupt file identifiers (VGEIN)
>>>>>> for systems implementing AIA specification. Each CPU maintains
>>>>>> a bitmap describing which guest interrupt files are currently in use.
>>>>>>
>>>>>> Add helpers to initialize the bitmap based on the number of available
>>>>>> guest interrupt files (GEILEN), assign a VGEIN to a vCPU, and release it
>>>>>> when no longer needed. When assigning a VGEIN, the corresponding value
>>>>>> is written to the VGEIN field of the guest hstatus register so that
>>>>>> VS-level external interrupts are delivered from the selected interrupt
>>>>>> file.
>>>>>
>>>>> And when exactly is this "assignment" intended to occur? vgein_assign() and
>>>>> vgein_release() have no callers here, so this remains entirely unclear.
>>>>
>>>> [A] Agreed, I should have added that information to the commit message:
>>>>
>>>> VGEIN is assigned (via vgein_assign()) before jumping to the new vCPU
>>>> execution context (in continue_new_vcpu()) and is re-assigned during
>>>> vCPU migration from one pCPU to another.
>>>>
>>>> VGEIN is released (via vgein_release()) on the old pCPU during migration.
>>>
>>> That is, state of that vCPU is held in hardware for perhaps an extended
>>> period of time after the vCPU was last de-scheduled. That's a fair
>>> optimization (we do something similar on x86, albeit that has been
>>> increasingly under question lately). However, doesn't this then require
>>> sync_local_execstate() to become non-empty?
>>
>> IIUC, sync_local_execstate() is needed for the lazy context switch case
>> when switching from vCPUA to the idle vCPU.
> 
> Or when full state is to be obtained for a vCPU, for example.

I assume you're referring to XEN_DOMCTL_getvcpucontext, right?

In general, it seems that sync_local_execstate() is primarily an 
optimization. If lazy switching isn't supported, then every time a vCPU 
is de-scheduled, its state must be fully saved to memory. My 
understanding is that everything will still work correctly, just less 
efficiently.

I'm curious how much this optimization actually helps. How often does it 
happen that a vCPU is de-scheduled from a pCPU and then immediately 
scheduled back onto the same pCPU without any other vCPU being scheduled 
in between?

I will add to my TODO list that it is nice to use sync_local_execstate() 
in future.

>>>>>> +unsigned int vgein_assign(struct vcpu *v)
>>>>>> +{
>>>>>> +    unsigned int vgein_id;
>>>>>> +    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
>>>>>> +    unsigned long *bmp = &vgein->bmp;
>>>>>> +    unsigned long flags;
>>>>>> +
>>>>>> +    if ( !vgein->geilen )
>>>>>> +        return 0;
>>>>>> +
>>>>>> +    spin_lock_irqsave(&vgein->lock, flags);
>>>>>
>>>>> Because it's unclear where this is to be called from, it's also unclear whether
>>>>> a lock is needed here (and if so whether a plain spin lock is appropriate).
>>>>
>>>> Based on what I wrote in [A] above a lock is defintely needed as it
>>>> could be that vgein_release() is called for old pCPU during migration
>>>> and at the same time old pCPU could call vgein_assign() so we want to
>>>> keep vgein bitmap consistent.
>>>
>>> Can this really happen? It almost sounds as if you were suspecting
>>> context-switch-in could race with context-switch-out. Yet again - none of
>>> this can sensibly be discussed without seeing how / where the functions are
>>> to be used.
>>
>> Maybe I didn't explain it clearly, but during migration (which,
>> according to my understanding of vcpu_move_irqs(), is executed on
>> pCPU1), when vCPU0 is migrated from pCPU0 to pCPU1, its old VGEIN on
>> pCPU0 needs to be released. I don't see any reason why, at the same
>> time, pCPU0 could not try to assign that VGEIN to another vCPU. Without
>> proper protection, this could lead to race conditions.
> 
> Doesn't migration of vCPU-s between pCPU-s happen under suitable scheduler
> locks?
> 

If I am not mistaken every path that reaches arch_move_irqs() drops the 
scheduler lock first. The only thing still held at that point is 
sched_res_rculock , and that is an RCU read-side critical section, not 
mutual exclusion: it merely keeps struct sched_resource alive across 
get_sched_res() dereferences, since cpupool/hotplug frees those via 
call_rcu(&sr->rcu, sched_res_free). Any number of pCPUs can be inside it 
concurrently, and it does not disable interrupts, so it serialises 
neither the source pCPU against the destination one nor hgei_interrupt() 
mentioned above against either.

~ Oleksii

Re: [PATCH v1 02/17] xen/riscv: add basic VGEIN management for AIA guests

[Jan Beulich]

On 31.07.2026 16:59, Oleksii Kurochko wrote:
> 
> 
> On 7/30/26 6:03 PM, Jan Beulich wrote:
>> On 30.07.2026 17:46, Oleksii Kurochko wrote:
>>> On 7/30/26 9:42 AM, Jan Beulich wrote:
>>>> On 29.07.2026 16:55, Oleksii Kurochko wrote:
>>>>> On 7/27/26 5:41 PM, Jan Beulich wrote:
>>>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>>>> It was decided to add support for IMSIC from the start instead of having APLIC
>>>>>>> operate in direct delivery mode, as it requires a trap-and-emulation approach,
>>>>>>> which is not optimal from a performance standpoint.
>>>>>>>
>>>>>>> AIA provides a hardware-accelerated mechanism for delivering external
>>>>>>> interrupts to domains via "guest interrupt files" located in IMSIC.
>>>>>>> A single physical hart can implement multiple such files (up to GEILEN),
>>>>>>> allowing several virtual harts to receive interrupts directly from hardware.
>>>>>>>
>>>>>>> Introduce per-CPU tracking of guest interrupt file identifiers (VGEIN)
>>>>>>> for systems implementing AIA specification. Each CPU maintains
>>>>>>> a bitmap describing which guest interrupt files are currently in use.
>>>>>>>
>>>>>>> Add helpers to initialize the bitmap based on the number of available
>>>>>>> guest interrupt files (GEILEN), assign a VGEIN to a vCPU, and release it
>>>>>>> when no longer needed. When assigning a VGEIN, the corresponding value
>>>>>>> is written to the VGEIN field of the guest hstatus register so that
>>>>>>> VS-level external interrupts are delivered from the selected interrupt
>>>>>>> file.
>>>>>>
>>>>>> And when exactly is this "assignment" intended to occur? vgein_assign() and
>>>>>> vgein_release() have no callers here, so this remains entirely unclear.
>>>>>
>>>>> [A] Agreed, I should have added that information to the commit message:
>>>>>
>>>>> VGEIN is assigned (via vgein_assign()) before jumping to the new vCPU
>>>>> execution context (in continue_new_vcpu()) and is re-assigned during
>>>>> vCPU migration from one pCPU to another.
>>>>>
>>>>> VGEIN is released (via vgein_release()) on the old pCPU during migration.
>>>>
>>>> That is, state of that vCPU is held in hardware for perhaps an extended
>>>> period of time after the vCPU was last de-scheduled. That's a fair
>>>> optimization (we do something similar on x86, albeit that has been
>>>> increasingly under question lately). However, doesn't this then require
>>>> sync_local_execstate() to become non-empty?
>>>
>>> IIUC, sync_local_execstate() is needed for the lazy context switch case
>>> when switching from vCPUA to the idle vCPU.
>>
>> Or when full state is to be obtained for a vCPU, for example.
> 
> I assume you're referring to XEN_DOMCTL_getvcpucontext, right?

Yes.

> In general, it seems that sync_local_execstate() is primarily an 
> optimization. If lazy switching isn't supported, then every time a vCPU 
> is de-scheduled, its state must be fully saved to memory. My 
> understanding is that everything will still work correctly, just less 
> efficiently.

The lazy switching is an optimization, yes. If any state is kept in
hardware, sync_local_execstate() has to be used when full state of a
vCPU is to be obtained. Supplying back stale state of "guest interrupt
files" can't be correct. (Of course you can also arrange to obtain
up-to-date state by custom means, but imo that's likely less desirable.)

> I'm curious how much this optimization actually helps. How often does it 
> happen that a vCPU is de-scheduled from a pCPU and then immediately 
> scheduled back onto the same pCPU without any other vCPU being scheduled 
> in between?

That heavily depends on overall load of the system. When pCPU-s aren't
over-subscribed, a HVM vCPU getting de-scheduled to wait for qemu to
handle a certain operation may very well be able to resume on the same
pCPU after completion of the ioreq. The less overhead there, the better.
(Just to give an example.)

Jan

Re: [PATCH v1 02/17] xen/riscv: add basic VGEIN management for AIA guests

[Baptiste Le Duc]

[-- Attachment #1: Type: text/plain, Size: 8153 bytes --]

> It was decided to add support for IMSIC from the start instead of having APLIC
> operate in direct delivery mode, as it requires a trap-and-emulation approach,
> which is not optimal from a performance standpoint.
> 
> AIA provides a hardware-accelerated mechanism for delivering external
> interrupts to domains via "guest interrupt files" located in IMSIC.
> A single physical hart can implement multiple such files (up to GEILEN),
> allowing several virtual harts to receive interrupts directly from hardware.
> 
> Introduce per-CPU tracking of guest interrupt file identifiers (VGEIN)
> for systems implementing AIA specification. Each CPU maintains
> a bitmap describing which guest interrupt files are currently in use.
> 
> Add helpers to initialize the bitmap based on the number of available
> guest interrupt files (GEILEN), assign a VGEIN to a vCPU, and release it
> when no longer needed. When assigning a VGEIN, the corresponding value
> is written to the VGEIN field of the guest hstatus register so that
> VS-level external interrupts are delivered from the selected interrupt
> file.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>
> diff --git a/xen/arch/riscv/aia.c b/xen/arch/riscv/aia.c
> index e31c9c2d24..4f7f46f58f 100644
> --- a/xen/arch/riscv/aia.c
> +++ b/xen/arch/riscv/aia.c
> @@ -1,11 +1,33 @@
>  /* SPDX-License-Identifier: GPL-2.0-only */
>  
> +#include <xen/bitmap.h>
> +#include <xen/cpu.h>
>  #include <xen/errno.h>
>  #include <xen/init.h>

Add a #include <xen/percpu.h> here instead of in aia.h.

>  #include <xen/sections.h>
> +#include <xen/sched.h>
> +#include <xen/spinlock.h>
>  #include <xen/types.h>
> +#include <xen/xvmalloc.h>
>  
> +#include <asm/aia.h>
>  #include <asm/cpufeature.h>
> +#include <asm/csr.h>
> +#include <asm/current.h>
> +
> +struct vgein_ctrl {
> +    unsigned long bmp;
> +    spinlock_t lock;
> +    struct vcpu **owners;
> +    /* The least-significant bits are implemented first, apart from bit 0 */
> +    unsigned int geilen;
> +};
> +
> +/*
> + * VGEIN control structure for each physical CPU to track which VS (guest)
> + * interrupt file IDs are in use.
> + */
> +static DEFINE_PER_CPU(struct vgein_ctrl, vgein);
>  
>  static bool __ro_after_init _aia_usable;
>  
> @@ -14,10 +36,133 @@ bool aia_usable(void)
>      return _aia_usable;
>  }
>  
> +static int vgein_init(unsigned int cpu)

Could we call this function with a different cpu arg than the current
one running? If yes, we would read hgeie of not the cpu we wanted.
> +{
> +    struct vgein_ctrl *vgein = &per_cpu(vgein, cpu);
> +
> +    csr_write(CSR_HGEIE, -1UL);
> +    vgein->geilen = flsl(csr_read(CSR_HGEIE) >> 1);
> +    csr_write(CSR_HGEIE, 0);
> +
> +    printk("cpu%u.geilen=%u\n", cpu, vgein->geilen);

> +
> +    if ( !vgein->geilen )
> +        return -EOPNOTSUPP;
> +
> +    vgein->owners = xvzalloc_array(struct vcpu *, vgein->geilen);
> +    if ( !vgein->owners )
> +        return -ENOMEM;
> +
> +    spin_lock_init(&vgein->lock);
> +
> +    return 0;
> +}
> +
> +static int cf_check cpu_callback(struct notifier_block *nfb, unsigned long action,
> +                        void *hcpu)
> +{
> +    unsigned int cpu = (unsigned long)hcpu;
> +    int rc = 0;
> +
> +    switch ( action )
> +    {
> +    case CPU_STARTING:
> +        rc = vgein_init(cpu);
> +        if ( rc )
> +            printk("AIA: failed to init vgein for CPU%u\n", cpu);
> +        break;
> +    }
> +
> +    return notifier_from_errno(rc);
> +}
> +
> +static struct notifier_block cpu_nfb = {
> +    .notifier_call = cpu_callback,
> +};
> +
>  void __init aia_init(void)
>  {
> +    int rc;
> +
>      if ( !riscv_isa_extension_available(NULL, RISCV_ISA_EXT_ssaia) )
> +    {
> +        dprintk(XENLOG_WARNING, "SSAIA isn't present in riscv,isa\n");
>          return;
> +    }
> +
> +    if ( (rc = vgein_init(0)) )

Why `0` rather than smp_processor_id()? As described above vgein_init() reads CSR_HGEIE
of the current hart but stores the result into per_cpu(vgein, cpu), so the two
must agree.

> +    {
> +        dprintk(XENLOG_ERR, "vgein_init() failed: %d\n", rc);
> +        return;
> +    }
>  
>      _aia_usable = true;
> +
> +    register_cpu_notifier(&cpu_nfb);
> +}
> +
> +unsigned int vgein_assign(struct vcpu *v)
> +{
> +    unsigned int vgein_id;
> +    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);

What happens if v->processor change between vgein_assign() and
vgein_release? Because it seems in such case the release will hit a
different pCPU's bitmap: the original bit will leak and an unrelated
CPU's bit will be cleared under another vCPU's feet.

> +    unsigned long *bmp = &vgein->bmp;
> +    unsigned long flags;
> +
> +    if ( !vgein->geilen )
> +        return 0;
> +
> +    spin_lock_irqsave(&vgein->lock, flags);
> +    /*
> +     * The vgein_id shouldn't be zero, as it will indicate that no guest
> +     * external interrupt source is selected for VS-level external interrupts
> +     * according to RISC-V privileged spec:
> +     *   Hypervisor Status Register (hstatus) in RISC-V privileged spec:
> +     *
> +     *   The VGEIN (Virtual Guest External Interrupt Number) field selects
> +     *   a guest external interrupt source for VS-level external interrupts.
> +     *   VGEIN is a WLRL field that must be able to hold values between zero
> +     *   and the maximum guest external interrupt number (known as GEILEN),
> +     *   inclusive.
> +     *   When VGEIN=0, no guest external interrupt source is selected for
> +     *   VS-level external interrupts.
> +     *
> +     * So start to search from bit number 1.
> +     */
> +    vgein_id = find_next_zero_bit(bmp, vgein->geilen + 1, 1);
> +
> +    if ( vgein_id > vgein->geilen )
> +        vgein_id = 0;
> +    else
> +    {

Potential index error, because above you did:

    vgein->owners = xvzalloc_array(struct vcpu*, vgein->geilen)

so valid index are 0...(vgein->geilen-1). Adopt either
one of those two options:
    1. vgein->owners[vgein_id-1] = v
    2. vgein->owners = xvzalloc_array(struct vcpu *, vgein->geilen+1) in
       vgein_init()

I think `2` could be better to have vgein->owners replicated hgeie CSR but
it would left the first entry read-only.

> +        __set_bit(vgein_id, bmp);
> +        vgein->owners[vgein_id] = v;
> +    }
> +
> +    spin_unlock_irqrestore(&vgein->lock, flags);
> +
> +#ifdef VGEIN_DEBUG

VGEIN_DEBUG is not defined anywhere in the patch, please use
gdprintk(XENLOG_DEBUG, ...) directly, or drop this branch.

> +    gprintk(XENLOG_DEBUG, "%s: %pv: vgein_id(%u), xen_cpu%u_bmp=%#lx\n",
> +           __func__, v, vgein_id, v->processor, *bmp);
> +#endif
> +
> +    return vgein_id;
> +}
> +
> +void vgein_release(struct vcpu *v, unsigned int vgein_id)
> +{
> +    unsigned long flags;
> +    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
> +
> +    if ( !vgein_id )
> +        return;
> +
> +    spin_lock_irqsave(&vgein->lock, flags);
> +    __clear_bit(vgein_id, &vgein->bmp);
> +    vgein->owners[vgein_id] = NULL;
> +    spin_unlock_irqrestore(&vgein->lock, flags);
> +
> +#ifdef VGEIN_DEBUG
> +    gprintk(XENLOG_DEBUG, "%s: vgein_id(%u), xen_cpu%u_bmp=%#lx\n",
> +           __func__, vgein_id, v->processor, vgein->bmp);
> +#endif
>  }
> diff --git a/xen/arch/riscv/include/asm/aia.h b/xen/arch/riscv/include/asm/aia.h
> index aaa4bf91fc..c67be0069a 100644
> --- a/xen/arch/riscv/include/asm/aia.h
> +++ b/xen/arch/riscv/include/asm/aia.h
> @@ -3,8 +3,16 @@
>  #ifndef RISCV_AIA_H
>  #define RISCV_AIA_H
>  
> +#include <xen/percpu.h>

asm/aia.h needs neither <xen/percpu.h> nor <xen/spinlock.h> as struct
vgein_ctrl and the per-CPU variable both live in aia.c. Please drop them
and add <xen/percpu.h> in aia.c

-- 
Baptiste Le Duc <baptiste.le-duc@vates.tech>


--
Baptiste Le Duc | Vates XCP-ng Intern

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech

Re: [PATCH v1 02/17] xen/riscv: add basic VGEIN management for AIA guests

[Oleksii Kurochko]

On 8/10/26 3:32 PM, Baptiste Le Duc wrote:
>> It was decided to add support for IMSIC from the start instead of having APLIC
>> operate in direct delivery mode, as it requires a trap-and-emulation approach,
>> which is not optimal from a performance standpoint.
>>
>> AIA provides a hardware-accelerated mechanism for delivering external
>> interrupts to domains via "guest interrupt files" located in IMSIC.
>> A single physical hart can implement multiple such files (up to GEILEN),
>> allowing several virtual harts to receive interrupts directly from hardware.
>>
>> Introduce per-CPU tracking of guest interrupt file identifiers (VGEIN)
>> for systems implementing AIA specification. Each CPU maintains
>> a bitmap describing which guest interrupt files are currently in use.
>>
>> Add helpers to initialize the bitmap based on the number of available
>> guest interrupt files (GEILEN), assign a VGEIN to a vCPU, and release it
>> when no longer needed. When assigning a VGEIN, the corresponding value
>> is written to the VGEIN field of the guest hstatus register so that
>> VS-level external interrupts are delivered from the selected interrupt
>> file.
>>
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>
>> diff --git a/xen/arch/riscv/aia.c b/xen/arch/riscv/aia.c
>> index e31c9c2d24..4f7f46f58f 100644
>> --- a/xen/arch/riscv/aia.c
>> +++ b/xen/arch/riscv/aia.c
>> @@ -1,11 +1,33 @@
>>   /* SPDX-License-Identifier: GPL-2.0-only */
>>   
>> +#include <xen/bitmap.h>
>> +#include <xen/cpu.h>
>>   #include <xen/errno.h>
>>   #include <xen/init.h>
> 
> Add a #include <xen/percpu.h> here instead of in aia.h.

Sorry, but I’m a little confused here. <asm/aia.h> doesn’t include 
<xen/percpu.h>.

> 
>>   #include <xen/sections.h>
>> +#include <xen/sched.h>
>> +#include <xen/spinlock.h>
>>   #include <xen/types.h>
>> +#include <xen/xvmalloc.h>
>>   
>> +#include <asm/aia.h>
>>   #include <asm/cpufeature.h>
>> +#include <asm/csr.h>
>> +#include <asm/current.h>
>> +
>> +struct vgein_ctrl {
>> +    unsigned long bmp;
>> +    spinlock_t lock;
>> +    struct vcpu **owners;
>> +    /* The least-significant bits are implemented first, apart from bit 0 */
>> +    unsigned int geilen;
>> +};
>> +
>> +/*
>> + * VGEIN control structure for each physical CPU to track which VS (guest)
>> + * interrupt file IDs are in use.
>> + */
>> +static DEFINE_PER_CPU(struct vgein_ctrl, vgein);
>>   
>>   static bool __ro_after_init _aia_usable;
>>   
>> @@ -14,10 +36,133 @@ bool aia_usable(void)
>>       return _aia_usable;
>>   }
>>   
>> +static int vgein_init(unsigned int cpu)
> 
> Could we call this function with a different cpu arg than the current
> one running? If yes, we would read hgeie of not the cpu we wanted.

Considering that it touches the CSR_HGIEI register, it can only be 
called on the currently running CPU.

That’s why I suggested in one of my replies to Jan B. that I would drop 
the argument altogether for this function.

>> +{
>> +    struct vgein_ctrl *vgein = &per_cpu(vgein, cpu);
>> +
>> +    csr_write(CSR_HGEIE, -1UL);
>> +    vgein->geilen = flsl(csr_read(CSR_HGEIE) >> 1);
>> +    csr_write(CSR_HGEIE, 0);
>> +
>> +    printk("cpu%u.geilen=%u\n", cpu, vgein->geilen);
> 
>> +
>> +    if ( !vgein->geilen )
>> +        return -EOPNOTSUPP;
>> +
>> +    vgein->owners = xvzalloc_array(struct vcpu *, vgein->geilen);
>> +    if ( !vgein->owners )
>> +        return -ENOMEM;
>> +
>> +    spin_lock_init(&vgein->lock);
>> +
>> +    return 0;
>> +}
>> +
>> +static int cf_check cpu_callback(struct notifier_block *nfb, unsigned long action,
>> +                        void *hcpu)
>> +{
>> +    unsigned int cpu = (unsigned long)hcpu;
>> +    int rc = 0;
>> +
>> +    switch ( action )
>> +    {
>> +    case CPU_STARTING:
>> +        rc = vgein_init(cpu);
>> +        if ( rc )
>> +            printk("AIA: failed to init vgein for CPU%u\n", cpu);
>> +        break;
>> +    }
>> +
>> +    return notifier_from_errno(rc);
>> +}
>> +
>> +static struct notifier_block cpu_nfb = {
>> +    .notifier_call = cpu_callback,
>> +};
>> +
>>   void __init aia_init(void)
>>   {
>> +    int rc;
>> +
>>       if ( !riscv_isa_extension_available(NULL, RISCV_ISA_EXT_ssaia) )
>> +    {
>> +        dprintk(XENLOG_WARNING, "SSAIA isn't present in riscv,isa\n");
>>           return;
>> +    }
>> +
>> +    if ( (rc = vgein_init(0)) )
> 
> Why `0` rather than smp_processor_id()? As described above vgein_init() reads CSR_HGEIE
> of the current hart but stores the result into per_cpu(vgein, cpu), so the two
> must agree.

aia_init() is executed on boot cpu only so it uses 0 as Xen boot cpu is 
always 0. But it won't be an issue anymore as I mentioned above an 
argument of vgein_init() will be dropped anyway so it will be guaranteed 
that a correct CPU is used.

> 
>> +    {
>> +        dprintk(XENLOG_ERR, "vgein_init() failed: %d\n", rc);
>> +        return;
>> +    }
>>   
>>       _aia_usable = true;
>> +
>> +    register_cpu_notifier(&cpu_nfb);
>> +}
>> +
>> +unsigned int vgein_assign(struct vcpu *v)
>> +{
>> +    unsigned int vgein_id;
>> +    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
> 
> What happens if v->processor change between vgein_assign() and
> vgein_release? Because it seems in such case the release will hit a
> different pCPU's bitmap: the original bit will leak and an unrelated
> CPU's bit will be cleared under another vCPU's feet.

So, if v->processor changes between the calls to vgein_assign() and 
vgein_release(), it means that migration has happened. If migration has 
happened, then it is the responsibility of the migration code to 
properly assign the new vgein and release the previous one.

All other cases where vgein_release() is called are when the vCPU is 
dying, so everything is okay there as migration cannot happen.


> 
>> +    unsigned long *bmp = &vgein->bmp;
>> +    unsigned long flags;
>> +
>> +    if ( !vgein->geilen )
>> +        return 0;
>> +
>> +    spin_lock_irqsave(&vgein->lock, flags);
>> +    /*
>> +     * The vgein_id shouldn't be zero, as it will indicate that no guest
>> +     * external interrupt source is selected for VS-level external interrupts
>> +     * according to RISC-V privileged spec:
>> +     *   Hypervisor Status Register (hstatus) in RISC-V privileged spec:
>> +     *
>> +     *   The VGEIN (Virtual Guest External Interrupt Number) field selects
>> +     *   a guest external interrupt source for VS-level external interrupts.
>> +     *   VGEIN is a WLRL field that must be able to hold values between zero
>> +     *   and the maximum guest external interrupt number (known as GEILEN),
>> +     *   inclusive.
>> +     *   When VGEIN=0, no guest external interrupt source is selected for
>> +     *   VS-level external interrupts.
>> +     *
>> +     * So start to search from bit number 1.
>> +     */
>> +    vgein_id = find_next_zero_bit(bmp, vgein->geilen + 1, 1);
>> +
>> +    if ( vgein_id > vgein->geilen )
>> +        vgein_id = 0;
>> +    else
>> +    {
> 
> Potential index error, because above you did:
> 
>      vgein->owners = xvzalloc_array(struct vcpu*, vgein->geilen)
> 
> so valid index are 0...(vgein->geilen-1). Adopt either
> one of those two options:
>      1. vgein->owners[vgein_id-1] = v
>      2. vgein->owners = xvzalloc_array(struct vcpu *, vgein->geilen+1) in
>         vgein_init()
> 
> I think `2` could be better to have vgein->owners replicated hgeie CSR but
> it would left the first entry read-only.

I've found that too during prepare a reply to Jan B. so fixed it already 
in v2. I've decided to go with what you suggested in 2.

> 
>> +        __set_bit(vgein_id, bmp);
>> +        vgein->owners[vgein_id] = v;
>> +    }
>> +
>> +    spin_unlock_irqrestore(&vgein->lock, flags);
>> +
>> +#ifdef VGEIN_DEBUG
> 
> VGEIN_DEBUG is not defined anywhere in the patch, please use
> gdprintk(XENLOG_DEBUG, ...) directly, or drop this branch.

It is intentionally not defined. If a user needs additional VGEIN debug 
information, they should define it themselves, as it can produce a 
pretty large amount of logs due to, for example, the migration process, 
where vgein_assign() and vgein_release() are used quite actively.

> 
>> +    gprintk(XENLOG_DEBUG, "%s: %pv: vgein_id(%u), xen_cpu%u_bmp=%#lx\n",
>> +           __func__, v, vgein_id, v->processor, *bmp);
>> +#endif
>> +
>> +    return vgein_id;
>> +}
>> +
>> +void vgein_release(struct vcpu *v, unsigned int vgein_id)
>> +{
>> +    unsigned long flags;
>> +    struct vgein_ctrl *vgein = &per_cpu(vgein, v->processor);
>> +
>> +    if ( !vgein_id )
>> +        return;
>> +
>> +    spin_lock_irqsave(&vgein->lock, flags);
>> +    __clear_bit(vgein_id, &vgein->bmp);
>> +    vgein->owners[vgein_id] = NULL;
>> +    spin_unlock_irqrestore(&vgein->lock, flags);
>> +
>> +#ifdef VGEIN_DEBUG
>> +    gprintk(XENLOG_DEBUG, "%s: vgein_id(%u), xen_cpu%u_bmp=%#lx\n",
>> +           __func__, vgein_id, v->processor, vgein->bmp);
>> +#endif
>>   }
>> diff --git a/xen/arch/riscv/include/asm/aia.h b/xen/arch/riscv/include/asm/aia.h
>> index aaa4bf91fc..c67be0069a 100644
>> --- a/xen/arch/riscv/include/asm/aia.h
>> +++ b/xen/arch/riscv/include/asm/aia.h
>> @@ -3,8 +3,16 @@
>>   #ifndef RISCV_AIA_H
>>   #define RISCV_AIA_H
>>   
>> +#include <xen/percpu.h>
> 
> asm/aia.h needs neither <xen/percpu.h> nor <xen/spinlock.h> as struct
> vgein_ctrl and the per-CPU variable both live in aia.c. Please drop them
> and add <xen/percpu.h> in aia.c
> 

Yes, it is redundant code that I missed removing. I’ve already noticed 
it and removed it in v2.

Thanks.

~ Oleksii

Re: [PATCH v1 02/17] xen/riscv: add basic VGEIN management for AIA guests

[Baptiste Le Duc]

On 2026-08-10 17:04:43+02:00, Oleksii Kurochko wrote:
> On 8/10/26 3:32 PM, Baptiste Le Duc wrote:
> 
> >> It was decided to add support for IMSIC from the start instead of having APLIC
> > 
> > Add a #include <xen/percpu.h> here instead of in aia.h.
> 
> Sorry, but I’m a little confused here. <asm/aia.h> doesn’t include 
> <xen/percpu.h>.
> 
Yes it is, in xen.git/xen/arch/riscv/asm/aia.h, you added, in this patch
    #include <xen/percpu.h>
Therefore, I think it could be included directly in the aia.c file as it
is the only place where it is used.
> > Could we call this function with a different cpu arg than the current
> > one running? If yes, we would read hgeie of not the cpu we wanted.
> 
> Considering that it touches the CSR_HGIEI register, it can only be 
> called on the currently running CPU.
> 
> That’s why I suggested in one of my replies to Jan B. that I would drop 
> the argument altogether for this function.
> 
> >> +{
> > 
> > 
> > Why `0` rather than smp_processor_id()? As described above vgein_init() reads CSR_HGEIE
> > of the current hart but stores the result into per_cpu(vgein, cpu), so the two
> > must agree.
> 
> aia_init() is executed on boot cpu only so it uses 0 as Xen boot cpu is 
> always 0. But it won't be an issue anymore as I mentioned above an 
> argument of vgein_init() will be dropped anyway so it will be guaranteed 
> that a correct CPU is used.
> 
> > What happens if v->processor change between vgein_assign() and
> > vgein_release? Because it seems in such case the release will hit a
> > different pCPU's bitmap: the original bit will leak and an unrelated
> > CPU's bit will be cleared under another vCPU's feet.
> 
> So, if v->processor changes between the calls to vgein_assign() and 
> vgein_release(), it means that migration has happened. If migration has 
> happened, then it is the responsibility of the migration code to 
> properly assign the new vgein and release the previous one.
> 
> All other cases where vgein_release() is called are when the vCPU is 
> dying, so everything is okay there as migration cannot happen.
> 
> > Potential index error, because above you did:
> > 
> >      vgein->owners = xvzalloc_array(struct vcpu*, vgein->geilen)
> > 
> > so valid index are 0...(vgein->geilen-1). Adopt either
> > one of those two options:
> >      1. vgein->owners[vgein_id-1] = v
> >      2. vgein->owners = xvzalloc_array(struct vcpu *, vgein->geilen+1) in
> >         vgein_init()
> > 
> > I think `2` could be better to have vgein->owners replicated hgeie CSR but
> > it would left the first entry read-only.
> 
> I've found that too during prepare a reply to Jan B. so fixed it already 
> in v2. I've decided to go with what you suggested in 2.
> 
> > VGEIN_DEBUG is not defined anywhere in the patch, please use
> > gdprintk(XENLOG_DEBUG, ...) directly, or drop this branch.
> 
> It is intentionally not defined. If a user needs additional VGEIN debug 
> information, they should define it themselves, as it can produce a 
> pretty large amount of logs due to, for example, the migration process, 
> where vgein_assign() and vgein_release() are used quite actively.
> 
Oh I didn't know it was a common practice, thanks for this explanation.
> > asm/aia.h needs neither <xen/percpu.h> nor <xen/spinlock.h> as struct
> > vgein_ctrl and the per-CPU variable both live in aia.c. Please drop them
> > and add <xen/percpu.h> in aia.c
> 
> Yes, it is redundant code that I missed removing. I’ve already noticed 
> it and removed it in v2.
It's what I wanted to mean in the comment above about <xen/percpu.h>
> Thanks.
Happy to help :)
> 
> ~ Oleksii

[PATCH v1 03/17] xen/riscv: add missing APLIC register offsets, masks to asm/aplic.h

[Oleksii Kurochko]

These definitions are required for correct decoding of APLIC MMIO
accesses and target configuration, and will be used by both the
physical and virtual APLIC implementations.

No functional change is intended by this patch; it only centralises
hardware definitions that were previously missing.

Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes in v3:
 - Drop APLIC_TARGET_IPRIO_MASK and APLIC_TARGET_GUEST_IDX_SHIFT (unused).
 - Add comments to each register field group (domaincfg, sourcecfg, target).
 - Group APLIC_TARGET_HART_IDX_SHIFT and APLIC_TARGET_EIID_MASK together
   under /* target register fields */ at the top of the field definitions.
---
Changes in v2:
 - new patch
---
---
 xen/arch/riscv/include/asm/aplic.h | 35 ++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/xen/arch/riscv/include/asm/aplic.h b/xen/arch/riscv/include/asm/aplic.h
index 07318aaac25d..f22622b9a23f 100644
--- a/xen/arch/riscv/include/asm/aplic.h
+++ b/xen/arch/riscv/include/asm/aplic.h
@@ -15,6 +15,8 @@
 
 #include <asm/imsic.h>
 
+#define APLIC_REG_OFFSET_MASK   0x3fff
+
 /*
  * domaincfg read-only fields (AIA spec):
  *  - bits [31:24] -> read-only 0x80
@@ -25,6 +27,7 @@
 #define APLIC_DOMAINCFG_DM      BIT(2, U)
 #define APLIC_DOMAINCFG_BE      BIT(0, U)
 
+/* sourcecfg register fields */
 #define APLIC_SOURCECFG_SM_INACTIVE     0x0
 #define APLIC_SOURCECFG_SM_DETACH       0x1
 #define APLIC_SOURCECFG_SM_EDGE_RISE    0x4
@@ -32,7 +35,39 @@
 #define APLIC_SOURCECFG_SM_LEVEL_HIGH   0x6
 #define APLIC_SOURCECFG_SM_LEVEL_LOW    0x7
 
+/* target register fields */
 #define APLIC_TARGET_HART_IDX_SHIFT 18
+#define APLIC_TARGET_EIID_MASK      0x7ff
+
+#define APLIC_DOMAINCFG         0x0000
+#define APLIC_SOURCECFG_BASE    0x0004
+#define APLIC_SOURCECFG_LAST    0x0ffc
+
+#define APLIC_SMSICFGADDR       0x1bc8
+#define APLIC_SMSICFGADDRH      0x1bcc
+
+#define APLIC_SETIP_BASE        0x1c00
+#define APLIC_SETIP_LAST        0x1c7c
+#define APLIC_SETIPNUM          0x1cdc
+
+#define APLIC_CLRIP_BASE        0x1d00
+#define APLIC_CLRIP_LAST        0x1d7c
+#define APLIC_CLRIPNUM          0x1ddc
+
+#define APLIC_SETIE_BASE        0x1e00
+#define APLIC_SETIE_LAST        0x1e7c
+#define APLIC_SETIENUM          0x1edc
+
+#define APLIC_CLRIE_BASE        0x1f00
+#define APLIC_CLRIE_LAST        0x1f7c
+#define APLIC_CLRIENUM          0x1fdc
+
+#define APLIC_SETIPNUM_LE       0x2000
+
+#define APLIC_GENMSI            0x3000
+
+#define APLIC_TARGET_BASE       0x3004
+#define APLIC_TARGET_LAST       0x3ffc
 
 #define APLIC_IDC_SIZE          32
 
-- 
2.54.0

Re: [PATCH v1 03/17] xen/riscv: add missing APLIC register offsets, masks to asm/aplic.h

[Jan Beulich]

On 20.07.2026 18:02, Oleksii Kurochko wrote:
> @@ -25,6 +27,7 @@
>  #define APLIC_DOMAINCFG_DM      BIT(2, U)
>  #define APLIC_DOMAINCFG_BE      BIT(0, U)
>  
> +/* sourcecfg register fields */
>  #define APLIC_SOURCECFG_SM_INACTIVE     0x0
>  #define APLIC_SOURCECFG_SM_DETACH       0x1
>  #define APLIC_SOURCECFG_SM_EDGE_RISE    0x4
> @@ -32,7 +35,39 @@
>  #define APLIC_SOURCECFG_SM_LEVEL_HIGH   0x6
>  #define APLIC_SOURCECFG_SM_LEVEL_LOW    0x7
>  
> +/* target register fields */
>  #define APLIC_TARGET_HART_IDX_SHIFT 18
> +#define APLIC_TARGET_EIID_MASK      0x7ff

So why would these live here, far ahead of ...

> +#define APLIC_DOMAINCFG         0x0000
> +#define APLIC_SOURCECFG_BASE    0x0004
> +#define APLIC_SOURCECFG_LAST    0x0ffc
> +
> +#define APLIC_SMSICFGADDR       0x1bc8
> +#define APLIC_SMSICFGADDRH      0x1bcc
> +
> +#define APLIC_SETIP_BASE        0x1c00
> +#define APLIC_SETIP_LAST        0x1c7c
> +#define APLIC_SETIPNUM          0x1cdc
> +
> +#define APLIC_CLRIP_BASE        0x1d00
> +#define APLIC_CLRIP_LAST        0x1d7c
> +#define APLIC_CLRIPNUM          0x1ddc
> +
> +#define APLIC_SETIE_BASE        0x1e00
> +#define APLIC_SETIE_LAST        0x1e7c
> +#define APLIC_SETIENUM          0x1edc
> +
> +#define APLIC_CLRIE_BASE        0x1f00
> +#define APLIC_CLRIE_LAST        0x1f7c
> +#define APLIC_CLRIENUM          0x1fdc
> +
> +#define APLIC_SETIPNUM_LE       0x2000
> +
> +#define APLIC_GENMSI            0x3000
> +
> +#define APLIC_TARGET_BASE       0x3004
> +#define APLIC_TARGET_LAST       0x3ffc

... the register definition itself. Would you mind taking a look at the top
third (or so) of arch/x86/include/asm/msr-index.h? There you'll find MSR
index values of MSRs we use, immediately followed by field definitions (of
course only where applicable). With such an arrangement, the extra comments
you add can easily be omitted.

Jan

Re: [PATCH v1 03/17] xen/riscv: add missing APLIC register offsets, masks to asm/aplic.h

[Oleksii Kurochko]

On 7/28/26 2:02 PM, Jan Beulich wrote:
> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>> @@ -25,6 +27,7 @@
>>   #define APLIC_DOMAINCFG_DM      BIT(2, U)
>>   #define APLIC_DOMAINCFG_BE      BIT(0, U)
>>   
>> +/* sourcecfg register fields */
>>   #define APLIC_SOURCECFG_SM_INACTIVE     0x0
>>   #define APLIC_SOURCECFG_SM_DETACH       0x1
>>   #define APLIC_SOURCECFG_SM_EDGE_RISE    0x4
>> @@ -32,7 +35,39 @@
>>   #define APLIC_SOURCECFG_SM_LEVEL_HIGH   0x6
>>   #define APLIC_SOURCECFG_SM_LEVEL_LOW    0x7
>>   
>> +/* target register fields */
>>   #define APLIC_TARGET_HART_IDX_SHIFT 18
>> +#define APLIC_TARGET_EIID_MASK      0x7ff
> 
> So why would these live here, far ahead of ...
> 
>> +#define APLIC_DOMAINCFG         0x0000
>> +#define APLIC_SOURCECFG_BASE    0x0004
>> +#define APLIC_SOURCECFG_LAST    0x0ffc
>> +
>> +#define APLIC_SMSICFGADDR       0x1bc8
>> +#define APLIC_SMSICFGADDRH      0x1bcc
>> +
>> +#define APLIC_SETIP_BASE        0x1c00
>> +#define APLIC_SETIP_LAST        0x1c7c
>> +#define APLIC_SETIPNUM          0x1cdc
>> +
>> +#define APLIC_CLRIP_BASE        0x1d00
>> +#define APLIC_CLRIP_LAST        0x1d7c
>> +#define APLIC_CLRIPNUM          0x1ddc
>> +
>> +#define APLIC_SETIE_BASE        0x1e00
>> +#define APLIC_SETIE_LAST        0x1e7c
>> +#define APLIC_SETIENUM          0x1edc
>> +
>> +#define APLIC_CLRIE_BASE        0x1f00
>> +#define APLIC_CLRIE_LAST        0x1f7c
>> +#define APLIC_CLRIENUM          0x1fdc
>> +
>> +#define APLIC_SETIPNUM_LE       0x2000
>> +
>> +#define APLIC_GENMSI            0x3000
>> +
>> +#define APLIC_TARGET_BASE       0x3004
>> +#define APLIC_TARGET_LAST       0x3ffc
> 
> ... the register definition itself. Would you mind taking a look at the top
> third (or so) of arch/x86/include/asm/msr-index.h? There you'll find MSR
> index values of MSRs we use, immediately followed by field definitions (of
> course only where applicable). With such an arrangement, the extra comments
> you add can easily be omitted.

I'll move the definitions mentioned above to the suggested location and 
follow this pattern in future changes. I just thought it would be better 
to keep the APLIC register definitions together in memory layout order.

Thanks.

~ Oleksii

Re: [PATCH v1 03/17] xen/riscv: add missing APLIC register offsets, masks to asm/aplic.h

[Jan Beulich]

On 29.07.2026 17:26, Oleksii Kurochko wrote:
> 
> 
> On 7/28/26 2:02 PM, Jan Beulich wrote:
>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>> @@ -25,6 +27,7 @@
>>>   #define APLIC_DOMAINCFG_DM      BIT(2, U)
>>>   #define APLIC_DOMAINCFG_BE      BIT(0, U)
>>>   
>>> +/* sourcecfg register fields */
>>>   #define APLIC_SOURCECFG_SM_INACTIVE     0x0
>>>   #define APLIC_SOURCECFG_SM_DETACH       0x1
>>>   #define APLIC_SOURCECFG_SM_EDGE_RISE    0x4
>>> @@ -32,7 +35,39 @@
>>>   #define APLIC_SOURCECFG_SM_LEVEL_HIGH   0x6
>>>   #define APLIC_SOURCECFG_SM_LEVEL_LOW    0x7
>>>   
>>> +/* target register fields */
>>>   #define APLIC_TARGET_HART_IDX_SHIFT 18
>>> +#define APLIC_TARGET_EIID_MASK      0x7ff
>>
>> So why would these live here, far ahead of ...
>>
>>> +#define APLIC_DOMAINCFG         0x0000
>>> +#define APLIC_SOURCECFG_BASE    0x0004
>>> +#define APLIC_SOURCECFG_LAST    0x0ffc
>>> +
>>> +#define APLIC_SMSICFGADDR       0x1bc8
>>> +#define APLIC_SMSICFGADDRH      0x1bcc
>>> +
>>> +#define APLIC_SETIP_BASE        0x1c00
>>> +#define APLIC_SETIP_LAST        0x1c7c
>>> +#define APLIC_SETIPNUM          0x1cdc
>>> +
>>> +#define APLIC_CLRIP_BASE        0x1d00
>>> +#define APLIC_CLRIP_LAST        0x1d7c
>>> +#define APLIC_CLRIPNUM          0x1ddc
>>> +
>>> +#define APLIC_SETIE_BASE        0x1e00
>>> +#define APLIC_SETIE_LAST        0x1e7c
>>> +#define APLIC_SETIENUM          0x1edc
>>> +
>>> +#define APLIC_CLRIE_BASE        0x1f00
>>> +#define APLIC_CLRIE_LAST        0x1f7c
>>> +#define APLIC_CLRIENUM          0x1fdc
>>> +
>>> +#define APLIC_SETIPNUM_LE       0x2000
>>> +
>>> +#define APLIC_GENMSI            0x3000
>>> +
>>> +#define APLIC_TARGET_BASE       0x3004
>>> +#define APLIC_TARGET_LAST       0x3ffc
>>
>> ... the register definition itself. Would you mind taking a look at the top
>> third (or so) of arch/x86/include/asm/msr-index.h? There you'll find MSR
>> index values of MSRs we use, immediately followed by field definitions (of
>> course only where applicable). With such an arrangement, the extra comments
>> you add can easily be omitted.
> 
> I'll move the definitions mentioned above to the suggested location and 
> follow this pattern in future changes. I just thought it would be better 
> to keep the APLIC register definitions together in memory layout order.

Retaining that order is certainly wanted. Keeping them together is also wanted,
just with the bit/field definitions inserted accordingly.

Jan

Re: [PATCH v1 03/17] xen/riscv: add missing APLIC register offsets, masks to asm/aplic.h

[Baptiste Le Duc]

[-- Attachment #1: Type: text/plain, Size: 1048 bytes --]

> These definitions are required for correct decoding of APLIC MMIO
> accesses and target configuration, and will be used by both the
> physical and virtual APLIC implementations.
> 
> No functional change is intended by this patch; it only centralises
> hardware definitions that were previously missing.
> 
> Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>
> diff --git a/xen/arch/riscv/include/asm/aplic.h b/xen/arch/riscv/include/asm/aplic.h
> index 07318aaac2..f22622b9a2 100644
> --- a/xen/arch/riscv/include/asm/aplic.h
> +++ b/xen/arch/riscv/include/asm/aplic.h
> @@ -15,6 +15,8 @@
>  
>  #include <asm/imsic.h>
>  
> +#define APLIC_REG_OFFSET_MASK   0x3fff

_REG stands for memory-mapped control "region" as explained in spec? If
yes, it'd be better to add a comment.

-- 
Baptiste Le Duc <baptiste.le-duc@vates.tech>


--
Baptiste Le Duc | Vates XCP-ng Intern

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech

Re: [PATCH v1 03/17] xen/riscv: add missing APLIC register offsets, masks to asm/aplic.h

[Oleksii Kurochko]

On 8/10/26 3:45 PM, Baptiste Le Duc wrote:
>> These definitions are required for correct decoding of APLIC MMIO
>> accesses and target configuration, and will be used by both the
>> physical and virtual APLIC implementations.
>>
>> No functional change is intended by this patch; it only centralises
>> hardware definitions that were previously missing.
>>
>> Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>
>> diff --git a/xen/arch/riscv/include/asm/aplic.h b/xen/arch/riscv/include/asm/aplic.h
>> index 07318aaac2..f22622b9a2 100644
>> --- a/xen/arch/riscv/include/asm/aplic.h
>> +++ b/xen/arch/riscv/include/asm/aplic.h
>> @@ -15,6 +15,8 @@
>>   
>>   #include <asm/imsic.h>
>>   
>> +#define APLIC_REG_OFFSET_MASK   0x3fff
> 
> _REG stands for memory-mapped control "region" as explained in spec? If
> yes, it'd be better to add a comment.
> 

Yes, it is a mask that allows us to get the offsets for the registers of 
an interrupt domain’s memory-mapped control region.

IMO, if the problem is with the name of the macro, it would be better to 
use a clearer name instead of adding a comment. For example, 
`APLIC_CTRL_REGION_OFFSET_MASK` sounds self-explanatory to me.

If you’re still not happy with the suggested name and it isn't 
self-explanatory, then:

/*
  * Offsets of the registers of an interrupt domain's memory-mapped control
  * region, which is APLIC_MIN_SIZE bytes large.
  */
#define APLIC_CTRL_REGION_OFFSET_MASK   0x3fff

Alternatively, I could keep the old name (APLIC_REG_OFFSET_MASK) and use 
the suggested comment.

Which option do you prefer?

~ Oleksii

Re: [PATCH v1 03/17] xen/riscv: add missing APLIC register offsets, masks to asm/aplic.h

[Baptiste Le Duc]

[-- Attachment #1: Type: text/plain, Size: 1346 bytes --]

On 2026-08-10 16:45:23+02:00, Oleksii Kurochko wrote:
> On 8/10/26 3:45 PM, Baptiste Le Duc wrote:
> 
> >> These definitions are required for correct decoding of APLIC MMIO
> > 
> > _REG stands for memory-mapped control "region" as explained in spec? If
> > yes, it'd be better to add a comment.
> 
> Yes, it is a mask that allows us to get the offsets for the registers of 
> an interrupt domain’s memory-mapped control region.
> 
> IMO, if the problem is with the name of the macro, it would be better to 
> use a clearer name instead of adding a comment. For example, 
> `APLIC_CTRL_REGION_OFFSET_MASK` sounds self-explanatory to me.

I think the rename of the macro is enough.
Lets keep this:

#define APLIC_CTRL_REGION_OFFSET_MASK   0x3fff

> 
> If you’re still not happy with the suggested name and it isn't 
> self-explanatory, then:
> 
> /*
>   * Offsets of the registers of an interrupt domain's memory-mapped control
>   * region, which is APLIC_MIN_SIZE bytes large.
>   */
> #define APLIC_CTRL_REGION_OFFSET_MASK   0x3fff
> 
> Alternatively, I could keep the old name (APLIC_REG_OFFSET_MASK) and use 
> the suggested comment.
> 
> Which option do you prefer?
> 
> ~ Oleksii




--
Baptiste Le Duc | Vates XCP-ng Intern

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech

[PATCH v1 04/17] xen/riscv: introduce device-agnostic MMIO emulation dispatch

[Oleksii Kurochko]

RISC-V guests can expose several virtual interrupt controllers at
distinct GPA ranges: vPLIC (hasn't been introduced yet) for legacy machines,
vAPLIC and vIMSIC for AIA-compliant ones (is being introduced in the follow
up patches). Routing MMIO faults via a per-device is_access() check in the
trap handler would couple it to every device it must serve, requiring a
new conditional branch in the fault path each time a new emulated device is
added.

Introduce a per-domain MMIO handler registration table, modeled
after the equivalent ARM framework, so that virtual devices
self-register their GPA ranges and read/write callbacks at domain
creation time. The MMIO fault path delegates to a single
try_handle_mmio() entry point and remains agnostic of which device
owns a particular address.

Subsequent patches wire this into arch_domain_create() and the MMIO fault
path in traps.c.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech>
---
Note that find_mmio_handler() and try_handle_mmio() is handling found
handler differently for now in comparison to Arm. But this behaviour will
be aligned at the end. Look at discussion:
  https://lore.kernel.org/xen-devel/cd78972e-88d5-471d-a201-5f9cd1392c73@gmail.com/T/#t
---
---
 xen/arch/riscv/Makefile             |   1 +
 xen/arch/riscv/domain.c             |   4 +
 xen/arch/riscv/include/asm/domain.h |   3 +
 xen/arch/riscv/include/asm/mmio.h   |  63 ++++++++++++
 xen/arch/riscv/mmio.c               | 145 ++++++++++++++++++++++++++++
 5 files changed, 216 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/mmio.h
 create mode 100644 xen/arch/riscv/mmio.c

diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
index 046f73f4d87c..c452ebc3cf61 100644
--- a/xen/arch/riscv/Makefile
+++ b/xen/arch/riscv/Makefile
@@ -14,6 +14,7 @@ obj-y += intc.o
 obj-y += irq.o
 obj-y += kernel.init.o
 obj-y += mm.o
+obj-y += mmio.o
 obj-y += p2m.o
 obj-y += paging.o
 obj-y += pt.o
diff --git a/xen/arch/riscv/domain.c b/xen/arch/riscv/domain.c
index 4db9c28662c7..1e6f0ef66c2f 100644
--- a/xen/arch/riscv/domain.c
+++ b/xen/arch/riscv/domain.c
@@ -12,6 +12,7 @@
 #include <asm/cpufeature.h>
 #include <asm/csr.h>
 #include <asm/intc.h>
+#include <asm/mmio.h>
 #include <asm/riscv_encoding.h>
 #include <asm/vtimer.h>
 
@@ -308,6 +309,9 @@ int arch_domain_create(struct domain *d,
     if ( (rc = p2m_init(d, config)) != 0)
         goto fail;
 
+    if ( (rc = domain_io_init(d, MAX_IO_HANDLER)) != 0 )
+        goto fail;
+
     if ( (rc = domain_vintc_init(d)) )
         goto fail;
 
diff --git a/xen/arch/riscv/include/asm/domain.h b/xen/arch/riscv/include/asm/domain.h
index e035b33ddfdc..15e8fa19685e 100644
--- a/xen/arch/riscv/include/asm/domain.h
+++ b/xen/arch/riscv/include/asm/domain.h
@@ -9,6 +9,7 @@
 
 #include <asm/cpufeature.h>
 #include <asm/guest-layout.h>
+#include <asm/mmio.h>
 #include <asm/p2m.h>
 #include <asm/vtimer.h>
 
@@ -101,6 +102,8 @@ struct arch_domain {
     const unsigned long *isa;
 
     struct vintc *vintc;
+
+    struct vmmio vmmio;
 };
 
 #include <xen/sched.h>
diff --git a/xen/arch/riscv/include/asm/mmio.h b/xen/arch/riscv/include/asm/mmio.h
new file mode 100644
index 000000000000..18df1133e621
--- /dev/null
+++ b/xen/arch/riscv/include/asm/mmio.h
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#ifndef RISCV_MMIO_H
+#define RISCV_MMIO_H
+
+#include <xen/lib.h>
+#include <xen/rwlock.h>
+
+#define MAX_IO_HANDLER  16
+
+typedef struct {
+    paddr_t gpa;
+    unsigned int len;  /* access width in bytes (1, 2, 4, 8) */
+    bool is_write;
+    register_t data;   /* store: value to write; load: value read (set by handler) */
+} mmio_info_t;
+
+enum io_state
+{
+    IO_ABORT,       /* The IO was handled and led to an abort. */
+    IO_HANDLED,     /* The IO was successfully handled. */
+    IO_UNHANDLED,   /* No handler found for the IO. */
+};
+
+typedef enum io_state (*mmio_read_t)(struct vcpu *v, mmio_info_t *info,
+                                     register_t *r);
+typedef enum io_state (*mmio_write_t)(struct vcpu *v, mmio_info_t *info,
+                                      register_t r);
+
+struct mmio_handler_ops {
+    mmio_read_t read;
+    mmio_write_t write;
+};
+
+struct mmio_handler {
+    paddr_t addr;
+    paddr_t size;
+    const struct mmio_handler_ops *ops;
+};
+
+struct vmmio {
+    unsigned int num_entries;
+    unsigned int max_num_entries;
+    rwlock_t lock;
+    struct mmio_handler *handlers;
+};
+
+enum io_state try_handle_mmio(mmio_info_t *info);
+void register_mmio_handler(struct domain *d,
+                           const struct mmio_handler_ops *ops,
+                           paddr_t addr, paddr_t size);
+int domain_io_init(struct domain *d, unsigned int max_count);
+void domain_io_free(struct domain *d);
+
+#endif /* RISCV_MMIO_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/riscv/mmio.c b/xen/arch/riscv/mmio.c
new file mode 100644
index 000000000000..7d56bc8b27c5
--- /dev/null
+++ b/xen/arch/riscv/mmio.c
@@ -0,0 +1,145 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) Vates
+ */
+
+#include <xen/bsearch.h>
+#include <xen/lib.h>
+#include <xen/rwlock.h>
+#include <xen/sched.h>
+#include <xen/sort.h>
+#include <xen/xvmalloc.h>
+
+#include <asm/current.h>
+#include <asm/mmio.h>
+
+static enum io_state handle_read(const struct mmio_handler *handler,
+                                 struct vcpu *v,
+                                 mmio_info_t *info)
+{
+    register_t r = 0;
+    enum io_state rc;
+
+    rc = handler->ops->read(v, info, &r);
+    if ( rc == IO_HANDLED )
+        info->data = r;
+
+    return rc;
+}
+
+static enum io_state handle_write(const struct mmio_handler *handler,
+                                  struct vcpu *v,
+                                  mmio_info_t *info)
+{
+    return handler->ops->write(v, info, info->data);
+}
+
+/* Assumes mmio regions are not overlapping. */
+static int cmp_mmio_handler(const void *key, const void *elem)
+{
+    const struct mmio_handler *handler0 = key;
+    const struct mmio_handler *handler1 = elem;
+
+    if ( handler0->addr < handler1->addr )
+        return -1;
+
+    if ( handler0->addr >= (handler1->addr + handler1->size) )
+        return 1;
+
+    return 0;
+}
+
+static void swap_mmio_handler(void *a, void *b)
+{
+    struct mmio_handler *t1 = a, *t2 = b;
+
+    SWAP(*t1, *t2);
+}
+
+/*
+ * Return a copy of the matching handler rather than a pointer into
+ * vmmio->handlers: a concurrent register_mmio_handler() re-sorts the
+ * array, so an escaped pointer could refer to a different (or torn)
+ * entry once the lock is dropped.  The copy stays valid as the ops
+ * structures are never freed.
+ */
+static bool find_mmio_handler(struct domain *d, paddr_t gpa,
+                              struct mmio_handler *out)
+{
+    struct vmmio *vmmio = &d->arch.vmmio;
+    struct mmio_handler key = { .addr = gpa };
+    const struct mmio_handler *handler;
+
+    read_lock(&vmmio->lock);
+    handler = bsearch(&key, vmmio->handlers, vmmio->num_entries,
+                      sizeof(*handler), cmp_mmio_handler);
+    if ( handler )
+        *out = *handler;
+    read_unlock(&vmmio->lock);
+
+    return handler != NULL;
+}
+
+enum io_state try_handle_mmio(mmio_info_t *info)
+{
+    struct vcpu *v = current;
+    struct mmio_handler handler = {};
+
+    if ( !find_mmio_handler(v->domain, info->gpa, &handler) )
+        return IO_UNHANDLED;
+
+    if ( info->is_write )
+        return handle_write(&handler, v, info);
+    else
+        return handle_read(&handler, v, info);
+}
+
+void register_mmio_handler(struct domain *d,
+                           const struct mmio_handler_ops *ops,
+                           paddr_t addr, paddr_t size)
+{
+    struct vmmio *vmmio = &d->arch.vmmio;
+    struct mmio_handler *handler;
+
+    write_lock(&vmmio->lock);
+
+    BUG_ON(vmmio->num_entries >= vmmio->max_num_entries);
+
+    handler = &vmmio->handlers[vmmio->num_entries];
+    handler->ops = ops;
+    handler->addr = addr;
+    handler->size = size;
+    vmmio->num_entries++;
+
+    /* Sort mmio handlers in ascending order based on base address */
+    sort(vmmio->handlers, vmmio->num_entries, sizeof(struct mmio_handler),
+         cmp_mmio_handler, swap_mmio_handler);
+
+    write_unlock(&vmmio->lock);
+}
+
+int domain_io_init(struct domain *d, unsigned int max_count)
+{
+    rwlock_init(&d->arch.vmmio.lock);
+    d->arch.vmmio.num_entries = 0;
+    d->arch.vmmio.max_num_entries = max_count;
+    d->arch.vmmio.handlers = xvzalloc_array(struct mmio_handler, max_count);
+    if ( !d->arch.vmmio.handlers )
+        return -ENOMEM;
+
+    return 0;
+}
+
+void domain_io_free(struct domain *d)
+{
+    XVFREE(d->arch.vmmio.handlers);
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.54.0

Re: [PATCH v1 04/17] xen/riscv: introduce device-agnostic MMIO emulation dispatch

[Jan Beulich]

On 20.07.2026 18:02, Oleksii Kurochko wrote:
> RISC-V guests can expose several virtual interrupt controllers at
> distinct GPA ranges: vPLIC (hasn't been introduced yet) for legacy machines,
> vAPLIC and vIMSIC for AIA-compliant ones (is being introduced in the follow
> up patches). Routing MMIO faults via a per-device is_access() check in the
> trap handler would couple it to every device it must serve, requiring a
> new conditional branch in the fault path each time a new emulated device is
> added.
> 
> Introduce a per-domain MMIO handler registration table, modeled
> after the equivalent ARM framework, so that virtual devices
> self-register their GPA ranges and read/write callbacks at domain
> creation time. The MMIO fault path delegates to a single
> try_handle_mmio() entry point and remains agnostic of which device
> owns a particular address.
> 
> Subsequent patches wire this into arch_domain_create() and the MMIO fault
> path in traps.c.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech>
> ---
> Note that find_mmio_handler() and try_handle_mmio() is handling found
> handler differently for now in comparison to Arm. But this behaviour will
> be aligned at the end. Look at discussion:
>   https://lore.kernel.org/xen-devel/cd78972e-88d5-471d-a201-5f9cd1392c73@gmail.com/T/#t
> ---
> ---
>  xen/arch/riscv/Makefile             |   1 +
>  xen/arch/riscv/domain.c             |   4 +
>  xen/arch/riscv/include/asm/domain.h |   3 +
>  xen/arch/riscv/include/asm/mmio.h   |  63 ++++++++++++
>  xen/arch/riscv/mmio.c               | 145 ++++++++++++++++++++++++++++
>  5 files changed, 216 insertions(+)
>  create mode 100644 xen/arch/riscv/include/asm/mmio.h
>  create mode 100644 xen/arch/riscv/mmio.c
> 
> diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
> index 046f73f4d87c..c452ebc3cf61 100644
> --- a/xen/arch/riscv/Makefile
> +++ b/xen/arch/riscv/Makefile
> @@ -14,6 +14,7 @@ obj-y += intc.o
>  obj-y += irq.o
>  obj-y += kernel.init.o
>  obj-y += mm.o
> +obj-y += mmio.o
>  obj-y += p2m.o
>  obj-y += paging.o
>  obj-y += pt.o
> diff --git a/xen/arch/riscv/domain.c b/xen/arch/riscv/domain.c
> index 4db9c28662c7..1e6f0ef66c2f 100644
> --- a/xen/arch/riscv/domain.c
> +++ b/xen/arch/riscv/domain.c
> @@ -12,6 +12,7 @@
>  #include <asm/cpufeature.h>
>  #include <asm/csr.h>
>  #include <asm/intc.h>
> +#include <asm/mmio.h>
>  #include <asm/riscv_encoding.h>
>  #include <asm/vtimer.h>
>  
> @@ -308,6 +309,9 @@ int arch_domain_create(struct domain *d,
>      if ( (rc = p2m_init(d, config)) != 0)
>          goto fail;
>  
> +    if ( (rc = domain_io_init(d, MAX_IO_HANDLER)) != 0 )
> +        goto fail;

Why does MAX_IO_HANDLER need passing into the function? Isn't that a global
boundary?

> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/mmio.h
> @@ -0,0 +1,63 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +#ifndef RISCV_MMIO_H
> +#define RISCV_MMIO_H
> +
> +#include <xen/lib.h>
> +#include <xen/rwlock.h>
> +
> +#define MAX_IO_HANDLER  16
> +
> +typedef struct {
> +    paddr_t gpa;
> +    unsigned int len;  /* access width in bytes (1, 2, 4, 8) */
> +    bool is_write;
> +    register_t data;   /* store: value to write; load: value read (set by handler) */
> +} mmio_info_t;
> +
> +enum io_state
> +{
> +    IO_ABORT,       /* The IO was handled and led to an abort. */
> +    IO_HANDLED,     /* The IO was successfully handled. */
> +    IO_UNHANDLED,   /* No handler found for the IO. */
> +};
> +
> +typedef enum io_state (*mmio_read_t)(struct vcpu *v, mmio_info_t *info,
> +                                     register_t *r);
> +typedef enum io_state (*mmio_write_t)(struct vcpu *v, mmio_info_t *info,
> +                                      register_t r);

Can't info be pointer-to-const in the write case? In both cases, why is there
both "r" passed into the function as well as the info->data field, supposedly
(as per the comment) serving the same purpose?

Furthermore I think it helps if ...

> +struct mmio_handler_ops {
> +    mmio_read_t read;
> +    mmio_write_t write;

... pointer-ness is easily seen at use sites. I.e.

typedef enum io_state mmio_read_t(struct vcpu *v, mmio_info_t *info,
                                  register_t *r);
typedef enum io_state mmio_write_t(struct vcpu *v, const mmio_info_t *info,
                                   register_t r);

struct mmio_handler_ops {
    mmio_read_t *read;
    mmio_write_t *write;
};

> +};
> +
> +struct mmio_handler {
> +    paddr_t addr;
> +    paddr_t size;
> +    const struct mmio_handler_ops *ops;
> +};
> +
> +struct vmmio {
> +    unsigned int num_entries;
> +    unsigned int max_num_entries;
> +    rwlock_t lock;
> +    struct mmio_handler *handlers;

There shouldn't be any writes through this pointer, should there? In which
case it (once again) wants to be pointer-to-const.

> --- /dev/null
> +++ b/xen/arch/riscv/mmio.c
> @@ -0,0 +1,145 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +/*
> + * Copyright (C) Vates
> + */
> +
> +#include <xen/bsearch.h>
> +#include <xen/lib.h>
> +#include <xen/rwlock.h>
> +#include <xen/sched.h>
> +#include <xen/sort.h>
> +#include <xen/xvmalloc.h>
> +
> +#include <asm/current.h>
> +#include <asm/mmio.h>
> +
> +static enum io_state handle_read(const struct mmio_handler *handler,
> +                                 struct vcpu *v,
> +                                 mmio_info_t *info)
> +{
> +    register_t r = 0;
> +    enum io_state rc;
> +
> +    rc = handler->ops->read(v, info, &r);
> +    if ( rc == IO_HANDLED )
> +        info->data = r;

Extending my earlier comment: Why could ->read() not put the value directly
into info->data? And why ...

> +static enum io_state handle_write(const struct mmio_handler *handler,
> +                                  struct vcpu *v,
> +                                  mmio_info_t *info)
> +{
> +    return handler->ops->write(v, info, info->data);

... can't write take the value directly from info->data?

> +}
> +
> +/* Assumes mmio regions are not overlapping. */

Are you guaranteeing this anywhere?

> +static int cmp_mmio_handler(const void *key, const void *elem)
> +{
> +    const struct mmio_handler *handler0 = key;
> +    const struct mmio_handler *handler1 = elem;
> +
> +    if ( handler0->addr < handler1->addr )
> +        return -1;
> +
> +    if ( handler0->addr >= (handler1->addr + handler1->size) )
> +        return 1;
> +
> +    return 0;
> +}
> +
> +static void swap_mmio_handler(void *a, void *b)
> +{
> +    struct mmio_handler *t1 = a, *t2 = b;
> +
> +    SWAP(*t1, *t2);
> +}
> +
> +/*
> + * Return a copy of the matching handler rather than a pointer into
> + * vmmio->handlers: a concurrent register_mmio_handler() re-sorts the
> + * array, so an escaped pointer could refer to a different (or torn)
> + * entry once the lock is dropped.  The copy stays valid as the ops
> + * structures are never freed.
> + */
> +static bool find_mmio_handler(struct domain *d, paddr_t gpa,
> +                              struct mmio_handler *out)
> +{
> +    struct vmmio *vmmio = &d->arch.vmmio;
> +    struct mmio_handler key = { .addr = gpa };
> +    const struct mmio_handler *handler;
> +
> +    read_lock(&vmmio->lock);
> +    handler = bsearch(&key, vmmio->handlers, vmmio->num_entries,
> +                      sizeof(*handler), cmp_mmio_handler);

So beyond the assumption stated further up you also assume the array to
be sorted. Which you ...

> +void register_mmio_handler(struct domain *d,
> +                           const struct mmio_handler_ops *ops,
> +                           paddr_t addr, paddr_t size)
> +{
> +    struct vmmio *vmmio = &d->arch.vmmio;
> +    struct mmio_handler *handler;
> +
> +    write_lock(&vmmio->lock);
> +
> +    BUG_ON(vmmio->num_entries >= vmmio->max_num_entries);

(Do we really need to crash in such a case? Can't we just fail domain
creation?)

> +    handler = &vmmio->handlers[vmmio->num_entries];
> +    handler->ops = ops;
> +    handler->addr = addr;
> +    handler->size = size;
> +    vmmio->num_entries++;
> +
> +    /* Sort mmio handlers in ascending order based on base address */
> +    sort(vmmio->handlers, vmmio->num_entries, sizeof(struct mmio_handler),
> +         cmp_mmio_handler, swap_mmio_handler);

... arrange for here, yet in a pretty inefficient way: Inserting in an
already sorted list can be had without recurring calls to sort().

> +int domain_io_init(struct domain *d, unsigned int max_count)
> +{
> +    rwlock_init(&d->arch.vmmio.lock);
> +    d->arch.vmmio.num_entries = 0;
> +    d->arch.vmmio.max_num_entries = max_count;
> +    d->arch.vmmio.handlers = xvzalloc_array(struct mmio_handler, max_count);

If already an allocation is needed in all cases, why not allocate struct
vmmio, defined like this:

struct vmmio {
    unsigned int num_entries;
    unsigned int max_num_entries;
    rwlock_t lock;
    struct mmio_handler handlers[];
};

and then using xvzalloc_flex_struct(). Or yet simpler if (as mentioned
elsewhere) max_count doesn't need passing into here:

struct vmmio {
    unsigned int num_entries;
    unsigned int max_num_entries;
    rwlock_t lock;
    struct mmio_handler handlers[MAX_IO_HANDLER];
};

Jan

Re: [PATCH v1 04/17] xen/riscv: introduce device-agnostic MMIO emulation dispatch

[Oleksii Kurochko]

On 7/28/26 2:23 PM, Jan Beulich wrote:
> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>> RISC-V guests can expose several virtual interrupt controllers at
>> distinct GPA ranges: vPLIC (hasn't been introduced yet) for legacy machines,
>> vAPLIC and vIMSIC for AIA-compliant ones (is being introduced in the follow
>> up patches). Routing MMIO faults via a per-device is_access() check in the
>> trap handler would couple it to every device it must serve, requiring a
>> new conditional branch in the fault path each time a new emulated device is
>> added.
>>
>> Introduce a per-domain MMIO handler registration table, modeled
>> after the equivalent ARM framework, so that virtual devices
>> self-register their GPA ranges and read/write callbacks at domain
>> creation time. The MMIO fault path delegates to a single
>> try_handle_mmio() entry point and remains agnostic of which device
>> owns a particular address.
>>
>> Subsequent patches wire this into arch_domain_create() and the MMIO fault
>> path in traps.c.
>>
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>> Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech>
>> ---
>> Note that find_mmio_handler() and try_handle_mmio() is handling found
>> handler differently for now in comparison to Arm. But this behaviour will
>> be aligned at the end. Look at discussion:
>>    https://lore.kernel.org/xen-devel/cd78972e-88d5-471d-a201-5f9cd1392c73@gmail.com/T/#t
>> ---
>> ---
>>   xen/arch/riscv/Makefile             |   1 +
>>   xen/arch/riscv/domain.c             |   4 +
>>   xen/arch/riscv/include/asm/domain.h |   3 +
>>   xen/arch/riscv/include/asm/mmio.h   |  63 ++++++++++++
>>   xen/arch/riscv/mmio.c               | 145 ++++++++++++++++++++++++++++
>>   5 files changed, 216 insertions(+)
>>   create mode 100644 xen/arch/riscv/include/asm/mmio.h
>>   create mode 100644 xen/arch/riscv/mmio.c
>>
>> diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
>> index 046f73f4d87c..c452ebc3cf61 100644
>> --- a/xen/arch/riscv/Makefile
>> +++ b/xen/arch/riscv/Makefile
>> @@ -14,6 +14,7 @@ obj-y += intc.o
>>   obj-y += irq.o
>>   obj-y += kernel.init.o
>>   obj-y += mm.o
>> +obj-y += mmio.o
>>   obj-y += p2m.o
>>   obj-y += paging.o
>>   obj-y += pt.o
>> diff --git a/xen/arch/riscv/domain.c b/xen/arch/riscv/domain.c
>> index 4db9c28662c7..1e6f0ef66c2f 100644
>> --- a/xen/arch/riscv/domain.c
>> +++ b/xen/arch/riscv/domain.c
>> @@ -12,6 +12,7 @@
>>   #include <asm/cpufeature.h>
>>   #include <asm/csr.h>
>>   #include <asm/intc.h>
>> +#include <asm/mmio.h>
>>   #include <asm/riscv_encoding.h>
>>   #include <asm/vtimer.h>
>>   
>> @@ -308,6 +309,9 @@ int arch_domain_create(struct domain *d,
>>       if ( (rc = p2m_init(d, config)) != 0)
>>           goto fail;
>>   
>> +    if ( (rc = domain_io_init(d, MAX_IO_HANDLER)) != 0 )
>> +        goto fail;
> 
> Why does MAX_IO_HANDLER need passing into the function? Isn't that a global
> boundary?

Good question. Considering that all domains are initialized with 
MAX_IO_HANDLER I think we could drop an argument for domain_io_init() 
and just use MAX_IO_HANDLER inside it for init. of handlers array.

> 
>> --- /dev/null
>> +++ b/xen/arch/riscv/include/asm/mmio.h
>> @@ -0,0 +1,63 @@
>> +/* SPDX-License-Identifier: GPL-2.0-or-later */
>> +#ifndef RISCV_MMIO_H
>> +#define RISCV_MMIO_H
>> +
>> +#include <xen/lib.h>
>> +#include <xen/rwlock.h>
>> +
>> +#define MAX_IO_HANDLER  16
>> +
>> +typedef struct {
>> +    paddr_t gpa;
>> +    unsigned int len;  /* access width in bytes (1, 2, 4, 8) */
>> +    bool is_write;
>> +    register_t data;   /* store: value to write; load: value read (set by handler) */
>> +} mmio_info_t;
>> +
>> +enum io_state
>> +{
>> +    IO_ABORT,       /* The IO was handled and led to an abort. */
>> +    IO_HANDLED,     /* The IO was successfully handled. */
>> +    IO_UNHANDLED,   /* No handler found for the IO. */
>> +};
>> +
>> +typedef enum io_state (*mmio_read_t)(struct vcpu *v, mmio_info_t *info,
>> +                                     register_t *r);
>> +typedef enum io_state (*mmio_write_t)(struct vcpu *v, mmio_info_t *info,
>> +                                      register_t r);
> 
> Can't info be pointer-to-const in the write case?

With the current implementaion it could be done for both mmio_read_t and 
mmio_write_t as value is return through r argument.

In both cases, why is there
> both "r" passed into the function as well as the info->data field, supposedly
> (as per the comment) serving the same purpose?

Agree, we don't need both "r" and info->data as they are serving the 
same purpose.

But I don't know which one option is actually better to drop "r" 
argument or drop ->data member in mmio_info_t.


> 
> Furthermore I think it helps if ...
> 
>> +struct mmio_handler_ops {
>> +    mmio_read_t read;
>> +    mmio_write_t write;
> 
> ... pointer-ness is easily seen at use sites. I.e.
> 
> typedef enum io_state mmio_read_t(struct vcpu *v, mmio_info_t *info,
>                                    register_t *r);
> typedef enum io_state mmio_write_t(struct vcpu *v, const mmio_info_t *info,
>                                     register_t r);
> 
> struct mmio_handler_ops {
>      mmio_read_t *read;
>      mmio_write_t *write;
> };
> 

I will apply that.

>> +};
>> +
>> +struct mmio_handler {
>> +    paddr_t addr;
>> +    paddr_t size;
>> +    const struct mmio_handler_ops *ops;
>> +};
>> +
>> +struct vmmio {
>> +    unsigned int num_entries;
>> +    unsigned int max_num_entries;
>> +    rwlock_t lock;
>> +    struct mmio_handler *handlers;
> 
> There shouldn't be any writes through this pointer, should there? In which
> case it (once again) wants to be pointer-to-const.

Agree, it should be const.

> 
>> --- /dev/null
>> +++ b/xen/arch/riscv/mmio.c
>> @@ -0,0 +1,145 @@
>> +/* SPDX-License-Identifier: GPL-2.0-or-later */
>> +/*
>> + * Copyright (C) Vates
>> + */
>> +
>> +#include <xen/bsearch.h>
>> +#include <xen/lib.h>
>> +#include <xen/rwlock.h>
>> +#include <xen/sched.h>
>> +#include <xen/sort.h>
>> +#include <xen/xvmalloc.h>
>> +
>> +#include <asm/current.h>
>> +#include <asm/mmio.h>
>> +
>> +static enum io_state handle_read(const struct mmio_handler *handler,
>> +                                 struct vcpu *v,
>> +                                 mmio_info_t *info)
>> +{
>> +    register_t r = 0;
>> +    enum io_state rc;
>> +
>> +    rc = handler->ops->read(v, info, &r);
>> +    if ( rc == IO_HANDLED )
>> +        info->data = r;
> 
> Extending my earlier comment: Why could ->read() not put the value directly
> into info->data? And why ...
> 
>> +static enum io_state handle_write(const struct mmio_handler *handler,
>> +                                  struct vcpu *v,
>> +                                  mmio_info_t *info)
>> +{
>> +    return handler->ops->write(v, info, info->data);
> 
> ... can't write take the value directly from info->data?

I totally agree, it can. Do you think it is better to keep ->data and 
drop an argument 'r' or vice versa?

> 
>> +}
>> +
>> +/* Assumes mmio regions are not overlapping. */
> 
> Are you guaranteeing this anywhere?

There is no such guarantee. register_mmio_handler() simply adds the 
handler to the handlers array without performing any checks. I can add 
such a check. The only question is whether it should be enabled only in 
debug builds or in all builds.

I assume this is a rare case, and overlapping regions would indicate 
that something is wrong with the guest's memory layout configuration so 
it seems like it would be enough to add only for debug builds.

> 
>> +static int cmp_mmio_handler(const void *key, const void *elem)
>> +{
>> +    const struct mmio_handler *handler0 = key;
>> +    const struct mmio_handler *handler1 = elem;
>> +
>> +    if ( handler0->addr < handler1->addr )
>> +        return -1;
>> +
>> +    if ( handler0->addr >= (handler1->addr + handler1->size) )
>> +        return 1;
>> +
>> +    return 0;
>> +}
>> +
>> +static void swap_mmio_handler(void *a, void *b)
>> +{
>> +    struct mmio_handler *t1 = a, *t2 = b;
>> +
>> +    SWAP(*t1, *t2);
>> +}
>> +
>> +/*
>> + * Return a copy of the matching handler rather than a pointer into
>> + * vmmio->handlers: a concurrent register_mmio_handler() re-sorts the
>> + * array, so an escaped pointer could refer to a different (or torn)
>> + * entry once the lock is dropped.  The copy stays valid as the ops
>> + * structures are never freed.
>> + */
>> +static bool find_mmio_handler(struct domain *d, paddr_t gpa,
>> +                              struct mmio_handler *out)
>> +{
>> +    struct vmmio *vmmio = &d->arch.vmmio;
>> +    struct mmio_handler key = { .addr = gpa };
>> +    const struct mmio_handler *handler;
>> +
>> +    read_lock(&vmmio->lock);
>> +    handler = bsearch(&key, vmmio->handlers, vmmio->num_entries,
>> +                      sizeof(*handler), cmp_mmio_handler);
> 
> So beyond the assumption stated further up you also assume the array to
> be sorted. Which you ...
> 
>> +void register_mmio_handler(struct domain *d,
>> +                           const struct mmio_handler_ops *ops,
>> +                           paddr_t addr, paddr_t size)
>> +{
>> +    struct vmmio *vmmio = &d->arch.vmmio;
>> +    struct mmio_handler *handler;
>> +
>> +    write_lock(&vmmio->lock);
>> +
>> +    BUG_ON(vmmio->num_entries >= vmmio->max_num_entries);
> 
> (Do we really need to crash in such a case? Can't we just fail domain
> creation?)

Generally, no. However, the approach used by Arm's dom0less solution is 
to crash as soon as any issue occurs instead of trying to continue 
running other domains, so I follow the same approach for RISC-V.

Even if I return an error here, the common dom0less code will panic anyway.

> 
>> +    handler = &vmmio->handlers[vmmio->num_entries];
>> +    handler->ops = ops;
>> +    handler->addr = addr;
>> +    handler->size = size;
>> +    vmmio->num_entries++;
>> +
>> +    /* Sort mmio handlers in ascending order based on base address */
>> +    sort(vmmio->handlers, vmmio->num_entries, sizeof(struct mmio_handler),
>> +         cmp_mmio_handler, swap_mmio_handler);
> 
> ... arrange for here, yet in a pretty inefficient way: Inserting in an
> already sorted list can be had without recurring calls to sort().

Good point. I will rework that.
> 
>> +int domain_io_init(struct domain *d, unsigned int max_count)
>> +{
>> +    rwlock_init(&d->arch.vmmio.lock);
>> +    d->arch.vmmio.num_entries = 0;
>> +    d->arch.vmmio.max_num_entries = max_count;
>> +    d->arch.vmmio.handlers = xvzalloc_array(struct mmio_handler, max_count);
> 
> If already an allocation is needed in all cases, why not allocate struct
> vmmio, defined like this:
> 
> struct vmmio {
>      unsigned int num_entries;
>      unsigned int max_num_entries;
>      rwlock_t lock;
>      struct mmio_handler handlers[];
> };
> 
> and then using xvzalloc_flex_struct(). Or yet simpler if (as mentioned
> elsewhere) max_count doesn't need passing into here:
> 
> struct vmmio {
>      unsigned int num_entries;
>      unsigned int max_num_entries;
>      rwlock_t lock;
>      struct mmio_handler handlers[MAX_IO_HANDLER];
> };
> 
Agree, both option are good to me. Considering that we are going to use 
MAX_IO_HANDLER then second option is really better for now.

Thanks!

~ Oleksii

Re: [PATCH v1 04/17] xen/riscv: introduce device-agnostic MMIO emulation dispatch

[Jan Beulich]

On 30.07.2026 18:03, Oleksii Kurochko wrote:
> On 7/28/26 2:23 PM, Jan Beulich wrote:
>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>> --- /dev/null
>>> +++ b/xen/arch/riscv/mmio.c
>>> @@ -0,0 +1,145 @@
>>> +/* SPDX-License-Identifier: GPL-2.0-or-later */
>>> +/*
>>> + * Copyright (C) Vates
>>> + */
>>> +
>>> +#include <xen/bsearch.h>
>>> +#include <xen/lib.h>
>>> +#include <xen/rwlock.h>
>>> +#include <xen/sched.h>
>>> +#include <xen/sort.h>
>>> +#include <xen/xvmalloc.h>
>>> +
>>> +#include <asm/current.h>
>>> +#include <asm/mmio.h>
>>> +
>>> +static enum io_state handle_read(const struct mmio_handler *handler,
>>> +                                 struct vcpu *v,
>>> +                                 mmio_info_t *info)
>>> +{
>>> +    register_t r = 0;
>>> +    enum io_state rc;
>>> +
>>> +    rc = handler->ops->read(v, info, &r);
>>> +    if ( rc == IO_HANDLED )
>>> +        info->data = r;
>>
>> Extending my earlier comment: Why could ->read() not put the value directly
>> into info->data? And why ...
>>
>>> +static enum io_state handle_write(const struct mmio_handler *handler,
>>> +                                  struct vcpu *v,
>>> +                                  mmio_info_t *info)
>>> +{
>>> +    return handler->ops->write(v, info, info->data);
>>
>> ... can't write take the value directly from info->data?
> 
> I totally agree, it can. Do you think it is better to keep ->data and 
> drop an argument 'r' or vice versa?

How can I know? You know future plans you have.

>>> +}
>>> +
>>> +/* Assumes mmio regions are not overlapping. */
>>
>> Are you guaranteeing this anywhere?
> 
> There is no such guarantee. register_mmio_handler() simply adds the 
> handler to the handlers array without performing any checks. I can add 
> such a check. The only question is whether it should be enabled only in 
> debug builds or in all builds.

Depends on what other badness can happen when this is violated. My gut
feeling is that checking in debug builds may be enough.

>>> +/*
>>> + * Return a copy of the matching handler rather than a pointer into
>>> + * vmmio->handlers: a concurrent register_mmio_handler() re-sorts the
>>> + * array, so an escaped pointer could refer to a different (or torn)
>>> + * entry once the lock is dropped.  The copy stays valid as the ops
>>> + * structures are never freed.
>>> + */
>>> +static bool find_mmio_handler(struct domain *d, paddr_t gpa,
>>> +                              struct mmio_handler *out)
>>> +{
>>> +    struct vmmio *vmmio = &d->arch.vmmio;
>>> +    struct mmio_handler key = { .addr = gpa };
>>> +    const struct mmio_handler *handler;
>>> +
>>> +    read_lock(&vmmio->lock);
>>> +    handler = bsearch(&key, vmmio->handlers, vmmio->num_entries,
>>> +                      sizeof(*handler), cmp_mmio_handler);
>>
>> So beyond the assumption stated further up you also assume the array to
>> be sorted. Which you ...
>>
>>> +void register_mmio_handler(struct domain *d,
>>> +                           const struct mmio_handler_ops *ops,
>>> +                           paddr_t addr, paddr_t size)
>>> +{
>>> +    struct vmmio *vmmio = &d->arch.vmmio;
>>> +    struct mmio_handler *handler;
>>> +
>>> +    write_lock(&vmmio->lock);
>>> +
>>> +    BUG_ON(vmmio->num_entries >= vmmio->max_num_entries);
>>
>> (Do we really need to crash in such a case? Can't we just fail domain
>> creation?)
> 
> Generally, no. However, the approach used by Arm's dom0less solution is 
> to crash as soon as any issue occurs instead of trying to continue 
> running other domains, so I follow the same approach for RISC-V.
> 
> Even if I return an error here, the common dom0less code will panic anyway.

That's the policy there, but you're writing code here also for the case where
Dom0 creates domains.

Jan

Re: [PATCH v1 04/17] xen/riscv: introduce device-agnostic MMIO emulation dispatch

[Oleksii Kurochko]

On 7/30/26 6:09 PM, Jan Beulich wrote:
> On 30.07.2026 18:03, Oleksii Kurochko wrote:
>> On 7/28/26 2:23 PM, Jan Beulich wrote:
>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>> --- /dev/null
>>>> +++ b/xen/arch/riscv/mmio.c
>>>> @@ -0,0 +1,145 @@
>>>> +/* SPDX-License-Identifier: GPL-2.0-or-later */
>>>> +/*
>>>> + * Copyright (C) Vates
>>>> + */
>>>> +
>>>> +#include <xen/bsearch.h>
>>>> +#include <xen/lib.h>
>>>> +#include <xen/rwlock.h>
>>>> +#include <xen/sched.h>
>>>> +#include <xen/sort.h>
>>>> +#include <xen/xvmalloc.h>
>>>> +
>>>> +#include <asm/current.h>
>>>> +#include <asm/mmio.h>
>>>> +
>>>> +static enum io_state handle_read(const struct mmio_handler *handler,
>>>> +                                 struct vcpu *v,
>>>> +                                 mmio_info_t *info)
>>>> +{
>>>> +    register_t r = 0;
>>>> +    enum io_state rc;
>>>> +
>>>> +    rc = handler->ops->read(v, info, &r);
>>>> +    if ( rc == IO_HANDLED )
>>>> +        info->data = r;
>>>
>>> Extending my earlier comment: Why could ->read() not put the value directly
>>> into info->data? And why ...
>>>
>>>> +static enum io_state handle_write(const struct mmio_handler *handler,
>>>> +                                  struct vcpu *v,
>>>> +                                  mmio_info_t *info)
>>>> +{
>>>> +    return handler->ops->write(v, info, info->data);
>>>
>>> ... can't write take the value directly from info->data?
>>
>> I totally agree, it can. Do you think it is better to keep ->data and
>> drop an argument 'r' or vice versa?
> 
> How can I know? You know future plans you have.
> 
>>>> +}
>>>> +
>>>> +/* Assumes mmio regions are not overlapping. */
>>>
>>> Are you guaranteeing this anywhere?
>>
>> There is no such guarantee. register_mmio_handler() simply adds the
>> handler to the handlers array without performing any checks. I can add
>> such a check. The only question is whether it should be enabled only in
>> debug builds or in all builds.
> 
> Depends on what other badness can happen when this is violated. My gut
> feeling is that checking in debug builds may be enough.

Overlapping regions would be a Xen bug rather than something a guest can 
trigger — register_mmio_handler() is only called from Xen's own emulated 
device code, so the layout isn't under guest control.

The badness is worse than just mis-emulating one device though: 
cmp_mmio_handler() is used both by bsearch() and by sort(). With 
overlapping regions it's no longer a consistent ordering, so sort() may 
produce an arbitrary order and lookups can then fail (or match the wrong 
handler) even for regions which don't overlap themselves. That would 
show up as a spurious fault injected into the guest, which is quite hard 
to debug.

So I agree a check is worthwhile; I'll add one under CONFIG_DEBUG in 
register_mmio_handler().

> 
>>>> +/*
>>>> + * Return a copy of the matching handler rather than a pointer into
>>>> + * vmmio->handlers: a concurrent register_mmio_handler() re-sorts the
>>>> + * array, so an escaped pointer could refer to a different (or torn)
>>>> + * entry once the lock is dropped.  The copy stays valid as the ops
>>>> + * structures are never freed.
>>>> + */
>>>> +static bool find_mmio_handler(struct domain *d, paddr_t gpa,
>>>> +                              struct mmio_handler *out)
>>>> +{
>>>> +    struct vmmio *vmmio = &d->arch.vmmio;
>>>> +    struct mmio_handler key = { .addr = gpa };
>>>> +    const struct mmio_handler *handler;
>>>> +
>>>> +    read_lock(&vmmio->lock);
>>>> +    handler = bsearch(&key, vmmio->handlers, vmmio->num_entries,
>>>> +                      sizeof(*handler), cmp_mmio_handler);
>>>
>>> So beyond the assumption stated further up you also assume the array to
>>> be sorted. Which you ...
>>>
>>>> +void register_mmio_handler(struct domain *d,
>>>> +                           const struct mmio_handler_ops *ops,
>>>> +                           paddr_t addr, paddr_t size)
>>>> +{
>>>> +    struct vmmio *vmmio = &d->arch.vmmio;
>>>> +    struct mmio_handler *handler;
>>>> +
>>>> +    write_lock(&vmmio->lock);
>>>> +
>>>> +    BUG_ON(vmmio->num_entries >= vmmio->max_num_entries);
>>>
>>> (Do we really need to crash in such a case? Can't we just fail domain
>>> creation?)
>>
>> Generally, no. However, the approach used by Arm's dom0less solution is
>> to crash as soon as any issue occurs instead of trying to continue
>> running other domains, so I follow the same approach for RISC-V.
>>
>> Even if I return an error here, the common dom0less code will panic anyway.
> 
> That's the policy there, but you're writing code here also for the case where
> Dom0 creates domains.

Missed that. In this case I agree that it would be nice to return something.

Thanks.

~ Oleksii

Re: [PATCH v1 04/17] xen/riscv: introduce device-agnostic MMIO emulation dispatch

[Jan Beulich]

On 31.07.2026 17:24, Oleksii Kurochko wrote:
> On 7/30/26 6:09 PM, Jan Beulich wrote:
>> On 30.07.2026 18:03, Oleksii Kurochko wrote:
>>> On 7/28/26 2:23 PM, Jan Beulich wrote:
>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>> --- /dev/null
>>>>> +++ b/xen/arch/riscv/mmio.c
>>>>> @@ -0,0 +1,145 @@
>>>>> +/* SPDX-License-Identifier: GPL-2.0-or-later */
>>>>> +/*
>>>>> + * Copyright (C) Vates
>>>>> + */
>>>>> +
>>>>> +#include <xen/bsearch.h>
>>>>> +#include <xen/lib.h>
>>>>> +#include <xen/rwlock.h>
>>>>> +#include <xen/sched.h>
>>>>> +#include <xen/sort.h>
>>>>> +#include <xen/xvmalloc.h>
>>>>> +
>>>>> +#include <asm/current.h>
>>>>> +#include <asm/mmio.h>
>>>>> +
>>>>> +static enum io_state handle_read(const struct mmio_handler *handler,
>>>>> +                                 struct vcpu *v,
>>>>> +                                 mmio_info_t *info)
>>>>> +{
>>>>> +    register_t r = 0;
>>>>> +    enum io_state rc;
>>>>> +
>>>>> +    rc = handler->ops->read(v, info, &r);
>>>>> +    if ( rc == IO_HANDLED )
>>>>> +        info->data = r;
>>>>
>>>> Extending my earlier comment: Why could ->read() not put the value directly
>>>> into info->data? And why ...
>>>>
>>>>> +static enum io_state handle_write(const struct mmio_handler *handler,
>>>>> +                                  struct vcpu *v,
>>>>> +                                  mmio_info_t *info)
>>>>> +{
>>>>> +    return handler->ops->write(v, info, info->data);
>>>>
>>>> ... can't write take the value directly from info->data?
>>>
>>> I totally agree, it can. Do you think it is better to keep ->data and
>>> drop an argument 'r' or vice versa?
>>
>> How can I know? You know future plans you have.
>>
>>>>> +}
>>>>> +
>>>>> +/* Assumes mmio regions are not overlapping. */
>>>>
>>>> Are you guaranteeing this anywhere?
>>>
>>> There is no such guarantee. register_mmio_handler() simply adds the
>>> handler to the handlers array without performing any checks. I can add
>>> such a check. The only question is whether it should be enabled only in
>>> debug builds or in all builds.
>>
>> Depends on what other badness can happen when this is violated. My gut
>> feeling is that checking in debug builds may be enough.
> 
> Overlapping regions would be a Xen bug rather than something a guest can 
> trigger — register_mmio_handler() is only called from Xen's own emulated 
> device code, so the layout isn't under guest control.
> 
> The badness is worse than just mis-emulating one device though: 
> cmp_mmio_handler() is used both by bsearch() and by sort(). With 
> overlapping regions it's no longer a consistent ordering, so sort() may 
> produce an arbitrary order and lookups can then fail (or match the wrong 
> handler) even for regions which don't overlap themselves. That would 
> show up as a spurious fault injected into the guest, which is quite hard 
> to debug.

Didn't you say you'd get rid of the use of sort()?

> So I agree a check is worthwhile; I'll add one under CONFIG_DEBUG in 
> register_mmio_handler().

Some assertion then hopefully, rather than an open-coded use of CONFIG_DEBUG.

Jan

Re: [PATCH v1 04/17] xen/riscv: introduce device-agnostic MMIO emulation dispatch

[Oleksii Kurochko]

On 8/3/26 12:41 PM, Jan Beulich wrote:
> On 31.07.2026 17:24, Oleksii Kurochko wrote:
>> On 7/30/26 6:09 PM, Jan Beulich wrote:
>>> On 30.07.2026 18:03, Oleksii Kurochko wrote:
>>>> On 7/28/26 2:23 PM, Jan Beulich wrote:
>>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>>> --- /dev/null
>>>>>> +++ b/xen/arch/riscv/mmio.c
>>>>>> @@ -0,0 +1,145 @@
>>>>>> +/* SPDX-License-Identifier: GPL-2.0-or-later */
>>>>>> +/*
>>>>>> + * Copyright (C) Vates
>>>>>> + */
>>>>>> +
>>>>>> +#include <xen/bsearch.h>
>>>>>> +#include <xen/lib.h>
>>>>>> +#include <xen/rwlock.h>
>>>>>> +#include <xen/sched.h>
>>>>>> +#include <xen/sort.h>
>>>>>> +#include <xen/xvmalloc.h>
>>>>>> +
>>>>>> +#include <asm/current.h>
>>>>>> +#include <asm/mmio.h>
>>>>>> +
>>>>>> +static enum io_state handle_read(const struct mmio_handler *handler,
>>>>>> +                                 struct vcpu *v,
>>>>>> +                                 mmio_info_t *info)
>>>>>> +{
>>>>>> +    register_t r = 0;
>>>>>> +    enum io_state rc;
>>>>>> +
>>>>>> +    rc = handler->ops->read(v, info, &r);
>>>>>> +    if ( rc == IO_HANDLED )
>>>>>> +        info->data = r;
>>>>>
>>>>> Extending my earlier comment: Why could ->read() not put the value directly
>>>>> into info->data? And why ...
>>>>>
>>>>>> +static enum io_state handle_write(const struct mmio_handler *handler,
>>>>>> +                                  struct vcpu *v,
>>>>>> +                                  mmio_info_t *info)
>>>>>> +{
>>>>>> +    return handler->ops->write(v, info, info->data);
>>>>>
>>>>> ... can't write take the value directly from info->data?
>>>>
>>>> I totally agree, it can. Do you think it is better to keep ->data and
>>>> drop an argument 'r' or vice versa?
>>>
>>> How can I know? You know future plans you have.
>>>
>>>>>> +}
>>>>>> +
>>>>>> +/* Assumes mmio regions are not overlapping. */
>>>>>
>>>>> Are you guaranteeing this anywhere?
>>>>
>>>> There is no such guarantee. register_mmio_handler() simply adds the
>>>> handler to the handlers array without performing any checks. I can add
>>>> such a check. The only question is whether it should be enabled only in
>>>> debug builds or in all builds.
>>>
>>> Depends on what other badness can happen when this is violated. My gut
>>> feeling is that checking in debug builds may be enough.
>>
>> Overlapping regions would be a Xen bug rather than something a guest can
>> trigger — register_mmio_handler() is only called from Xen's own emulated
>> device code, so the layout isn't under guest control.
>>
>> The badness is worse than just mis-emulating one device though:
>> cmp_mmio_handler() is used both by bsearch() and by sort(). With
>> overlapping regions it's no longer a consistent ordering, so sort() may
>> produce an arbitrary order and lookups can then fail (or match the wrong
>> handler) even for regions which don't overlap themselves. That would
>> show up as a spurious fault injected into the guest, which is quite hard
>> to debug.
> 
> Didn't you say you'd get rid of the use of sort()?
> 
Yes, I will. I just wrote that for the case if sort() will still present.

~ Oleksii

Re: [PATCH v1 04/17] xen/riscv: introduce device-agnostic MMIO emulation dispatch

[Baptiste Le Duc]

[-- Attachment #1: Type: text/plain, Size: 5561 bytes --]

> RISC-V guests can expose several virtual interrupt controllers at
> distinct GPA ranges: vPLIC (hasn't been introduced yet) for legacy machines,
> vAPLIC and vIMSIC for AIA-compliant ones (is being introduced in the follow
> up patches). Routing MMIO faults via a per-device is_access() check in the
> trap handler would couple it to every device it must serve, requiring a
> new conditional branch in the fault path each time a new emulated device is
> added.
> 
> Introduce a per-domain MMIO handler registration table, modeled
> after the equivalent ARM framework, so that virtual devices
> self-register their GPA ranges and read/write callbacks at domain
> creation time. The MMIO fault path delegates to a single
> try_handle_mmio() entry point and remains agnostic of which device
> owns a particular address.
> 
> Subsequent patches wire this into arch_domain_create() and the MMIO fault
> path in traps.c.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech>
>
> diff --git a/xen/arch/riscv/Makefile b/xen/arch/riscv/Makefile
> index 046f73f4d8..c452ebc3cf 100644
> --- a/xen/arch/riscv/Makefile
> +++ b/xen/arch/riscv/Makefile
> @@ -14,6 +14,7 @@ obj-y += intc.o
>  obj-y += irq.o
>  obj-y += kernel.init.o
>  obj-y += mm.o
> +obj-y += mmio.o
>  obj-y += p2m.o
>  obj-y += paging.o
>  obj-y += pt.o
> diff --git a/xen/arch/riscv/domain.c b/xen/arch/riscv/domain.c
> index 4db9c28662..1e6f0ef66c 100644
> --- a/xen/arch/riscv/domain.c
> +++ b/xen/arch/riscv/domain.c
> @@ -12,6 +12,7 @@
>  #include <asm/cpufeature.h>
>  #include <asm/csr.h>
>  #include <asm/intc.h>
> +#include <asm/mmio.h>
>  #include <asm/riscv_encoding.h>
>  #include <asm/vtimer.h>
>  
> @@ -308,6 +309,9 @@ int arch_domain_create(struct domain *d,
>      if ( (rc = p2m_init(d, config)) != 0)
>          goto fail;
>  
> +    if ( (rc = domain_io_init(d, MAX_IO_HANDLER)) != 0 )


> +        goto fail;


> +
>      if ( (rc = domain_vintc_init(d)) )
>          goto fail;
>  
> diff --git a/xen/arch/riscv/include/asm/domain.h b/xen/arch/riscv/include/asm/domain.h
> index e035b33ddf..15e8fa1968 100644
> --- a/xen/arch/riscv/include/asm/domain.h
> +++ b/xen/arch/riscv/include/asm/domain.h
> @@ -9,6 +9,7 @@
>  
>  #include <asm/cpufeature.h>
>  #include <asm/guest-layout.h>
> +#include <asm/mmio.h>
>  #include <asm/p2m.h>
>  #include <asm/vtimer.h>
>  
> @@ -101,6 +102,8 @@ struct arch_domain {
>      const unsigned long *isa;
>  
>      struct vintc *vintc;
> +
> +    struct vmmio vmmio;
>  };
>  
>  #include <xen/sched.h>
> diff --git a/xen/arch/riscv/include/asm/mmio.h b/xen/arch/riscv/include/asm/mmio.h
> new file mode 100644
> index 0000000000..18df1133e6
> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/mmio.h
> @@ -0,0 +1,63 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */

According to coding style, it should be GPL-2.0-only.

> +#ifndef RISCV_MMIO_H
> +#define RISCV_MMIO_H
> +
> +#include <xen/lib.h>
> +#include <xen/rwlock.h>
> +
> +#define MAX_IO_HANDLER  16
> +
> +typedef struct {
> +    paddr_t gpa;
> +    unsigned int len;  /* access width in bytes (1, 2, 4, 8) */
> +    bool is_write;
> +    register_t data;   /* store: value to write; load: value read (set by handler) */

Nit: line too long (85)

> +} mmio_info_t;
> +
> +enum io_state
> +{
> +    IO_ABORT,       /* The IO was handled and led to an abort. */
> +    IO_HANDLED,     /* The IO was successfully handled. */
> +    IO_UNHANDLED,   /* No handler found for the IO. */
> +};
> +
> +typedef enum io_state (*mmio_read_t)(struct vcpu *v, mmio_info_t *info,
> +                                     register_t *r);
> +typedef enum io_state (*mmio_write_t)(struct vcpu *v, mmio_info_t *info,
> +                                      register_t r);


> +
> +struct mmio_handler_ops {
> +    mmio_read_t read;
> +    mmio_write_t write;


> +};
> +
> +struct mmio_handler {
> +    paddr_t addr;
> +    paddr_t size;
> +    const struct mmio_handler_ops *ops;
> +};
> +
> +struct vmmio {
> +    unsigned int num_entries;
> +    unsigned int max_num_entries;
> +    rwlock_t lock;
> +    struct mmio_handler *handlers;


> +};
> +
> +enum io_state try_handle_mmio(mmio_info_t *info);

> +void register_mmio_handler(struct domain *d,
> +                           const struct mmio_handler_ops *ops,
> +                           paddr_t addr, paddr_t size);
> +int domain_io_init(struct domain *d, unsigned int max_count);
> +void domain_io_free(struct domain *d);


> +
> +#endif /* RISCV_MMIO_H */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/riscv/mmio.c b/xen/arch/riscv/mmio.c
> new file mode 100644
> index 0000000000..7d56bc8b27
> --- /dev/null
> +++ b/xen/arch/riscv/mmio.c
> @@ -0,0 +1,145 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
Should be GPL-2.0-only.
> +/*
> + * Copyright (C) Vates
> + */
Why have you included a copyright notice here, but not in the other
files? I don’t know if you can keep it, but I just wanted to point out
that there are other files where this type of copyright notice includes
the year.

-- 
Baptiste Le Duc <baptiste.le-duc@vates.tech>


--
Baptiste Le Duc | Vates XCP-ng Intern

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech

Re: [PATCH v1 04/17] xen/riscv: introduce device-agnostic MMIO emulation dispatch

[Oleksii Kurochko]

On 8/10/26 4:49 PM, Baptiste Le Duc wrote:
>> diff --git a/xen/arch/riscv/include/asm/mmio.h b/xen/arch/riscv/include/asm/mmio.h
>> new file mode 100644
>> index 0000000000..18df1133e6
>> --- /dev/null
>> +++ b/xen/arch/riscv/include/asm/mmio.h
>> @@ -0,0 +1,63 @@
>> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> 
> According to coding style, it should be GPL-2.0-only.

Could you please point me to the line in the coding style document where 
this is mentioned?

If you are referring to:
   New files should start with a single-line SPDX comment to express the
   license, e.g.:

   /* SPDX-License-Identifier: GPL-2.0-only */

   See LICENSES/ for a list of licenses and SPDX tags currently used.

Then my understanding is that /* SPDX-License-Identifier: GPL-2.0-only 
*/ is used only as an example, and I can choose any license from 
LICENSES/. There, it is mentioned:
   Valid-License-Identifier: LGPL-2.0-only
   Valid-License-Identifier: LGPL-2.0-or-later

I am pretty sure that I am free to choose any license that does not 
conflict with the other licenses used in the project.

>> +#ifndef RISCV_MMIO_H
>> +#define RISCV_MMIO_H
>> +
>> +#include <xen/lib.h>
>> +#include <xen/rwlock.h>
>> +
>> +#define MAX_IO_HANDLER  16
>> +
>> +typedef struct {
>> +    paddr_t gpa;
>> +    unsigned int len;  /* access width in bytes (1, 2, 4, 8) */
>> +    bool is_write;
>> +    register_t data;   /* store: value to write; load: value read (set by handler) */
> 
> Nit: line too long (85)

I will apply that. Actually I've already fixed that by putting the 
comment above:
   /* store: value to write; load: value read (set by handler) */
   register_t data;


>> diff --git a/xen/arch/riscv/mmio.c b/xen/arch/riscv/mmio.c
>> new file mode 100644
>> index 0000000000..7d56bc8b27
>> --- /dev/null
>> +++ b/xen/arch/riscv/mmio.c
>> @@ -0,0 +1,145 @@
>> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> Should be GPL-2.0-only.

Regarding license I've wrote a comment above so lets continue discussion 
there.

>> +/*
>> + * Copyright (C) Vates
>> + */
> Why have you included a copyright notice here, but not in the other
> files? 

So I just decided to do that for new files as I am not using corporate 
e-mail.

I don’t know if you can keep it,

Good point, I have to ask then someone from our legal department...

  but I just wanted to point out
> that there are other files where this type of copyright notice includes
> the year.
> 

Before, I used to include the year, but someone pointed out (or perhaps 
I misunderstood) that there isn’t much point in including it and that it 
is enough to have just (c) <company name>.

Thanks.

~ Oleksii

Re: [PATCH v1 04/17] xen/riscv: introduce device-agnostic MMIO emulation dispatch

[Baptiste Le Duc]

On 2026-08-10 17:36:56+02:00, Oleksii Kurochko wrote:
> On 8/10/26 4:49 PM, Baptiste Le Duc wrote:
> 
> >> diff --git a/xen/arch/riscv/include/asm/mmio.h b/xen/arch/riscv/include/asm/mmio.h
> > 
> > According to coding style, it should be GPL-2.0-only.
> 
> Could you please point me to the line in the coding style document where 
> this is mentioned?
> 
> If you are referring to:
>    New files should start with a single-line SPDX comment to express the
>    license, e.g.:
> 
>    /* SPDX-License-Identifier: GPL-2.0-only */
> 
>    See LICENSES/ for a list of licenses and SPDX tags currently used.
> 
> Then my understanding is that /* SPDX-License-Identifier: GPL-2.0-only 
> */ is used only as an example, and I can choose any license from 
> LICENSES/. There, it is mentioned:
>    Valid-License-Identifier: LGPL-2.0-only
>    Valid-License-Identifier: LGPL-2.0-or-later
> 
> I am pretty sure that I am free to choose any license that does not 
> conflict with the other licenses used in the project.
> 
Oh ok I didn't know, thanks for these explanations. Could you let me
know how do you choose one instead of the other in that case? Is there a
rule from our company to follow somewhere?
> >> +#ifndef RISCV_MMIO_H
> > 
> > Nit: line too long (85)
> 
> I will apply that. Actually I've already fixed that by putting the 
> comment above:
>    /* store: value to write; load: value read (set by handler) */
>    register_t data;
> 
> >> diff --git a/xen/arch/riscv/mmio.c b/xen/arch/riscv/mmio.c
> > Should be GPL-2.0-only.
> 
> Regarding license I've wrote a comment above so lets continue discussion 
> there.
> 
> >> +/*
> > Why have you included a copyright notice here, but not in the other
> > files?
> 
> So I just decided to do that for new files as I am not using corporate 
> e-mail.
But why didn't you do it for all new files of this series?
> 
> I don’t know if you can keep it,
> 
> Good point, I have to ask then someone from our legal department...
> 
>   but I just wanted to point out
> 
> > that there are other files where this type of copyright notice includes
> > the year.
> 
> Before, I used to include the year, but someone pointed out (or perhaps 
> I misunderstood) that there isn’t much point in including it and that it 
> is enough to have just (c) <company name>.
> 
Okay thanks.
> Thanks.
> 
> ~ Oleksii

Re: [PATCH v1 04/17] xen/riscv: introduce device-agnostic MMIO emulation dispatch

[Oleksii Kurochko]

On 8/11/26 10:17 AM, Baptiste Le Duc wrote:
> On 2026-08-10 17:36:56+02:00, Oleksii Kurochko wrote:
>> On 8/10/26 4:49 PM, Baptiste Le Duc wrote:
>>
>>>> diff --git a/xen/arch/riscv/include/asm/mmio.h b/xen/arch/riscv/include/asm/mmio.h
>>>
>>> According to coding style, it should be GPL-2.0-only.
>>
>> Could you please point me to the line in the coding style document where
>> this is mentioned?
>>
>> If you are referring to:
>>     New files should start with a single-line SPDX comment to express the
>>     license, e.g.:
>>
>>     /* SPDX-License-Identifier: GPL-2.0-only */
>>
>>     See LICENSES/ for a list of licenses and SPDX tags currently used.
>>
>> Then my understanding is that /* SPDX-License-Identifier: GPL-2.0-only
>> */ is used only as an example, and I can choose any license from
>> LICENSES/. There, it is mentioned:
>>     Valid-License-Identifier: LGPL-2.0-only
>>     Valid-License-Identifier: LGPL-2.0-or-later
>>
>> I am pretty sure that I am free to choose any license that does not
>> conflict with the other licenses used in the project.
>>
> Oh ok I didn't know, thanks for these explanations. Could you let me
> know how do you choose one instead of the other in that case? Is there a
> rule from our company to follow somewhere?

I don't know about any specific rule from our company.

In different situations different licenses could/should be used. 
Specifically here I used GPL-2.0-or-later as this code partially is 
based on Arm code which uses this license so I just re-use it.

>>>> +#ifndef RISCV_MMIO_H
>>>
>>> Nit: line too long (85)
>>
>> I will apply that. Actually I've already fixed that by putting the
>> comment above:
>>     /* store: value to write; load: value read (set by handler) */
>>     register_t data;
>>
>>>> diff --git a/xen/arch/riscv/mmio.c b/xen/arch/riscv/mmio.c
>>> Should be GPL-2.0-only.
>>
>> Regarding license I've wrote a comment above so lets continue discussion
>> there.
>>
>>>> +/*
>>> Why have you included a copyright notice here, but not in the other
>>> files?
>>
>> So I just decided to do that for new files as I am not using corporate
>> e-mail.
> But why didn't you do it for all new files of this series?
If there are such cases then I just missed to add it. I will double 
check during preparation of v2.

~ Oleksii

Re: [PATCH v1 04/17] xen/riscv: introduce device-agnostic MMIO emulation dispatch

[Jan Beulich]

On 11.08.2026 13:49, Oleksii Kurochko wrote:
> On 8/11/26 10:17 AM, Baptiste Le Duc wrote:
>> On 2026-08-10 17:36:56+02:00, Oleksii Kurochko wrote:
>>> On 8/10/26 4:49 PM, Baptiste Le Duc wrote:
>>>>> +/*
>>>> Why have you included a copyright notice here, but not in the other
>>>> files?
>>>
>>> So I just decided to do that for new files as I am not using corporate
>>> e-mail.
>> But why didn't you do it for all new files of this series?
> If there are such cases then I just missed to add it. I will double 
> check during preparation of v2.

Hmm, I'm a little irritated by "missed to add". In past discussions (plural)
the usefulness of such copyright notices in comments was put under question.
They go stale anyway as code moves around. And likely there were other
aspects that I forgot.

Jan

Re: [PATCH v1 04/17] xen/riscv: introduce device-agnostic MMIO emulation dispatch

[Oleksii Kurochko]

On 8/12/26 9:21 AM, Jan Beulich wrote:
> On 11.08.2026 13:49, Oleksii Kurochko wrote:
>> On 8/11/26 10:17 AM, Baptiste Le Duc wrote:
>>> On 2026-08-10 17:36:56+02:00, Oleksii Kurochko wrote:
>>>> On 8/10/26 4:49 PM, Baptiste Le Duc wrote:
>>>>>> +/*
>>>>> Why have you included a copyright notice here, but not in the other
>>>>> files?
>>>>
>>>> So I just decided to do that for new files as I am not using corporate
>>>> e-mail.
>>> But why didn't you do it for all new files of this series?
>> If there are such cases then I just missed to add it. I will double
>> check during preparation of v2.
> 
> Hmm, I'm a little irritated by "missed to add". In past discussions (plural)
> the usefulness of such copyright notices in comments was put under question.
> They go stale anyway as code moves around. And likely there were other
> aspects that I forgot.

Then lets follow a strategy not to add such copyright notices in comments.

~ Oleksii

[PATCH v1 05/17] xen/riscv: implement virtual APLIC MMIO emulation

[Oleksii Kurochko]

Guests running under Xen program interrupt routing by writing to APLIC
MMIO registers. Xen must intercept these accesses to enforce interrupt
isolation between domains and to translate guest routing intent into the
underlying physical MSI topology.

Writes are gated by the domain's authorised interrupt bitmap so that a
guest cannot affect interrupts it does not own. TARGET register writes
additionally require translation of the hart and IMSIC guest-file
indices from virtual to physical, as the APLIC uses these fields
directly to compute the MSI delivery address.

Delegation (APLIC_SOURCECFG_D) is not yet supported.

Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech> # vaplic_mmio_{read,write}

The downstream changes related to `vaplic_mmio_{read,write}` were originally
in a separate patch (which was reviewed by Baptiste). However, before
upstreaming, it was decided to merge them into the current patch.
I added `Reviewed-by: Baptiste` in this form for now, but Baptiste will
probably review the remaining changes as well.
Once that happens, I'll simply move the `Reviewed-by` tag up and
remove the `#`.

If it will be easier I can move that changes to separate commit.
---
Changes in v3:
 - Drop ->is_access(), ->emulate_{store,load}() from struct vintc_ops and
   use MMIO framework instead.
 - Rename local variable base_ppn to tppn to not confuse it with base_ppn
   from AIA specification in the correspondent formula.
 - Move aplic_msi_target_gen() from vaplic.c to aplic.c.
 - Extract static aplic_hart_field() helper to compute the combined
   hart + group-index field for the TARGET register.
 - Use MASK_INSR() with APLIC_TARGET_GUEST_IDX_MASK and
   APLIC_TARGET_HART_IDX_MASK instead of open-coded shifts in
   aplic_msi_target_gen().
 - Add APLIC_xMSICFGADDR_PPN_SHIFT, _HHX_MASK, and _HHX_SHIFT macros
   to asm/aplic.h; use them in aplic_hart_field() to extract the group
   index from base_ppn.
 - Move APLIC_TARGET_{HART,GUEST}_IDX_MASK definitions to the preceding
   patch ("add missing APLIC register offsets, masks").
 - xen/arch/riscv/aplic.c:
   - Extend ASSERT() in aplic_hw_read_reg() and aplic_hw_write_reg() to
     also check 4-byte alignment via IS_ALIGNED(offset, sizeof(uint32_t)).
   - Use (volatile void __iomem *)aplic.regs + offset in readl()/writel()
     instead of the uintptr_t cast.
 - xen/arch/riscv/include/asm/aplic.h:
   - Drop unused APLIC_NUM_REGS macro.
   - Rewrite APLIC_SETCLR_OFFSET_MASK as
     (sizeof_field(struct aplic_regs, setip) - sizeof(uint32_t)) and add a
     comment explaining the choice of setip as a representative field.
 - xen/arch/riscv/include/asm/vaplic.h:
   - Change regs_size type from paddr_t to unsigned int.
 - xen/arch/riscv/vaplic.c:
   - s/regindx_to_irqn/regoffset_to_word_idx and add an explanatory comment.
   - s/irqsn/word_idx in generate_auth_mask().
   - Replace pointer-cast bitmap access in generate_auth_mask() with proper
     index arithmetic via first_bit to avoid strict-aliasing violation.
   - Add cf_check to vaplic_emulate_load().
   - s/vcpu/v in vaplic_emulate_load(), vaplic_emulate_store() and
     vaplic_is_access().
   - Fix comment typos: s/start for/start from/, s/substracting/subtracting/,
     drop stray 'of' after 'subtracting' in two comments.
   - AUTH_IRQ_BIT() intentionally uses '<' rather than '<=' when
     comparing irqn against nr_virqs: nr_virqs is a count of virtual IRQs
     and irqn is a 0-based index, so the valid range is
     [0, nr_virqs - 1] and irqn == nr_virqs is already out of bounds.
   - Update handling of 'case APLIC_DOMAINCFG'.
---
Changes in v2:
 - Merge the following patches into one:
    xen/riscv: add vaplic access check:
      - Add check that address is properly aligned.
      - Check vaplic range intead of APLIC one.
      - Return bool from vaplic_is_access instead of int.
    xen/riscv: emulate guest writes to virtual APLIC MMIO
      - Drop CALC_REG_VALUE.
      - Use unsigned int instead of uin32_t for offset.
      - s/.../subtracting in the comment.
      - start one line comments from the upper case.
      - Check the value before being written to sourcecfg register.
      - 'unsigned int' for loop index.
      - Omit unneessary braces.
      - s/vaplic_update_target/aplic_msi_target_gen.
      - Use IMSIC_MMIO_PAGE_SHIFT instead of 12 in aplic_msi_target_gen().
      - Drop explicit usage of APLIC register in store function.
      - Drop APLIC_REG_{GET,SET} macros and introudce APLIC specific funtcions.
      - Ignore write to SOURCECFG_BASE when value is out-of-range.
      - Drop ASSERT(!target_vcpu) inside handler of targer register setting,
        just avoid such writings + debug message.
      - domain_crash() instead of panic() in the case of default case.
      - Drop ASSERT() in APLIC_SOURCE_CFG_BASE case and use domain_crash()
        instead.
    xen/riscv: emulate guest reads from virtual APLIC MMIO:
      - s/regval_to_irqn/regindx_to_irqn.
      - pass to to_vaplic() a domain instead of vintc.
      - add check that load access is aligned.
      - instead of panic() just crash a domain().
      - use 'unsigned int' for local variable offset.
      - Return 0 in the case APLIC_CLRIE_BASE ...APLIC_CLRIE_LAST reading to
        follow AIA spec.
      - Drop explicit usage of physical APLIC registers.
---
---
 xen/arch/riscv/aplic-priv.h         |   2 +
 xen/arch/riscv/aplic.c              |  55 +++++
 xen/arch/riscv/include/asm/aplic.h  |  24 +++
 xen/arch/riscv/include/asm/imsic.h  |  10 +
 xen/arch/riscv/include/asm/vaplic.h |   3 +
 xen/arch/riscv/vaplic.c             | 301 ++++++++++++++++++++++++++++
 6 files changed, 395 insertions(+)

diff --git a/xen/arch/riscv/aplic-priv.h b/xen/arch/riscv/aplic-priv.h
index 1391837f89b9..96bc56dbe585 100644
--- a/xen/arch/riscv/aplic-priv.h
+++ b/xen/arch/riscv/aplic-priv.h
@@ -48,4 +48,6 @@ struct aplic_priv {
  */
 extern unsigned int guest_aplic_num_sources;
 
+uint32_t aplic_msi_target_gen(const struct vcpu *target_vcpu, uint32_t base_val);
+
 #endif /* ASM_RISCV_APLIC_PRIV_H */
diff --git a/xen/arch/riscv/aplic.c b/xen/arch/riscv/aplic.c
index 3681f0669efb..87f2134bc561 100644
--- a/xen/arch/riscv/aplic.c
+++ b/xen/arch/riscv/aplic.c
@@ -16,6 +16,7 @@
 #include <xen/irq.h>
 #include <xen/mm.h>
 #include <xen/sections.h>
+#include <xen/sched.h>
 #include <xen/spinlock.h>
 #include <xen/types.h>
 #include <xen/vmap.h>
@@ -38,6 +39,60 @@ static struct intc_info __ro_after_init aplic_info = {
     .hw_variant = INTC_APLIC,
 };
 
+static unsigned long aplic_hart_field(unsigned long hartid)
+{
+    const struct imsic_config *imsic = imsic_get_config();
+    unsigned int lhxw = imsic->hart_index_bits;
+    unsigned int hhxw = imsic->group_index_bits;
+    unsigned int hhxs =
+        imsic->group_index_shift - APLIC_xMSICFGADDR_PPN_SHIFT * 2;
+    unsigned long tppn =
+        imsic->msi[hartid].base_addr >> APLIC_xMSICFGADDR_PPN_SHIFT;
+    unsigned long group_index =
+        (tppn >> APLIC_xMSICFGADDR_PPN_HHX_SHIFT(hhxs)) &
+        APLIC_xMSICFGADDR_PPN_HHX_MASK(hhxw);
+
+    return (group_index << lhxw) | hartid;
+}
+
+uint32_t aplic_msi_target_gen(const struct vcpu *target_vcpu, uint32_t base_val)
+{
+    unsigned int guest_id = vcpu_guest_file_id(target_vcpu);
+    unsigned long hart_id = cpuid_to_hartid(target_vcpu->processor);
+    unsigned long hart_field = aplic_hart_field(hart_id);
+
+    base_val &= APLIC_TARGET_EIID_MASK;
+    base_val |= MASK_INSR(guest_id, APLIC_TARGET_GUEST_IDX_MASK);
+    base_val |= MASK_INSR(hart_field, APLIC_TARGET_HART_IDX_MASK);
+
+    return base_val;
+}
+
+uint32_t aplic_hw_read_reg(unsigned int offset, uint32_t mask)
+{
+    unsigned long flags;
+    uint32_t val;
+
+    ASSERT((offset < aplic.size) && IS_ALIGNED(offset, sizeof(uint32_t)));
+
+    spin_lock_irqsave(&aplic.lock, flags);
+    val = readl((volatile void __iomem *)aplic.regs + offset) & mask;
+    spin_unlock_irqrestore(&aplic.lock, flags);
+
+    return val;
+}
+
+void aplic_hw_write_reg(unsigned int offset, uint32_t value)
+{
+    unsigned long flags;
+
+    ASSERT((offset < aplic.size) && IS_ALIGNED(offset, sizeof(uint32_t)));
+
+    spin_lock_irqsave(&aplic.lock, flags);
+    writel(value, (volatile void __iomem *)aplic.regs + offset);
+    spin_unlock_irqrestore(&aplic.lock, flags);
+}
+
 static void __init aplic_init_hw_interrupts(void)
 {
     unsigned int i;
diff --git a/xen/arch/riscv/include/asm/aplic.h b/xen/arch/riscv/include/asm/aplic.h
index f22622b9a23f..4ae5fb8f26d1 100644
--- a/xen/arch/riscv/include/asm/aplic.h
+++ b/xen/arch/riscv/include/asm/aplic.h
@@ -28,6 +28,8 @@
 #define APLIC_DOMAINCFG_BE      BIT(0, U)
 
 /* sourcecfg register fields */
+#define APLIC_SOURCECFG_D       BIT(10, U)
+
 #define APLIC_SOURCECFG_SM_INACTIVE     0x0
 #define APLIC_SOURCECFG_SM_DETACH       0x1
 #define APLIC_SOURCECFG_SM_EDGE_RISE    0x4
@@ -38,6 +40,16 @@
 /* target register fields */
 #define APLIC_TARGET_HART_IDX_SHIFT 18
 #define APLIC_TARGET_EIID_MASK      0x7ff
+#define APLIC_TARGET_HART_IDX_MASK  0xfffc0000
+#define APLIC_TARGET_GUEST_IDX_MASK 0x3f000
+
+/* xmsicfgaddr/h register fields */
+#define APLIC_xMSICFGADDR_PPN_SHIFT IMSIC_MMIO_PAGE_SHIFT
+
+#define APLIC_xMSICFGADDR_PPN_HHX_MASK(hhxw) \
+    (BIT(hhxw, UL) - 1)
+#define APLIC_xMSICFGADDR_PPN_HHX_SHIFT(hhxs) \
+    ((hhxs) + APLIC_xMSICFGADDR_PPN_SHIFT)
 
 #define APLIC_DOMAINCFG         0x0000
 #define APLIC_SOURCECFG_BASE    0x0004
@@ -77,6 +89,15 @@
 #define APLIC_SIZE(nr_cpus)     (APLIC_MIN_SIZE + \
                                  APLIC_SIZE_ALIGN(APLIC_IDC_SIZE * (nr_cpus)))
 
+/*
+ * Using setip is fine here, as all SET* and CLR* register groups consist of 32
+ * registers and therefore have identical sizes.
+ *
+ * Lowest 2 bits are always zero for SET* and CLR* registers.
+ */
+#define APLIC_SETCLR_OFFSET_MASK \
+    (sizeof_field(struct aplic_regs, setip) - sizeof(uint32_t))
+
 struct aplic_regs {
     uint32_t domaincfg;         /* 0x0000 */
     uint32_t sourcecfg[1023];   /* 0x0004 */
@@ -120,4 +141,7 @@ struct aplic_regs {
     uint32_t target[1023];      /* 0x3008 */
 };
 
+uint32_t aplic_hw_read_reg(unsigned int offset, uint32_t mask);
+void aplic_hw_write_reg(unsigned int offset, uint32_t value);
+
 #endif /* ASM_RISCV_APLIC_H */
diff --git a/xen/arch/riscv/include/asm/imsic.h b/xen/arch/riscv/include/asm/imsic.h
index e1ec3d03c4e9..612f503b5799 100644
--- a/xen/arch/riscv/include/asm/imsic.h
+++ b/xen/arch/riscv/include/asm/imsic.h
@@ -40,6 +40,16 @@ struct imsic_config {
     /* Base address */
     paddr_t base_addr;
 
+    /*
+     * MSI Target Address Scheme
+     *
+     * XLEN-1                                                12     0
+     * |                                                     |     |
+     * -------------------------------------------------------------
+     * |xxxxxx|Group Index|xxxxxxxxxxx|HART Index|Guest Index|  0  |
+     * -------------------------------------------------------------
+     */
+
     /* Bits representing Guest index, HART index, and Group index */
     unsigned int guest_index_bits;
     unsigned int hart_index_bits;
diff --git a/xen/arch/riscv/include/asm/vaplic.h b/xen/arch/riscv/include/asm/vaplic.h
index 96080bfbc23b..7bf9247f4eae 100644
--- a/xen/arch/riscv/include/asm/vaplic.h
+++ b/xen/arch/riscv/include/asm/vaplic.h
@@ -26,6 +26,9 @@ struct vaplic_regs {
 struct vaplic {
     struct vintc vintc;
     struct vaplic_regs regs;
+
+    paddr_t regs_start;
+    unsigned int regs_size;
 };
 
 int domain_vaplic_init(struct domain *d);
diff --git a/xen/arch/riscv/vaplic.c b/xen/arch/riscv/vaplic.c
index 449240c5cd23..03240730e344 100644
--- a/xen/arch/riscv/vaplic.c
+++ b/xen/arch/riscv/vaplic.c
@@ -17,6 +17,7 @@
 #include <asm/aia.h>
 #include <asm/imsic.h>
 #include <asm/intc.h>
+#include <asm/mmio.h>
 #include <asm/vaplic.h>
 
 #include "aplic-priv.h"
@@ -27,6 +28,256 @@ unsigned int __ro_after_init guest_aplic_num_sources;
 
 #define FDT_VAPLIC_INT_CELLS 2
 
+#define AUTH_IRQ_BIT(d, irqn) ( \
+    ((irqn) < (d)->arch.vintc->nr_virqs) && \
+    test_bit(irqn, (d)->arch.vintc->used_irqs) )
+
+/*
+ * Convert a byte offset (within a SETIP/CLRIP/SETIE/CLRIE register group) to
+ * a 32-bit word index into the allocated_irqs bitmap.  Each word covers 32
+ * interrupt sources.  For SOURCECFG and TARGET groups the same division also
+ * yields the interrupt number directly, because those arrays store one 32-bit
+ * register per source.
+ */
+#define regoffset_to_word_idx(reg_val) ((reg_val) / sizeof(uint32_t))
+
+static inline uint32_t generate_auth_mask(const struct domain *d,
+                                          unsigned int word_idx)
+{
+    unsigned int first_bit = word_idx * sizeof(uint32_t) * BITS_PER_BYTE;
+
+    if ( word_idx >= DIV_ROUND_UP(d->arch.vintc->nr_virqs,
+                                  sizeof(uint32_t) * BITS_PER_BYTE) )
+    {
+        dprintk(XENLOG_DEBUG, "incorrect word_idx(%u) is passed\n", word_idx);
+
+        return 0U;
+    }
+
+    return (uint32_t)(d->arch.vintc->used_irqs[first_bit / BITS_PER_LONG] >>
+                      (first_bit % BITS_PER_LONG));
+}
+
+static int cf_check vaplic_emulate_load(const struct vcpu *v,
+                                        const unsigned long addr,
+                                        uint32_t *out)
+{
+    const struct domain *d = v->domain;
+    const struct vaplic *vaplic = to_vaplic(d);
+    const unsigned int offset = addr & APLIC_REG_OFFSET_MASK;
+    uint32_t auth_mask;
+    unsigned int i;
+
+    switch ( offset )
+    {
+    case APLIC_DOMAINCFG:
+        *out = vaplic->regs.domaincfg;
+
+        return 0;
+
+    case APLIC_SETIPNUM:
+    case APLIC_SETIPNUM_LE:
+    case APLIC_CLRIPNUM:
+    case APLIC_SETIENUM:
+    case APLIC_CLRIENUM:
+    case APLIC_CLRIE_BASE ... APLIC_CLRIE_LAST:
+        /*
+         * Based on the RISC-V AIA spec a read of these registers
+         * always returns zero
+         */
+        *out = 0;
+
+        return 0;
+
+    case APLIC_SETIP_BASE ... APLIC_SETIP_LAST:
+    case APLIC_CLRIP_BASE ... APLIC_CLRIP_LAST:
+    case APLIC_SETIE_BASE ... APLIC_SETIE_LAST:
+        i = regoffset_to_word_idx(offset & APLIC_SETCLR_OFFSET_MASK);
+        auth_mask = generate_auth_mask(d, i);
+
+        break;
+
+    case APLIC_TARGET_BASE ... APLIC_TARGET_LAST:
+        /*
+         * As target registers start from 1:
+         *  0x3000 genmsi
+         *  0x3004 target[1]
+         *  0x3008 target[2]
+         *   ...
+         *  0x3FFC target[1023]
+         * It is necessary to calculate an interrupt number by subtracting
+         * APLIC_GENMSI instead of APLIC_TARGET_BASE.
+         */
+        i = regoffset_to_word_idx(offset - APLIC_GENMSI);
+
+        if ( !AUTH_IRQ_BIT(d, i) )
+        {
+            *out = 0;
+
+            return 0;
+        }
+
+        auth_mask = ~0U;
+
+        break;
+
+    default:
+        gdprintk(XENLOG_WARNING, "Unhandled APLIC read at offset %#x\n",
+                 offset);
+
+        return -EINVAL;
+    }
+
+    *out = aplic_hw_read_reg(offset, auth_mask);
+
+    return 0;
+}
+
+static int cf_check vaplic_emulate_store(const struct vcpu *v,
+                                         unsigned long addr, uint32_t value)
+{
+    int rc = -EINVAL;
+    const struct domain *d = v->domain;
+    unsigned int offset = addr & APLIC_REG_OFFSET_MASK;
+
+    switch ( offset )
+    {
+    case APLIC_SETIP_BASE ... APLIC_SETIP_LAST:
+    case APLIC_CLRIP_BASE ... APLIC_CLRIP_LAST:
+    case APLIC_SETIE_BASE ... APLIC_SETIE_LAST:
+    case APLIC_CLRIE_BASE ... APLIC_CLRIE_LAST:
+    {
+        unsigned int word_idx =
+            regoffset_to_word_idx(offset & APLIC_SETCLR_OFFSET_MASK);
+
+        value &= generate_auth_mask(d, word_idx);
+
+        break;
+    }
+
+    case APLIC_SOURCECFG_BASE ... APLIC_SOURCECFG_LAST:
+        if ( value & APLIC_SOURCECFG_D )
+        {
+            rc = -EOPNOTSUPP;
+
+            dprintk(XENLOG_ERR, "APLIC_SOURCECFG_D isn't supported\n");
+
+            goto fail;
+        }
+
+        /*
+         * As sourcecfg register starts from 1:
+         *   0x0000 domaincfg
+         *   0x0004 sourcecfg[1]
+         *   0x0008 sourcecfg[2]
+         *    ...
+         *   0x0FFC sourcecfg[1023]
+         * It is necessary to calculate an interrupt number by subtracting
+         * APLIC_DOMAINCFG instead of APLIC_SOURCECFG_BASE.
+         */
+        if ( !AUTH_IRQ_BIT(d, regoffset_to_word_idx(offset - APLIC_DOMAINCFG)) )
+            /* Interrupt not enabled, ignore it */
+            return 0;
+
+        if ( value > APLIC_SOURCECFG_SM_LEVEL_LOW )
+        {
+            gdprintk(XENLOG_ERR,
+                     "value(%u) is incorrect for sourcecfg register\n", value);
+
+            return 0;
+        }
+
+        break;
+
+    case APLIC_TARGET_BASE ... APLIC_TARGET_LAST:
+    {
+        struct vcpu *target_vcpu = NULL;
+        unsigned int hart_idx = value >> APLIC_TARGET_HART_IDX_SHIFT;
+
+        /*
+         * Look at vaplic_emulate_load() for explanation why
+         * APLIC_GENMSI is subtracted.
+         */
+        if ( !AUTH_IRQ_BIT(d, regoffset_to_word_idx(offset - APLIC_GENMSI)) )
+            /* Interrupt not enabled, ignore it */
+            return 0;
+
+        if ( hart_idx < v->domain->max_vcpus )
+            target_vcpu = v->domain->vcpu[hart_idx];
+
+        if ( !target_vcpu )
+        {
+            dprintk(XENLOG_ERR, "Invalid vCPU id in target register\n");
+
+            /* Ignore such writings */
+            return 0;
+        }
+
+        value = aplic_msi_target_gen(target_vcpu, value);
+
+        break;
+    }
+
+    case APLIC_SETIPNUM:
+    case APLIC_SETIPNUM_LE:
+    case APLIC_CLRIPNUM:
+    case APLIC_SETIENUM:
+    case APLIC_CLRIENUM:
+        if ( !value || !AUTH_IRQ_BIT(d, value) )
+            return 0;
+
+        break;
+
+    case APLIC_DOMAINCFG:
+    {
+        struct vaplic *vaplic = to_vaplic(v->domain);
+
+        /*
+         * The domaincfg register has this format:
+         * bits 31:24 read-only 0x80
+         * bit 8      IE
+         * bit 7      read-only 0
+         * bit 2      DM (WARL)
+         * bit 0      BE (WARL)
+         *
+         * The most interesting bit for us is IE(Interrupt Enable) bit.
+         * At the moment, at least, Linux doesn't use domaincfg.IE bit to
+         * disable interrupts globally, but if one day someone will use it
+         * then extra actions should be done.
+         *
+         * Only DM (bit 2) and IE (bit 8) are writable here. They are assigned
+         * (not OR-ed) so that a write of 0 can also clear them (WARL), and the
+         * read-only high byte (0x80) is always kept set on read-back.
+         */
+        if ( value & ~(APLIC_DOMAINCFG_RO | APLIC_DOMAINCFG_DM |
+                       APLIC_DOMAINCFG_IE) )
+            printk_once("%s: Ignore writes to non-writable domaincfg bits as "
+                        "they are set by aplic during initialization in Xen\n",
+                        __func__);
+
+        vaplic->regs.domaincfg = APLIC_DOMAINCFG_RO |
+                                 (value & (APLIC_DOMAINCFG_DM |
+                                           APLIC_DOMAINCFG_IE));
+
+        return 0;
+    }
+
+    default:
+        goto fail;
+    }
+
+    aplic_hw_write_reg(offset, value);
+
+    return 0;
+
+ fail:
+    gdprintk(XENLOG_WARNING,
+             "Unhandled APLIC write at offset %#x (value %#x)\n", offset,
+             value);
+
+    return rc;
+}
+
 static int cf_check vaplic_init(struct vcpu *v)
 {
     return vcpu_imsic_init(v);
@@ -105,6 +356,50 @@ static const struct vintc_init_ops __initconstrel init_ops = {
     .make_domu_dt_node = vaplic_make_domu_dt_node,
 };
 
+static enum io_state cf_check vaplic_mmio_read(struct vcpu *v, mmio_info_t *info,
+                                               register_t *r)
+{
+    uint32_t data = 0;
+
+    if ( info->len != sizeof(uint32_t) ||
+         !IS_ALIGNED(info->gpa, sizeof(uint32_t)) )
+    {
+        gdprintk(XENLOG_DEBUG,
+                 "VAPLIC: unaligned/wrong-width read gpa=%"PRIpaddr" len=%u\n",
+                 info->gpa, info->len);
+        return IO_ABORT;
+    }
+
+    if ( vaplic_emulate_load(v, info->gpa, &data) < 0 )
+        return IO_ABORT;
+
+    *r = data;
+    return IO_HANDLED;
+}
+
+static enum io_state cf_check vaplic_mmio_write(struct vcpu *v, mmio_info_t *info,
+                                                register_t r)
+{
+    if ( info->len != sizeof(uint32_t) ||
+         !IS_ALIGNED(info->gpa, sizeof(uint32_t)) )
+    {
+        gdprintk(XENLOG_DEBUG,
+                 "VAPLIC: unaligned/wrong-width write gpa=%"PRIpaddr" len=%u\n",
+                 info->gpa, info->len);
+        return IO_ABORT;
+    }
+
+    if ( vaplic_emulate_store(v, info->gpa, r) < 0 )
+        return IO_ABORT;
+
+    return IO_HANDLED;
+}
+
+static const struct mmio_handler_ops vaplic_mmio_ops = {
+    .read  = vaplic_mmio_read,
+    .write = vaplic_mmio_write,
+};
+
 static const struct vintc_ops vintc_ops = {
     .vcpu_init = vaplic_init,
     .vcpu_deinit = vaplic_deinit,
@@ -132,6 +427,12 @@ int domain_vaplic_init(struct domain *d)
      */
     d->arch.vintc->nr_virqs = guest_aplic_num_sources + 1;
 
+    vaplic->regs_start = GUEST_APLIC_S_BASE;
+    vaplic->regs_size = APLIC_SIZE(d->max_vcpus);
+
+    register_mmio_handler(d, &vaplic_mmio_ops,
+                          vaplic->regs_start, vaplic->regs_size);
+
     return 0;
 }
 
-- 
2.54.0

Re: [PATCH v1 05/17] xen/riscv: implement virtual APLIC MMIO emulation

[Jan Beulich]

On 20.07.2026 18:02, Oleksii Kurochko wrote:
> Guests running under Xen program interrupt routing by writing to APLIC
> MMIO registers. Xen must intercept these accesses to enforce interrupt
> isolation between domains and to translate guest routing intent into the
> underlying physical MSI topology.
> 
> Writes are gated by the domain's authorised interrupt bitmap so that a
> guest cannot affect interrupts it does not own. TARGET register writes
> additionally require translation of the hart and IMSIC guest-file
> indices from virtual to physical, as the APLIC uses these fields
> directly to compute the MSI delivery address.
> 
> Delegation (APLIC_SOURCECFG_D) is not yet supported.
> 
> Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
> Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech> # vaplic_mmio_{read,write}

For this tag to have any meaning, it should move ahead of the --- above;
the explanations ...

> The downstream changes related to `vaplic_mmio_{read,write}` were originally
> in a separate patch (which was reviewed by Baptiste). However, before
> upstreaming, it was decided to merge them into the current patch.
> I added `Reviewed-by: Baptiste` in this form for now, but Baptiste will
> probably review the remaining changes as well.
> Once that happens, I'll simply move the `Reviewed-by` tag up and
> remove the `#`.

... here rather explain the restriction on the R-b, not its odd placement.

> ---
> Changes in v3:

As this looks to be recurring - please get versioning of your series right.
The series is supposedly v1, but here you give the impression of it being
v3. If there really was an earlier v2 posting, why isn't the entire series
here v3?

> --- a/xen/arch/riscv/aplic-priv.h
> +++ b/xen/arch/riscv/aplic-priv.h
> @@ -48,4 +48,6 @@ struct aplic_priv {
>   */
>  extern unsigned int guest_aplic_num_sources;
>  
> +uint32_t aplic_msi_target_gen(const struct vcpu *target_vcpu, uint32_t base_val);

PLease can you, before submitting, self-review your patches? I'm really
getting tired of having to repeatedly point out basic style issues, like
the overlong line here.

> @@ -38,6 +39,60 @@ static struct intc_info __ro_after_init aplic_info = {
>      .hw_variant = INTC_APLIC,
>  };
>  
> +static unsigned long aplic_hart_field(unsigned long hartid)
> +{
> +    const struct imsic_config *imsic = imsic_get_config();
> +    unsigned int lhxw = imsic->hart_index_bits;
> +    unsigned int hhxw = imsic->group_index_bits;

It extends to the other local variables here, but I'll use these two to
try to make my point: I'm struggling to associate the names with the
values they are set to. Likely "hxw" is an abbreviation of hart index
width, but (a) what's the leading 'l' then and (b) why is there no 'g'
in "hhxw"? By using hard to grasp names, you make it hard to actually
understand the subsequent expressions, in particular ...

> +    unsigned int hhxs =
> +        imsic->group_index_shift - APLIC_xMSICFGADDR_PPN_SHIFT * 2;
> +    unsigned long tppn =
> +        imsic->msi[hartid].base_addr >> APLIC_xMSICFGADDR_PPN_SHIFT;
> +    unsigned long group_index =
> +        (tppn >> APLIC_xMSICFGADDR_PPN_HHX_SHIFT(hhxs)) &
> +        APLIC_xMSICFGADDR_PPN_HHX_MASK(hhxw);
> +
> +    return (group_index << lhxw) | hartid;

... these last two. As it stands, they may be easier to understand if
you didn't have the local variables at all, despite them then getting
textually longer.

> +uint32_t aplic_msi_target_gen(const struct vcpu *target_vcpu, uint32_t base_val)

Same issue as with the decl.

> +{
> +    unsigned int guest_id = vcpu_guest_file_id(target_vcpu);
> +    unsigned long hart_id = cpuid_to_hartid(target_vcpu->processor);
> +    unsigned long hart_field = aplic_hart_field(hart_id);
> +
> +    base_val &= APLIC_TARGET_EIID_MASK;
> +    base_val |= MASK_INSR(guest_id, APLIC_TARGET_GUEST_IDX_MASK);
> +    base_val |= MASK_INSR(hart_field, APLIC_TARGET_HART_IDX_MASK);
> +
> +    return base_val;
> +}
> +
> +uint32_t aplic_hw_read_reg(unsigned int offset, uint32_t mask)
> +{
> +    unsigned long flags;
> +    uint32_t val;
> +
> +    ASSERT((offset < aplic.size) && IS_ALIGNED(offset, sizeof(uint32_t)));
> +
> +    spin_lock_irqsave(&aplic.lock, flags);
> +    val = readl((volatile void __iomem *)aplic.regs + offset) & mask;

Wouldn't this applying of a mask better be done in those callers which
actually need it? It's not the least the asymmetry with ...

> +    spin_unlock_irqrestore(&aplic.lock, flags);
> +
> +    return val;
> +}
> +
> +void aplic_hw_write_reg(unsigned int offset, uint32_t value)

... this which I consider unhelpful.

> --- a/xen/arch/riscv/include/asm/aplic.h
> +++ b/xen/arch/riscv/include/asm/aplic.h
> @@ -28,6 +28,8 @@
>  #define APLIC_DOMAINCFG_BE      BIT(0, U)
>  
>  /* sourcecfg register fields */
> +#define APLIC_SOURCECFG_D       BIT(10, U)

As to the comment - this indeed looks to be a field, but ...

>  #define APLIC_SOURCECFG_SM_INACTIVE     0x0
>  #define APLIC_SOURCECFG_SM_DETACH       0x1
>  #define APLIC_SOURCECFG_SM_EDGE_RISE    0x4

... these look to be values of some other field which isn't described. Please
may I (again) ask that definitions are their commentary at the very least not
misguide readers?

> --- a/xen/arch/riscv/include/asm/imsic.h
> +++ b/xen/arch/riscv/include/asm/imsic.h
> @@ -40,6 +40,16 @@ struct imsic_config {
>      /* Base address */
>      paddr_t base_addr;
>  
> +    /*
> +     * MSI Target Address Scheme
> +     *
> +     * XLEN-1                                                12     0
> +     * |                                                     |     |
> +     * -------------------------------------------------------------
> +     * |xxxxxx|Group Index|xxxxxxxxxxx|HART Index|Guest Index|  0  |
> +     * -------------------------------------------------------------
> +     */

And the xxx-es in here mean what exactly? Don't care? Some other, unrelated
values? Yet something else?

> --- a/xen/arch/riscv/vaplic.c
> +++ b/xen/arch/riscv/vaplic.c
> @@ -17,6 +17,7 @@
>  #include <asm/aia.h>
>  #include <asm/imsic.h>
>  #include <asm/intc.h>
> +#include <asm/mmio.h>
>  #include <asm/vaplic.h>
>  
>  #include "aplic-priv.h"
> @@ -27,6 +28,256 @@ unsigned int __ro_after_init guest_aplic_num_sources;
>  
>  #define FDT_VAPLIC_INT_CELLS 2
>  
> +#define AUTH_IRQ_BIT(d, irqn) ( \
> +    ((irqn) < (d)->arch.vintc->nr_virqs) && \
> +    test_bit(irqn, (d)->arch.vintc->used_irqs) )

Nit: Indentation.

> +/*
> + * Convert a byte offset (within a SETIP/CLRIP/SETIE/CLRIE register group) to
> + * a 32-bit word index into the allocated_irqs bitmap.  Each word covers 32
> + * interrupt sources.  For SOURCECFG and TARGET groups the same division also
> + * yields the interrupt number directly, because those arrays store one 32-bit
> + * register per source.
> + */
> +#define regoffset_to_word_idx(reg_val) ((reg_val) / sizeof(uint32_t))
> +
> +static inline uint32_t generate_auth_mask(const struct domain *d,
> +                                          unsigned int word_idx)
> +{
> +    unsigned int first_bit = word_idx * sizeof(uint32_t) * BITS_PER_BYTE;
> +
> +    if ( word_idx >= DIV_ROUND_UP(d->arch.vintc->nr_virqs,
> +                                  sizeof(uint32_t) * BITS_PER_BYTE) )
> +    {
> +        dprintk(XENLOG_DEBUG, "incorrect word_idx(%u) is passed\n", word_idx);

Is this really meant to stay?

> +        return 0U;

The U suffix is mainly (even if only slightly) obfuscating things, I think.

> +    }
> +
> +    return (uint32_t)(d->arch.vintc->used_irqs[first_bit / BITS_PER_LONG] >>
> +                      (first_bit % BITS_PER_LONG));

I don't quite understand the need for the cast.

> +static int cf_check vaplic_emulate_load(const struct vcpu *v,

Why the cf_check (also for the store counterpart)?

> +static int cf_check vaplic_emulate_store(const struct vcpu *v,
> +                                         unsigned long addr, uint32_t value)
> +{
> +    int rc = -EINVAL;
> +    const struct domain *d = v->domain;
> +    unsigned int offset = addr & APLIC_REG_OFFSET_MASK;
> +
> +    switch ( offset )
> +    {
> +    case APLIC_SETIP_BASE ... APLIC_SETIP_LAST:
> +    case APLIC_CLRIP_BASE ... APLIC_CLRIP_LAST:
> +    case APLIC_SETIE_BASE ... APLIC_SETIE_LAST:
> +    case APLIC_CLRIE_BASE ... APLIC_CLRIE_LAST:
> +    {
> +        unsigned int word_idx =
> +            regoffset_to_word_idx(offset & APLIC_SETCLR_OFFSET_MASK);
> +
> +        value &= generate_auth_mask(d, word_idx);
> +
> +        break;
> +    }
> +
> +    case APLIC_SOURCECFG_BASE ... APLIC_SOURCECFG_LAST:
> +        if ( value & APLIC_SOURCECFG_D )
> +        {
> +            rc = -EOPNOTSUPP;
> +
> +            dprintk(XENLOG_ERR, "APLIC_SOURCECFG_D isn't supported\n");
> +
> +            goto fail;
> +        }
> +
> +        /*
> +         * As sourcecfg register starts from 1:
> +         *   0x0000 domaincfg
> +         *   0x0004 sourcecfg[1]
> +         *   0x0008 sourcecfg[2]
> +         *    ...
> +         *   0x0FFC sourcecfg[1023]
> +         * It is necessary to calculate an interrupt number by subtracting
> +         * APLIC_DOMAINCFG instead of APLIC_SOURCECFG_BASE.
> +         */
> +        if ( !AUTH_IRQ_BIT(d, regoffset_to_word_idx(offset - APLIC_DOMAINCFG)) )
> +            /* Interrupt not enabled, ignore it */
> +            return 0;
> +
> +        if ( value > APLIC_SOURCECFG_SM_LEVEL_LOW )
> +        {
> +            gdprintk(XENLOG_ERR,
> +                     "value(%u) is incorrect for sourcecfg register\n", value);
> +
> +            return 0;
> +        }
> +
> +        break;
> +
> +    case APLIC_TARGET_BASE ... APLIC_TARGET_LAST:
> +    {
> +        struct vcpu *target_vcpu = NULL;
> +        unsigned int hart_idx = value >> APLIC_TARGET_HART_IDX_SHIFT;
> +
> +        /*
> +         * Look at vaplic_emulate_load() for explanation why
> +         * APLIC_GENMSI is subtracted.
> +         */
> +        if ( !AUTH_IRQ_BIT(d, regoffset_to_word_idx(offset - APLIC_GENMSI)) )
> +            /* Interrupt not enabled, ignore it */
> +            return 0;
> +
> +        if ( hart_idx < v->domain->max_vcpus )

You have d as a local variable.

> +            target_vcpu = v->domain->vcpu[hart_idx];

Use domain_vcpu()?

> +        if ( !target_vcpu )
> +        {
> +            dprintk(XENLOG_ERR, "Invalid vCPU id in target register\n");
> +
> +            /* Ignore such writings */
> +            return 0;
> +        }
> +
> +        value = aplic_msi_target_gen(target_vcpu, value);
> +
> +        break;
> +    }
> +
> +    case APLIC_SETIPNUM:
> +    case APLIC_SETIPNUM_LE:
> +    case APLIC_CLRIPNUM:
> +    case APLIC_SETIENUM:
> +    case APLIC_CLRIENUM:
> +        if ( !value || !AUTH_IRQ_BIT(d, value) )
> +            return 0;
> +
> +        break;
> +
> +    case APLIC_DOMAINCFG:
> +    {
> +        struct vaplic *vaplic = to_vaplic(v->domain);
> +
> +        /*
> +         * The domaincfg register has this format:
> +         * bits 31:24 read-only 0x80
> +         * bit 8      IE
> +         * bit 7      read-only 0
> +         * bit 2      DM (WARL)
> +         * bit 0      BE (WARL)
> +         *
> +         * The most interesting bit for us is IE(Interrupt Enable) bit.
> +         * At the moment, at least, Linux doesn't use domaincfg.IE bit to
> +         * disable interrupts globally, but if one day someone will use it
> +         * then extra actions should be done.
> +         *
> +         * Only DM (bit 2) and IE (bit 8) are writable here. They are assigned
> +         * (not OR-ed) so that a write of 0 can also clear them (WARL), and the
> +         * read-only high byte (0x80) is always kept set on read-back.
> +         */
> +        if ( value & ~(APLIC_DOMAINCFG_RO | APLIC_DOMAINCFG_DM |
> +                       APLIC_DOMAINCFG_IE) )
> +            printk_once("%s: Ignore writes to non-writable domaincfg bits as "
> +                        "they are set by aplic during initialization in Xen\n",
> +                        __func__);
> +
> +        vaplic->regs.domaincfg = APLIC_DOMAINCFG_RO |
> +                                 (value & (APLIC_DOMAINCFG_DM |
> +                                           APLIC_DOMAINCFG_IE));
> +
> +        return 0;
> +    }
> +
> +    default:
> +        goto fail;

Instead of this goto, I think you simply want to move the label here.
That'll also make the function more similar to its load counterpart.

> @@ -105,6 +356,50 @@ static const struct vintc_init_ops __initconstrel init_ops = {
>      .make_domu_dt_node = vaplic_make_domu_dt_node,
>  };
>  
> +static enum io_state cf_check vaplic_mmio_read(struct vcpu *v, mmio_info_t *info,
> +                                               register_t *r)
> +{
> +    uint32_t data = 0;
> +
> +    if ( info->len != sizeof(uint32_t) ||
> +         !IS_ALIGNED(info->gpa, sizeof(uint32_t)) )
> +    {
> +        gdprintk(XENLOG_DEBUG,
> +                 "VAPLIC: unaligned/wrong-width read gpa=%"PRIpaddr" len=%u\n",
> +                 info->gpa, info->len);

You have v passed in here, but you'd log current. If passing in v is
necessary (i.e. here or elsewhere it may be other than current), then you
need to either ASSERT(v == current) at the top of the funciton or otherwise
handle v != current correctly.

> +        return IO_ABORT;
> +    }
> +
> +    if ( vaplic_emulate_load(v, info->gpa, &data) < 0 )

If all you care about is a boolean result, why not make the function return
bool?

> +        return IO_ABORT;
> +
> +    *r = data;
> +    return IO_HANDLED;

Nit: Blank line please ahead of <etc>.

> +static enum io_state cf_check vaplic_mmio_write(struct vcpu *v, mmio_info_t *info,
> +                                                register_t r)
> +{
> +    if ( info->len != sizeof(uint32_t) ||
> +         !IS_ALIGNED(info->gpa, sizeof(uint32_t)) )
> +    {
> +        gdprintk(XENLOG_DEBUG,
> +                 "VAPLIC: unaligned/wrong-width write gpa=%"PRIpaddr" len=%u\n",
> +                 info->gpa, info->len);
> +        return IO_ABORT;
> +    }
> +
> +    if ( vaplic_emulate_store(v, info->gpa, r) < 0 )

Same here.

Jan

Re: [PATCH v1 05/17] xen/riscv: implement virtual APLIC MMIO emulation

[Oleksii Kurochko]

On 8/6/26 4:28 PM, Jan Beulich wrote:
> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>> Guests running under Xen program interrupt routing by writing to APLIC
>> MMIO registers. Xen must intercept these accesses to enforce interrupt
>> isolation between domains and to translate guest routing intent into the
>> underlying physical MSI topology.
>>
>> Writes are gated by the domain's authorised interrupt bitmap so that a
>> guest cannot affect interrupts it does not own. TARGET register writes
>> additionally require translation of the hart and IMSIC guest-file
>> indices from virtual to physical, as the APLIC uses these fields
>> directly to compute the MSI delivery address.
>>
>> Delegation (APLIC_SOURCECFG_D) is not yet supported.
>>
>> Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>> ---
>> Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech> # vaplic_mmio_{read,write}
> 
> For this tag to have any meaning, it should move ahead of the --- above;
> the explanations ...
> 
>> The downstream changes related to `vaplic_mmio_{read,write}` were originally
>> in a separate patch (which was reviewed by Baptiste). However, before
>> upstreaming, it was decided to merge them into the current patch.
>> I added `Reviewed-by: Baptiste` in this form for now, but Baptiste will
>> probably review the remaining changes as well.
>> Once that happens, I'll simply move the `Reviewed-by` tag up and
>> remove the `#`.
> 
> ... here rather explain the restriction on the R-b, not its odd placement.
> 
>> ---
>> Changes in v3:
> 
> As this looks to be recurring - please get versioning of your series right.
> The series is supposedly v1, but here you give the impression of it being
> v3. If there really was an earlier v2 posting, why isn't the entire series
> here v3?

It is v3 before before it was a part of another patch series connected 
to dom0less config enablement.

Would it be better to just write in "Change in v3" that it is moved from 
another patch series + link to that patch series? Or it will be enough 
just to drop "Changes in v2 and v1" and just start from v1?

> 
>> --- a/xen/arch/riscv/aplic-priv.h
>> +++ b/xen/arch/riscv/aplic-priv.h
>> @@ -48,4 +48,6 @@ struct aplic_priv {
>>    */
>>   extern unsigned int guest_aplic_num_sources;
>>   
>> +uint32_t aplic_msi_target_gen(const struct vcpu *target_vcpu, uint32_t base_val);
> 
> PLease can you, before submitting, self-review your patches? I'm really
> getting tired of having to repeatedly point out basic style issues, like
> the overlong line here.

Sorry for that, I will write an extra checker for such cases to not miss 
them.

> 
>> @@ -38,6 +39,60 @@ static struct intc_info __ro_after_init aplic_info = {
>>       .hw_variant = INTC_APLIC,
>>   };
>>   
>> +static unsigned long aplic_hart_field(unsigned long hartid)
>> +{
>> +    const struct imsic_config *imsic = imsic_get_config();
>> +    unsigned int lhxw = imsic->hart_index_bits;
>> +    unsigned int hhxw = imsic->group_index_bits;
> 
> It extends to the other local variables here, but I'll use these two to
> try to make my point: I'm struggling to associate the names with the
> values they are set to. Likely "hxw" is an abbreviation of hart index
> width, but (a) what's the leading 'l' then and (b) why is there no 'g'
> in "hhxw"? By using hard to grasp names, you make it hard to actually
> understand the subsequent expressions, in particular ...

The names it taken directly from AIA spec:

The use of this value and fields HHXS (High Hart Index Shift), LHXS (Low 
Hart Index Shift), HHXW (High Hart Index Width), and LHXW (Low Hart 
Index Width) for determining target addresses for MSIs is described 
later, in Section 4.9.1.

The AIA specification interprets the machine-level hart index as a 
combination of the **group index** (`g`) and the **hart index within the 
group** (`h`), according to the following formulas:

```
(1) g = (machine-level hart index >> LHXW) & (2^HHXW − 1)
(2) h = machine-level hart index & (2^LHXW − 1)
```

(In our case, the machine-level hart index is equal to `mhartid`, i.e. 
the hart index.)

For systems that use IMSIC groups, the IMSIC address layout is defined 
by the following parameters:

* `lhxw` (Low Hart Index Width, or *k*): the number of bits used for the 
hart number within a group.
* `hhxw` (High Hart Index Width, or *j*): the number of bits used for 
the group number.
* `hhxs` (High Hart Index Shift): the bit offset of the combined 
hart/group index field within the physical address.

To extract the group index, we first shift the address by `hhxs` so that 
the group index bits are aligned, and then apply a mask derived from 
`hhxw` to isolate those bits.

The hardware performs the same operation to extract the hart index from 
the MSI address. However, in our case we already know which hart should 
receive the interrupt (`hartid`), so there is no need to extract the 
hart index from the base address. We only need to recover the group 
index and combine it with `hartid` to construct the value expected by 
the `target` register.

> 
>> +    unsigned int hhxs =
>> +        imsic->group_index_shift - APLIC_xMSICFGADDR_PPN_SHIFT * 2;
>> +    unsigned long tppn =
>> +        imsic->msi[hartid].base_addr >> APLIC_xMSICFGADDR_PPN_SHIFT;
>> +    unsigned long group_index =
>> +        (tppn >> APLIC_xMSICFGADDR_PPN_HHX_SHIFT(hhxs)) &
>> +        APLIC_xMSICFGADDR_PPN_HHX_MASK(hhxw);
>> +
>> +    return (group_index << lhxw) | hartid;
> 
> ... these last two. As it stands, they may be easier to understand if
> you didn't have the local variables at all, despite them then getting
> textually longer.

With the explanation above, do the variable names make sense?

To be closer to AIA spec I think it would be better to rename 
group_index to g and hart_id to h. Does it make sense to you?

>> +
>> +uint32_t aplic_hw_read_reg(unsigned int offset, uint32_t mask)
>> +{
>> +    unsigned long flags;
>> +    uint32_t val;
>> +
>> +    ASSERT((offset < aplic.size) && IS_ALIGNED(offset, sizeof(uint32_t)));
>> +
>> +    spin_lock_irqsave(&aplic.lock, flags);
>> +    val = readl((volatile void __iomem *)aplic.regs + offset) & mask;
> 
> Wouldn't this applying of a mask better be done in those callers which
> actually need it? It's not the least the asymmetry with ...

Agree, that to be in sync, I will drop mask argument and apply it on 
caller side.

> 
>> +    spin_unlock_irqrestore(&aplic.lock, flags);
>> +
>> +    return val;
>> +}
>> +
>> +void aplic_hw_write_reg(unsigned int offset, uint32_t value)
> 
> ... this which I consider unhelpful.
> 
>> --- a/xen/arch/riscv/include/asm/aplic.h
>> +++ b/xen/arch/riscv/include/asm/aplic.h
>> @@ -28,6 +28,8 @@
>>   #define APLIC_DOMAINCFG_BE      BIT(0, U)
>>   
>>   /* sourcecfg register fields */
>> +#define APLIC_SOURCECFG_D       BIT(10, U)
> 
> As to the comment - this indeed looks to be a field, but ...
> 
>>   #define APLIC_SOURCECFG_SM_INACTIVE     0x0
>>   #define APLIC_SOURCECFG_SM_DETACH       0x1
>>   #define APLIC_SOURCECFG_SM_EDGE_RISE    0x4
> 
> ... these look to be values of some other field which isn't described. Please
> may I (again) ask that definitions are their commentary at the very least not
> misguide readers?

Thanks for pointing this out. You're right, the comment is misleading as 
written. APLIC_SOURCECFG_D is a field, whereas the APLIC_SOURCECFG_SM_* 
definitions are values for the source mode (SM) field, and the comment 
doesn't make that distinction.

I'll update the comments to describe the fields more accurately:

#define APLIC_SOURCECFG_BASE            0x0004
#define APLIC_SOURCECFG_LAST            0x0ffc
/*
  * sourcecfg[] register fields:
  *  - bit 10 (D) selects the layout of the remaining bits;
  *  - D = 1: bits [9:0] hold the Child Index, i.e. the source is delegated
  *           to a child domain (unsupported by Xen);
  *  - D = 0: bits [2:0] hold the source mode SM (WARL).
  */
#define  APLIC_SOURCECFG_D              BIT(10, U)
/* SM field values (0x2 and 0x3 are reserved): */
#define   APLIC_SOURCECFG_SM_INACTIVE   0x0
#define   APLIC_SOURCECFG_SM_DETACH     0x1
#define   APLIC_SOURCECFG_SM_EDGE_RISE  0x4
#define   APLIC_SOURCECFG_SM_EDGE_FALL  0x5
#define   APLIC_SOURCECFG_SM_LEVEL_HIGH 0x6
#define   APLIC_SOURCECFG_SM_LEVEL_LOW  0x7

Does it look better? Probably there is not sense for two extra spaces 
for APLIC_SOURCECFG_SM_*. I want to show by such identation that it is 
values for SM field of APLIC_SOURCECFG.


> 
>> --- a/xen/arch/riscv/include/asm/imsic.h
>> +++ b/xen/arch/riscv/include/asm/imsic.h
>> @@ -40,6 +40,16 @@ struct imsic_config {
>>       /* Base address */
>>       paddr_t base_addr;
>>   
>> +    /*
>> +     * MSI Target Address Scheme
>> +     *
>> +     * XLEN-1                                                12     0
>> +     * |                                                     |     |
>> +     * -------------------------------------------------------------
>> +     * |xxxxxx|Group Index|xxxxxxxxxxx|HART Index|Guest Index|  0  |
>> +     * -------------------------------------------------------------
>> +     */
> 
> And the xxx-es in here mean what exactly? Don't care? Some other, unrelated
> values? Yet something else?

  The `x` bits denote address bits that are constant across all IMSIC 
interrupt files. They are not used to encode the group, HART, or guest 
index; instead, they correspond to the fixed portion of the IMSIC 
address determined by the platform's memory map.

For example, consider the IMSIC DT binding:

     interrupt-controller@28000000 {
       compatible = "qemu,imsics", "riscv,imsics";
       interrupts-extended = <&cpu1_intc 9>,
                             <&cpu2_intc 9>,
                             <&cpu3_intc 9>,
                             <&cpu4_intc 9>;
       reg = <0x28000000 0x2000>, /* Group0 IMSICs */
             <0x29000000 0x2000>; /* Group1 IMSICs */
       interrupt-controller;
       #interrupt-cells = <0>;
       msi-controller;
       #msi-cells = <0>;
       riscv,num-ids = <127>;
       riscv,group-index-bits = <1>;
       riscv,group-index-shift = <24>;
     };


Here, `hart_index_bits = 2` (4 CPUs) and `guest_index_bits = 0`, so the 
address layout becomes:

31          25 24 23         14 13 12 11          0
+-------------+-+-------------+-----+-------------+
| constant    |G|  constant   |HART |    zeros    |
+-------------+-+-------------+-----+-------------+


I can update the comment to say:
"x denotes bits that are constant across all interrupt file addresses."

or, if you think it's clearer: "x denotes bits whose values are 
platform-defined and common to all interrupt file addresses."

Does it make sense any of suggested options?


> 
>> --- a/xen/arch/riscv/vaplic.c
>> +++ b/xen/arch/riscv/vaplic.c
>> @@ -17,6 +17,7 @@
>>   #include <asm/aia.h>
>>   #include <asm/imsic.h>
>>   #include <asm/intc.h>
>> +#include <asm/mmio.h>
>>   #include <asm/vaplic.h>
>>   
>>   #include "aplic-priv.h"
>> @@ -27,6 +28,256 @@ unsigned int __ro_after_init guest_aplic_num_sources;
>>   
>>   #define FDT_VAPLIC_INT_CELLS 2
>>   
>> +#define AUTH_IRQ_BIT(d, irqn) ( \
>> +    ((irqn) < (d)->arch.vintc->nr_virqs) && \
>> +    test_bit(irqn, (d)->arch.vintc->used_irqs) )
> 
> Nit: Indentation.

I will use the following indentation:

... (((irqn) < (d)->arch.vintc->nr_virqs) && \
      test_bit(irqn, (d)->arch.vintc->used_irqs))

> 
>> +/*
>> + * Convert a byte offset (within a SETIP/CLRIP/SETIE/CLRIE register group) to
>> + * a 32-bit word index into the allocated_irqs bitmap.  Each word covers 32
>> + * interrupt sources.  For SOURCECFG and TARGET groups the same division also
>> + * yields the interrupt number directly, because those arrays store one 32-bit
>> + * register per source.
>> + */
>> +#define regoffset_to_word_idx(reg_val) ((reg_val) / sizeof(uint32_t))
>> +
>> +static inline uint32_t generate_auth_mask(const struct domain *d,
>> +                                          unsigned int word_idx)
>> +{
>> +    unsigned int first_bit = word_idx * sizeof(uint32_t) * BITS_PER_BYTE;
>> +
>> +    if ( word_idx >= DIV_ROUND_UP(d->arch.vintc->nr_virqs,
>> +                                  sizeof(uint32_t) * BITS_PER_BYTE) )
>> +    {
>> +        dprintk(XENLOG_DEBUG, "incorrect word_idx(%u) is passed\n", word_idx);
> 
> Is this really meant to stay?

For debug purpose it could be useful, so I prefer to have it with 
changing it to gprintk(XENLOG_DEBUG, ...) to understand which domain is 
trying to access something wrong.

> 
>> +        return 0U;
> 
> The U suffix is mainly (even if only slightly) obfuscating things, I think.

Agree, I will drop U.

> 
>> +    }
>> +
>> +    return (uint32_t)(d->arch.vintc->used_irqs[first_bit / BITS_PER_LONG] >>
>> +                      (first_bit % BITS_PER_LONG));
> 
> I don't quite understand the need for the cast.

Functionally it isn't need but it documents that it is expected that 
translation from unsinged long  to uint32_t will happen. I will drop the 
cast.

> 
>> +static int cf_check vaplic_emulate_load(const struct vcpu *v,
> 
> Why the cf_check (also for the store counterpart)?

Missed to drop. Before vaplic_emulate_load() was used to initialize 
vints_ops. It should be dropped here.

> 
>> +static int cf_check vaplic_emulate_store(const struct vcpu *v,
>> +                                         unsigned long addr, uint32_t value)
>> +{
>> +    int rc = -EINVAL;
>> +    const struct domain *d = v->domain;
>> +    unsigned int offset = addr & APLIC_REG_OFFSET_MASK;
>> +
>> +    switch ( offset )
>> +    {
>> +    case APLIC_SETIP_BASE ... APLIC_SETIP_LAST:
>> +    case APLIC_CLRIP_BASE ... APLIC_CLRIP_LAST:
>> +    case APLIC_SETIE_BASE ... APLIC_SETIE_LAST:
>> +    case APLIC_CLRIE_BASE ... APLIC_CLRIE_LAST:
>> +    {
>> +        unsigned int word_idx =
>> +            regoffset_to_word_idx(offset & APLIC_SETCLR_OFFSET_MASK);
>> +
>> +        value &= generate_auth_mask(d, word_idx);
>> +
>> +        break;
>> +    }
>> +
>> +    case APLIC_SOURCECFG_BASE ... APLIC_SOURCECFG_LAST:
>> +        if ( value & APLIC_SOURCECFG_D )
>> +        {
>> +            rc = -EOPNOTSUPP;
>> +
>> +            dprintk(XENLOG_ERR, "APLIC_SOURCECFG_D isn't supported\n");
>> +
>> +            goto fail;
>> +        }
>> +
>> +        /*
>> +         * As sourcecfg register starts from 1:
>> +         *   0x0000 domaincfg
>> +         *   0x0004 sourcecfg[1]
>> +         *   0x0008 sourcecfg[2]
>> +         *    ...
>> +         *   0x0FFC sourcecfg[1023]
>> +         * It is necessary to calculate an interrupt number by subtracting
>> +         * APLIC_DOMAINCFG instead of APLIC_SOURCECFG_BASE.
>> +         */
>> +        if ( !AUTH_IRQ_BIT(d, regoffset_to_word_idx(offset - APLIC_DOMAINCFG)) )
>> +            /* Interrupt not enabled, ignore it */
>> +            return 0;
>> +
>> +        if ( value > APLIC_SOURCECFG_SM_LEVEL_LOW )
>> +        {
>> +            gdprintk(XENLOG_ERR,
>> +                     "value(%u) is incorrect for sourcecfg register\n", value);
>> +
>> +            return 0;
>> +        }
>> +
>> +        break;
>> +
>> +    case APLIC_TARGET_BASE ... APLIC_TARGET_LAST:
>> +    {
>> +        struct vcpu *target_vcpu = NULL;
>> +        unsigned int hart_idx = value >> APLIC_TARGET_HART_IDX_SHIFT;
>> +
>> +        /*
>> +         * Look at vaplic_emulate_load() for explanation why
>> +         * APLIC_GENMSI is subtracted.
>> +         */
>> +        if ( !AUTH_IRQ_BIT(d, regoffset_to_word_idx(offset - APLIC_GENMSI)) )
>> +            /* Interrupt not enabled, ignore it */
>> +            return 0;
>> +
>> +        if ( hart_idx < v->domain->max_vcpus )
> 
> You have d as a local variable.
> 
>> +            target_vcpu = v->domain->vcpu[hart_idx];
> 
> Use domain_vcpu()?

It will be better, thanks.

> 
>> +        if ( !target_vcpu )
>> +        {
>> +            dprintk(XENLOG_ERR, "Invalid vCPU id in target register\n");
>> +
>> +            /* Ignore such writings */
>> +            return 0;
>> +        }
>> +
>> +        value = aplic_msi_target_gen(target_vcpu, value);
>> +
>> +        break;
>> +    }
>> +
>> +    case APLIC_SETIPNUM:
>> +    case APLIC_SETIPNUM_LE:
>> +    case APLIC_CLRIPNUM:
>> +    case APLIC_SETIENUM:
>> +    case APLIC_CLRIENUM:
>> +        if ( !value || !AUTH_IRQ_BIT(d, value) )
>> +            return 0;
>> +
>> +        break;
>> +
>> +    case APLIC_DOMAINCFG:
>> +    {
>> +        struct vaplic *vaplic = to_vaplic(v->domain);
>> +
>> +        /*
>> +         * The domaincfg register has this format:
>> +         * bits 31:24 read-only 0x80
>> +         * bit 8      IE
>> +         * bit 7      read-only 0
>> +         * bit 2      DM (WARL)
>> +         * bit 0      BE (WARL)
>> +         *
>> +         * The most interesting bit for us is IE(Interrupt Enable) bit.
>> +         * At the moment, at least, Linux doesn't use domaincfg.IE bit to
>> +         * disable interrupts globally, but if one day someone will use it
>> +         * then extra actions should be done.
>> +         *
>> +         * Only DM (bit 2) and IE (bit 8) are writable here. They are assigned
>> +         * (not OR-ed) so that a write of 0 can also clear them (WARL), and the
>> +         * read-only high byte (0x80) is always kept set on read-back.
>> +         */
>> +        if ( value & ~(APLIC_DOMAINCFG_RO | APLIC_DOMAINCFG_DM |
>> +                       APLIC_DOMAINCFG_IE) )
>> +            printk_once("%s: Ignore writes to non-writable domaincfg bits as "
>> +                        "they are set by aplic during initialization in Xen\n",
>> +                        __func__);
>> +
>> +        vaplic->regs.domaincfg = APLIC_DOMAINCFG_RO |
>> +                                 (value & (APLIC_DOMAINCFG_DM |
>> +                                           APLIC_DOMAINCFG_IE));
>> +
>> +        return 0;
>> +    }
>> +
>> +    default:
>> +        goto fail;
> 
> Instead of this goto, I think you simply want to move the label here.
> That'll also make the function more similar to its load counterpart.

Good point. I am curious how fail label should be aligned:

     default:
  fail:
         gdprintk(XENLOG_WARNING,
                  "Unhandled APLIC write at offset %#x (value %#x)\n", 
offset,
                  value);

         return rc;
     }

or default:
     fail:

?

> 
>> @@ -105,6 +356,50 @@ static const struct vintc_init_ops __initconstrel init_ops = {
>>       .make_domu_dt_node = vaplic_make_domu_dt_node,
>>   };
>>   
>> +static enum io_state cf_check vaplic_mmio_read(struct vcpu *v, mmio_info_t *info,
>> +                                               register_t *r)
>> +{
>> +    uint32_t data = 0;
>> +
>> +    if ( info->len != sizeof(uint32_t) ||
>> +         !IS_ALIGNED(info->gpa, sizeof(uint32_t)) )
>> +    {
>> +        gdprintk(XENLOG_DEBUG,
>> +                 "VAPLIC: unaligned/wrong-width read gpa=%"PRIpaddr" len=%u\n",
>> +                 info->gpa, info->len);
> 
> You have v passed in here, but you'd log current. If passing in v is
> necessary (i.e. here or elsewhere it may be other than current), then you
> need to either ASSERT(v == current) at the top of the funciton or otherwise
> handle v != current correctly.

It makes sense. I will add ASSERT(v == current) here and for 
vaplic_mmio_write().

> 
>> +        return IO_ABORT;
>> +    }
>> +
>> +    if ( vaplic_emulate_load(v, info->gpa, &data) < 0 )
> 
> If all you care about is a boolean result, why not make the function return
> bool?

Agree, bool will be enough for vaplic_emulate_load() and 
vaplic_emulate_save().

Thanks!

~ Oleksii

Re: [PATCH v1 05/17] xen/riscv: implement virtual APLIC MMIO emulation

[Baptiste Le Duc]

On 2026-08-07 18:08:21+02:00, Oleksii Kurochko wrote:
> On 8/6/26 4:28 PM, Jan Beulich wrote:
> 
> > On 20.07.2026 18:02, Oleksii Kurochko wrote:
> > 
> > For this tag to have any meaning, it should move ahead of the --- above;
> > the explanations ...
> > 
> > 
> > ... here rather explain the restriction on the R-b, not its odd placement.
> > 
> > 
> > As this looks to be recurring - please get versioning of your series right.
> > The series is supposedly v1, but here you give the impression of it being
> > v3. If there really was an earlier v2 posting, why isn't the entire series
> > here v3?
> 
> It is v3 before before it was a part of another patch series connected 
> to dom0less config enablement.
> 
> Would it be better to just write in "Change in v3" that it is moved from 
> another patch series + link to that patch series? Or it will be enough 
> just to drop "Changes in v2 and v1" and just start from v1?
> 
> > PLease can you, before submitting, self-review your patches? I'm really
> > getting tired of having to repeatedly point out basic style issues, like
> > the overlong line here.
> 
> Sorry for that, I will write an extra checker for such cases to not miss 
> them.
> 
> > It extends to the other local variables here, but I'll use these two to
> > try to make my point: I'm struggling to associate the names with the
> > values they are set to. Likely "hxw" is an abbreviation of hart index
> > width, but (a) what's the leading 'l' then and (b) why is there no 'g'
> > in "hhxw"? By using hard to grasp names, you make it hard to actually
> > understand the subsequent expressions, in particular ...
> 
> The names it taken directly from AIA spec:
> 
> The use of this value and fields HHXS (High Hart Index Shift), LHXS (Low 
> Hart Index Shift), HHXW (High Hart Index Width), and LHXW (Low Hart 
> Index Width) for determining target addresses for MSIs is described 
> later, in Section 4.9.1.
> 
> The AIA specification interprets the machine-level hart index as a 
> combination of the **group index** (`g`) and the **hart index within the 
> group** (`h`), according to the following formulas:
> 
> ```
> (1) g = (machine-level hart index >> LHXW) & (2^HHXW − 1)
> (2) h = machine-level hart index & (2^LHXW − 1)
> ```
> 
> (In our case, the machine-level hart index is equal to `mhartid`, i.e. 
> the hart index.)
Therefore, if I understand correclty, if we take the Hart Index as
defined in the AIA spec, we should have:
Hart Index = (g << LHXW) | h
Is it correct?
> 
> For systems that use IMSIC groups, the IMSIC address layout is defined 
> by the following parameters:
> 
> * `lhxw` (Low Hart Index Width, or *k*): the number of bits used for the 
> hart number within a group.
> * `hhxw` (High Hart Index Width, or *j*): the number of bits used for 
> the group number.
Is group number appelation equivalent to group index?

I think with if what I wrote above is correct, the proper definition for
`hhxw` and `hhxs` should be:
* `hhxw` (High Hart Index Width, or *j*): the number of bits used for
the `Hart Index` field within the physical address.
> * `hhxs` (High Hart Index Shift): the bit offset of the combined 
> hart/group index field within the physical address.
* `hhxs` (High Hart Index Shift): the bit offset of the `Hart Index`
field within the physical address.
> To extract the group index, we first shift the address by `hhxs` so that 
> the group index bits are aligned, and then apply a mask derived from 
> `hhxw` to isolate those bits.
> 
> The hardware performs the same operation to extract the hart index from 
> the MSI address. However, in our case we already know which hart should 
> receive the interrupt (`hartid`), so there is no need to extract the 
> hart index from the base address. We only need to recover the group 
> index and combine it with `hartid` to construct the value expected by 
> the `target` register.

Why don't we direclty extract the Hart Index as target directly needs it
as explained in the 4.5.16.2 point of the AIA spec:
target[31:18] = Hart Index
target[17:12] = Guest Index
target[10:0] = EEID
It'd be easier as we just have to do shift from HHXS and apply HHXW.
> 
> > ... these last two. As it stands, they may be easier to understand if
> > you didn't have the local variables at all, despite them then getting
> > textually longer.
> 
> With the explanation above, do the variable names make sense?
> 
> To be closer to AIA spec I think it would be better to rename 
> group_index to g and hart_id to h. Does it make sense to you?
> 
> >> +
> > 
> > Wouldn't this applying of a mask better be done in those callers which
> > actually need it? It's not the least the asymmetry with ...
> 
> Agree, that to be in sync, I will drop mask argument and apply it on 
> caller side.
> 
> > ... this which I consider unhelpful.
> > 
> > 
> > As to the comment - this indeed looks to be a field, but ...
> > 
> > 
> > ... these look to be values of some other field which isn't described. Please
> > may I (again) ask that definitions are their commentary at the very least not
> > misguide readers?
> 
> Thanks for pointing this out. You're right, the comment is misleading as 
> written. APLIC_SOURCECFG_D is a field, whereas the APLIC_SOURCECFG_SM_* 
> definitions are values for the source mode (SM) field, and the comment 
> doesn't make that distinction.
> 
> I'll update the comments to describe the fields more accurately:
> 
> #define APLIC_SOURCECFG_BASE            0x0004
> #define APLIC_SOURCECFG_LAST            0x0ffc
> /*
>   * sourcecfg[] register fields:
>   *  - bit 10 (D) selects the layout of the remaining bits;
>   *  - D = 1: bits [9:0] hold the Child Index, i.e. the source is delegated
>   *           to a child domain (unsupported by Xen);
Just to know, what is a child domain?
>   *  - D = 0: bits [2:0] hold the source mode SM (WARL).
>   */
> #define  APLIC_SOURCECFG_D              BIT(10, U)
> /* SM field values (0x2 and 0x3 are reserved): */
> #define   APLIC_SOURCECFG_SM_INACTIVE   0x0
> #define   APLIC_SOURCECFG_SM_DETACH     0x1
> #define   APLIC_SOURCECFG_SM_EDGE_RISE  0x4
> #define   APLIC_SOURCECFG_SM_EDGE_FALL  0x5
> #define   APLIC_SOURCECFG_SM_LEVEL_HIGH 0x6
> #define   APLIC_SOURCECFG_SM_LEVEL_LOW  0x7
> 
> Does it look better? Probably there is not sense for two extra spaces 
> for APLIC_SOURCECFG_SM_*. I want to show by such identation that it is 
> values for SM field of APLIC_SOURCECFG.
> 
> > And the xxx-es in here mean what exactly? Don't care? Some other, unrelated
> > values? Yet something else?
> 
> The `x` bits denote address bits that are constant across all IMSIC 
> interrupt files. They are not used to encode the group, HART, or guest 
> index; instead, they correspond to the fixed portion of the IMSIC 
> address determined by the platform's memory map.
> 
> For example, consider the IMSIC DT binding:
> 
>      interrupt-controller@28000000 {
>        compatible = "qemu,imsics", "riscv,imsics";
>        interrupts-extended = <&cpu1_intc 9>,
>                              <&cpu2_intc 9>,
>                              <&cpu3_intc 9>,
>                              <&cpu4_intc 9>;
>        reg = <0x28000000 0x2000>, /* Group0 IMSICs */
>              <0x29000000 0x2000>; /* Group1 IMSICs */
>        interrupt-controller;
>        #interrupt-cells = <0>;
>        msi-controller;
>        #msi-cells = <0>;
>        riscv,num-ids = <127>;
>        riscv,group-index-bits = <1>;
>        riscv,group-index-shift = <24>;
>      };
> 
> 
> Here, `hart_index_bits = 2` (4 CPUs) and `guest_index_bits = 0`, so the 
> address layout becomes:
> 
> 31          25 24 23         14 13 12 11          0
> +-------------+-+-------------+-----+-------------+
> | constant    |G|  constant   |HART |    zeros    |
> +-------------+-+-------------+-----+-------------+
> 
> 
> I can update the comment to say:
> "x denotes bits that are constant across all interrupt file addresses."
> 
> or, if you think it's clearer: "x denotes bits whose values are 
> platform-defined and common to all interrupt file addresses."
> 
> Does it make sense any of suggested options?
> 
> > Nit: Indentation.
> 
> I will use the following indentation:
> 
> ... (((irqn) < (d)->arch.vintc->nr_virqs) && \
>       test_bit(irqn, (d)->arch.vintc->used_irqs))
> 
> > Is this really meant to stay?
> 
> For debug purpose it could be useful, so I prefer to have it with 
> changing it to gprintk(XENLOG_DEBUG, ...) to understand which domain is 
> trying to access something wrong.
> 
> > The U suffix is mainly (even if only slightly) obfuscating things, I think.
> 
> Agree, I will drop U.
> 
> > I don't quite understand the need for the cast.
> 
> Functionally it isn't need but it documents that it is expected that 
> translation from unsinged long  to uint32_t will happen. I will drop the 
> cast.
> 
> > Why the cf_check (also for the store counterpart)?
> 
> Missed to drop. Before vaplic_emulate_load() was used to initialize 
> vints_ops. It should be dropped here.
> 
> > You have d as a local variable.
> > 
> > 
> > Use domain_vcpu()?
> 
> It will be better, thanks.
> 
> > Instead of this goto, I think you simply want to move the label here.
> > That'll also make the function more similar to its load counterpart.
> 
> Good point. I am curious how fail label should be aligned:
> 
>      default:
>   fail:
>          gdprintk(XENLOG_WARNING,
>                   "Unhandled APLIC write at offset %#x (value %#x)\n", 
> offset,
>                   value);
> 
>          return rc;
>      }
> 
> or default:
>      fail:
> 
> ?
> 
> > You have v passed in here, but you'd log current. If passing in v is
> > necessary (i.e. here or elsewhere it may be other than current), then you
> > need to either ASSERT(v == current) at the top of the funciton or otherwise
> > handle v != current correctly.
> 
> It makes sense. I will add ASSERT(v == current) here and for 
> vaplic_mmio_write().
> 
> > If all you care about is a boolean result, why not make the function return
> > bool?
> 
> Agree, bool will be enough for vaplic_emulate_load() and 
> vaplic_emulate_save().
> 
> Thanks!
> 
> ~ Oleksii

I will try to draw some schema to make the AIA spec more explicit. Maybe
it could be part of this series, I don't know what is the xen policy
about diagram and stuff like that. Do you know more about that? In order
to not do a job with no needed at all.

Re: [PATCH v1 05/17] xen/riscv: implement virtual APLIC MMIO emulation

[Oleksii Kurochko]

On 8/11/26 11:21 AM, Baptiste Le Duc wrote:
> On 2026-08-07 18:08:21+02:00, Oleksii Kurochko wrote:
>> On 8/6/26 4:28 PM, Jan Beulich wrote:
>>
>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>
>>> For this tag to have any meaning, it should move ahead of the --- above;
>>> the explanations ...
>>>
>>>
>>> ... here rather explain the restriction on the R-b, not its odd placement.
>>>
>>>
>>> As this looks to be recurring - please get versioning of your series right.
>>> The series is supposedly v1, but here you give the impression of it being
>>> v3. If there really was an earlier v2 posting, why isn't the entire series
>>> here v3?
>>
>> It is v3 before before it was a part of another patch series connected
>> to dom0less config enablement.
>>
>> Would it be better to just write in "Change in v3" that it is moved from
>> another patch series + link to that patch series? Or it will be enough
>> just to drop "Changes in v2 and v1" and just start from v1?
>>
>>> PLease can you, before submitting, self-review your patches? I'm really
>>> getting tired of having to repeatedly point out basic style issues, like
>>> the overlong line here.
>>
>> Sorry for that, I will write an extra checker for such cases to not miss
>> them.
>>
>>> It extends to the other local variables here, but I'll use these two to
>>> try to make my point: I'm struggling to associate the names with the
>>> values they are set to. Likely "hxw" is an abbreviation of hart index
>>> width, but (a) what's the leading 'l' then and (b) why is there no 'g'
>>> in "hhxw"? By using hard to grasp names, you make it hard to actually
>>> understand the subsequent expressions, in particular ...
>>
>> The names it taken directly from AIA spec:
>>
>> The use of this value and fields HHXS (High Hart Index Shift), LHXS (Low
>> Hart Index Shift), HHXW (High Hart Index Width), and LHXW (Low Hart
>> Index Width) for determining target addresses for MSIs is described
>> later, in Section 4.9.1.
>>
>> The AIA specification interprets the machine-level hart index as a
>> combination of the **group index** (`g`) and the **hart index within the
>> group** (`h`), according to the following formulas:
>>
>> ```
>> (1) g = (machine-level hart index >> LHXW) & (2^HHXW − 1)
>> (2) h = machine-level hart index & (2^LHXW − 1)
>> ```
>>
>> (In our case, the machine-level hart index is equal to `mhartid`, i.e.
>> the hart index.)
> Therefore, if I understand correclty, if we take the Hart Index as
> defined in the AIA spec, we should have:
> Hart Index = (g << LHXW) | h
> Is it correct?

Yes.

But note that in the current version of aplic_hart_field(), hart_id is 
passed directly, so there is no need to extract h as described in the 
AIA specification. We only need to concatenate it with the group index 
that we have already extracted.

This is partly because aplic_hart_field() uses only .base_addr, which 
does not contain hart_index.

If we want to follow the AIA specification fully, using its terminology, 
the code should look something like:

static unsigned long aplic_hart_field(unsigned int cpu)
{
     const struct imsic_config *imsic = imsic_get_config();
     const struct imsic_msi *msi = &imsic->msi[cpu];
     unsigned int lhxs = imsic->guest_index_bits;
     unsigned int lhxw = imsic->hart_index_bits;
     unsigned int hhxw = imsic->group_index_bits;
     unsigned int hhxs =
         imsic->group_index_shift - APLIC_xMSICFGADDR_PPN_SHIFT * 2;
     /*
      * msi->base_addr is the base of the MMIO regset this CPU's interrupt
      * files live in, and one regset can cover several harts; msi->offset
      * selects this CPU's block inside it.  The hart index bits are part of
      * that offset, so both indexes have to be derived from the full 
address.
      */
     paddr_t target_addr = msi->base_addr + msi->offset;
     unsigned long tppn = target_addr >> APLIC_xMSICFGADDR_PPN_SHIFT;
     unsigned long group_index =
         (tppn >> APLIC_xMSICFGADDR_PPN_HHX_SHIFT(hhxs)) &
         APLIC_xMSICFGADDR_PPN_HHX_MASK(hhxw);
     unsigned long hart_index =
         (tppn >> APLIC_xMSICFGADDR_PPN_LHX_SHIFT(lhxs)) &
         APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw);

     return (group_index << lhxw) | hart_index;
}

(note that during writing that I found an issue, it should be really 
passed Xen cpu id, not hartid as msi[] is iterated through Xen cpu id so 
I've taken that into account when wrote an implementation mentioned above)

Generally I think I am okay with both version of how to get hart_index 
(or pass it by an argument or extract it).

>>
>> For systems that use IMSIC groups, the IMSIC address layout is defined
>> by the following parameters:
>>
>> * `lhxw` (Low Hart Index Width, or *k*): the number of bits used for the
>> hart number within a group.
>> * `hhxw` (High Hart Index Width, or *j*): the number of bits used for
>> the group number.
> Is group number appelation equivalent to group index?
> 
> I think with if what I wrote above is correct, the proper definition for
> `hhxw` and `hhxs` should be:
> * `hhxw` (High Hart Index Width, or *j*): the number of bits used for
> the `Hart Index` field within the physical address.
>> * `hhxs` (High Hart Index Shift): the bit offset of the combined
>> hart/group index field within the physical address.
> * `hhxs` (High Hart Index Shift): the bit offset of the `Hart Index`
> field within the physical address.
>> To extract the group index, we first shift the address by `hhxs` so that
>> the group index bits are aligned, and then apply a mask derived from
>> `hhxw` to isolate those bits.
>>
>> The hardware performs the same operation to extract the hart index from
>> the MSI address. However, in our case we already know which hart should
>> receive the interrupt (`hartid`), so there is no need to extract the
>> hart index from the base address. We only need to recover the group
>> index and combine it with `hartid` to construct the value expected by
>> the `target` register.
> 
> Why don't we direclty extract the Hart Index as target directly needs it
> as explained in the 4.5.16.2 point of the AIA spec:
> target[31:18] = Hart Index
> target[17:12] = Guest Index
> target[10:0] = EEID
> It'd be easier as we just have to do shift from HHXS and apply HHXW.

 From IMSIC's DT-binding description we have:

   XLEN-1            > (HART Index MSB)                  12    0
   |                  |                                  |     |
   -------------------------------------------------------------
   |xxxxxx|Group Index|xxxxxxxxxxx|HART Index|Guest Index|  0  |
   -------------------------------------------------------------

If you see there is a set of "xxxxxx" between HART and Group Indexes 
that is the reason why we have to extract HART and Group Index 
separately as when h/w will work with target register it doesn't know 
about "xxxxx" at all so from h/w point of view target's register hart 
field looks like |Group Index|Hart Index|. In other words, h/w will do
the following with TARGET's hart index field:
     group_idx = hart_idx >> lhxw;
     hart_idx &= APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw);

and then embed group_idx and hart_idx into the structure above.

Does it make sense?

> 
> I will try to draw some schema to make the AIA spec more explicit. Maybe
> it could be part of this series, I don't know what is the xen policy
> about diagram and stuff like that. Do you know more about that?

Unfortunately, no, I don't.

> In order 
> to not do a job with no needed at all.
> 

IMO, it is enough only AIA spec here to understand. At least, it is 
clear to me.

~ Oleksii

Re: [PATCH v1 05/17] xen/riscv: implement virtual APLIC MMIO emulation

[Baptiste Le Duc]

On 2026-08-11 16:36 +0200, Oleksii Kurochko wrote:
> 
> 
> On 8/11/26 11:21 AM, Baptiste Le Duc wrote:
> > On 2026-08-07 18:08:21+02:00, Oleksii Kurochko wrote:
> >> On 8/6/26 4:28 PM, Jan Beulich wrote:
> >>
> >>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
> >>>
> >>> For this tag to have any meaning, it should move ahead of the --- above;
> >>> the explanations ...
> >>>
> >>>
> >>> ... here rather explain the restriction on the R-b, not its odd placement.
> >>>
> >>>
> >>> As this looks to be recurring - please get versioning of your series right.
> >>> The series is supposedly v1, but here you give the impression of it being
> >>> v3. If there really was an earlier v2 posting, why isn't the entire series
> >>> here v3?
> >>
> >> It is v3 before before it was a part of another patch series connected
> >> to dom0less config enablement.
> >>
> >> Would it be better to just write in "Change in v3" that it is moved from
> >> another patch series + link to that patch series? Or it will be enough
> >> just to drop "Changes in v2 and v1" and just start from v1?
> >>
> >>> PLease can you, before submitting, self-review your patches? I'm really
> >>> getting tired of having to repeatedly point out basic style issues, like
> >>> the overlong line here.
> >>
> >> Sorry for that, I will write an extra checker for such cases to not miss
> >> them.
> >>
> >>> It extends to the other local variables here, but I'll use these two to
> >>> try to make my point: I'm struggling to associate the names with the
> >>> values they are set to. Likely "hxw" is an abbreviation of hart index
> >>> width, but (a) what's the leading 'l' then and (b) why is there no 'g'
> >>> in "hhxw"? By using hard to grasp names, you make it hard to actually
> >>> understand the subsequent expressions, in particular ...
> >>
> >> The names it taken directly from AIA spec:
> >>
> >> The use of this value and fields HHXS (High Hart Index Shift), LHXS (Low
> >> Hart Index Shift), HHXW (High Hart Index Width), and LHXW (Low Hart
> >> Index Width) for determining target addresses for MSIs is described
> >> later, in Section 4.9.1.
> >>
> >> The AIA specification interprets the machine-level hart index as a
> >> combination of the **group index** (`g`) and the **hart index within the
> >> group** (`h`), according to the following formulas:
> >>
> >> ```
> >> (1) g = (machine-level hart index >> LHXW) & (2^HHXW − 1)
> >> (2) h = machine-level hart index & (2^LHXW − 1)
> >> ```
> >>
> >> (In our case, the machine-level hart index is equal to `mhartid`, i.e.
> >> the hart index.)
> > Therefore, if I understand correclty, if we take the Hart Index as
> > defined in the AIA spec, we should have:
> > Hart Index = (g << LHXW) | h
> > Is it correct?
> 
> Yes.
> 
> But note that in the current version of aplic_hart_field(), hart_id is 
> passed directly, so there is no need to extract h as described in the 
> AIA specification. We only need to concatenate it with the group index 
> that we have already extracted.
> 
> This is partly because aplic_hart_field() uses only .base_addr, which 
> does not contain hart_index.
> 
> If we want to follow the AIA specification fully, using its terminology, 
> the code should look something like:
> 
> static unsigned long aplic_hart_field(unsigned int cpu)
> {
>      const struct imsic_config *imsic = imsic_get_config();
>      const struct imsic_msi *msi = &imsic->msi[cpu];
Could you please specify how this function will be used and when? It's
hard for me to understand how imsic->msi[cpu] is filled.
>      unsigned int lhxs = imsic->guest_index_bits;
>      unsigned int lhxw = imsic->hart_index_bits;
>      unsigned int hhxw = imsic->group_index_bits;
>      unsigned int hhxs =
>          imsic->group_index_shift - APLIC_xMSICFGADDR_PPN_SHIFT * 2;
>      /*
>       * msi->base_addr is the base of the MMIO regset this CPU's interrupt
>       * files live in, and one regset can cover several harts; msi->offset
>       * selects this CPU's block inside it.  The hart index bits are part of
>       * that offset, so both indexes have to be derived from the full 
> address.
>       */
>      paddr_t target_addr = msi->base_addr + msi->offset;
>      unsigned long tppn = target_addr >> APLIC_xMSICFGADDR_PPN_SHIFT;
>      unsigned long group_index =
>          (tppn >> APLIC_xMSICFGADDR_PPN_HHX_SHIFT(hhxs)) &
>          APLIC_xMSICFGADDR_PPN_HHX_MASK(hhxw);
>      unsigned long hart_index =
>          (tppn >> APLIC_xMSICFGADDR_PPN_LHX_SHIFT(lhxs)) &
>          APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw);
> 
>      return (group_index << lhxw) | hart_index;
> }
> 
> (note that during writing that I found an issue, it should be really 
> passed Xen cpu id, not hartid as msi[] is iterated through Xen cpu id so 
> I've taken that into account when wrote an implementation mentioned above)
> 
> Generally I think I am okay with both version of how to get hart_index 
> (or pass it by an argument or extract it).
> 
> >>
> >> For systems that use IMSIC groups, the IMSIC address layout is defined
> >> by the following parameters:
> >>
> >> * `lhxw` (Low Hart Index Width, or *k*): the number of bits used for the
> >> hart number within a group.
> >> * `hhxw` (High Hart Index Width, or *j*): the number of bits used for
> >> the group number.
> > Is group number appelation equivalent to group index?
> > 
> > I think with if what I wrote above is correct, the proper definition for
> > `hhxw` and `hhxs` should be:
> > * `hhxw` (High Hart Index Width, or *j*): the number of bits used for
> > the `Hart Index` field within the physical address.
> >> * `hhxs` (High Hart Index Shift): the bit offset of the combined
> >> hart/group index field within the physical address.
> > * `hhxs` (High Hart Index Shift): the bit offset of the `Hart Index`
> > field within the physical address.
> >> To extract the group index, we first shift the address by `hhxs` so that
> >> the group index bits are aligned, and then apply a mask derived from
> >> `hhxw` to isolate those bits.
> >>
> >> The hardware performs the same operation to extract the hart index from
> >> the MSI address. However, in our case we already know which hart should
> >> receive the interrupt (`hartid`), so there is no need to extract the
> >> hart index from the base address. We only need to recover the group
> >> index and combine it with `hartid` to construct the value expected by
> >> the `target` register.
> > 
> > Why don't we direclty extract the Hart Index as target directly needs it
> > as explained in the 4.5.16.2 point of the AIA spec:
> > target[31:18] = Hart Index
> > target[17:12] = Guest Index
> > target[10:0] = EEID
> > It'd be easier as we just have to do shift from HHXS and apply HHXW.
> 
>  From IMSIC's DT-binding description we have:
> 
>    XLEN-1            > (HART Index MSB)                  12    0
>    |                  |                                  |     |
>    -------------------------------------------------------------
>    |xxxxxx|Group Index|xxxxxxxxxxx|HART Index|Guest Index|  0  |
>    -------------------------------------------------------------
> 
> If you see there is a set of "xxxxxx" between HART and Group Indexes 
I think I'm missunderstanding the spec, as I wrote before I thought that
[1] `Hart index` = group_idx << LHXW | hart_idx_within_the_group so,
does the Hart Index in the schema refer to hart_idx_within_the_group or
to [1]? The naming makes me a bit confuse.
> that is the reason why we have to extract HART and Group Index 
> separately as when h/w will work with target register it doesn't know 
> about "xxxxx" at all so from h/w point of view target's register hart 
> field looks like |Group Index|Hart Index|. In other words, h/w will do
> the following with TARGET's hart index field:
>      group_idx = hart_idx >> lhxw;
>      hart_idx &= APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw);
> 
> and then embed group_idx and hart_idx into the structure above.
> 
> Does it make sense?
> 
> > 
> > I will try to draw some schema to make the AIA spec more explicit. Maybe
> > it could be part of this series, I don't know what is the xen policy
> > about diagram and stuff like that. Do you know more about that?
> 
> Unfortunately, no, I don't.
> 
> > In order 
> > to not do a job with no needed at all.
> > 
> 
> IMO, it is enough only AIA spec here to understand. At least, it is 
> clear to me.
> 
> ~ Oleksii
> 
> 
> 

Re: [PATCH v1 05/17] xen/riscv: implement virtual APLIC MMIO emulation

[Oleksii Kurochko]

On 8/11/26 5:29 PM, Baptiste Le Duc wrote:
> On 2026-08-11 16:36 +0200, Oleksii Kurochko wrote:
>>
>>
>> On 8/11/26 11:21 AM, Baptiste Le Duc wrote:
>>> On 2026-08-07 18:08:21+02:00, Oleksii Kurochko wrote:
>>>> On 8/6/26 4:28 PM, Jan Beulich wrote:
>>>>
>>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>>
>>>>> For this tag to have any meaning, it should move ahead of the --- above;
>>>>> the explanations ...
>>>>>
>>>>>
>>>>> ... here rather explain the restriction on the R-b, not its odd placement.
>>>>>
>>>>>
>>>>> As this looks to be recurring - please get versioning of your series right.
>>>>> The series is supposedly v1, but here you give the impression of it being
>>>>> v3. If there really was an earlier v2 posting, why isn't the entire series
>>>>> here v3?
>>>>
>>>> It is v3 before before it was a part of another patch series connected
>>>> to dom0less config enablement.
>>>>
>>>> Would it be better to just write in "Change in v3" that it is moved from
>>>> another patch series + link to that patch series? Or it will be enough
>>>> just to drop "Changes in v2 and v1" and just start from v1?
>>>>
>>>>> PLease can you, before submitting, self-review your patches? I'm really
>>>>> getting tired of having to repeatedly point out basic style issues, like
>>>>> the overlong line here.
>>>>
>>>> Sorry for that, I will write an extra checker for such cases to not miss
>>>> them.
>>>>
>>>>> It extends to the other local variables here, but I'll use these two to
>>>>> try to make my point: I'm struggling to associate the names with the
>>>>> values they are set to. Likely "hxw" is an abbreviation of hart index
>>>>> width, but (a) what's the leading 'l' then and (b) why is there no 'g'
>>>>> in "hhxw"? By using hard to grasp names, you make it hard to actually
>>>>> understand the subsequent expressions, in particular ...
>>>>
>>>> The names it taken directly from AIA spec:
>>>>
>>>> The use of this value and fields HHXS (High Hart Index Shift), LHXS (Low
>>>> Hart Index Shift), HHXW (High Hart Index Width), and LHXW (Low Hart
>>>> Index Width) for determining target addresses for MSIs is described
>>>> later, in Section 4.9.1.
>>>>
>>>> The AIA specification interprets the machine-level hart index as a
>>>> combination of the **group index** (`g`) and the **hart index within the
>>>> group** (`h`), according to the following formulas:
>>>>
>>>> ```
>>>> (1) g = (machine-level hart index >> LHXW) & (2^HHXW − 1)
>>>> (2) h = machine-level hart index & (2^LHXW − 1)
>>>> ```
>>>>
>>>> (In our case, the machine-level hart index is equal to `mhartid`, i.e.
>>>> the hart index.)
>>> Therefore, if I understand correclty, if we take the Hart Index as
>>> defined in the AIA spec, we should have:
>>> Hart Index = (g << LHXW) | h
>>> Is it correct?
>>
>> Yes.
>>
>> But note that in the current version of aplic_hart_field(), hart_id is
>> passed directly, so there is no need to extract h as described in the
>> AIA specification. We only need to concatenate it with the group index
>> that we have already extracted.
>>
>> This is partly because aplic_hart_field() uses only .base_addr, which
>> does not contain hart_index.
>>
>> If we want to follow the AIA specification fully, using its terminology,
>> the code should look something like:
>>
>> static unsigned long aplic_hart_field(unsigned int cpu)
>> {
>>       const struct imsic_config *imsic = imsic_get_config();
>>       const struct imsic_msi *msi = &imsic->msi[cpu];
> Could you please specify how this function will be used and when? It's
> hard for me to understand how imsic->msi[cpu] is filled.

imsic->msi[] is filled during IMSIC initialization in imsic_init(), 
based on the MMIO regset specified in the IMSIC node’s reg property and 
the number of parents specified in the interrupts-extended property. 
This is explained to some extent in the comment above local target_addr 
in aplic_hart_field() (a little further down).

I am not 100% sure that I fully understand the connection between your 
question and the sentence after it, but I planned to write the following 
above the function declaration:


/*
  * The arrangement of IMSIC interrupt files in MMIO space follows a 
topology
  * defined by the RISC-V AIA specification. An IMSIC group is a set of
  * interrupt files (e.g., in a cluster or socket) co-located in memory.
  *
  * The physical address of an outgoing MSI is calculated by bitwise ORing a
  * Base Physical Page Number (Base PPN) with the Group Index (g), the Hart
  * Index (h) and, for a supervisor-level interrupt domain, the Guest Index:
  *
  *   ( Base PPN | (g << (HHXS + 12)) | (h << LHXS) | guest ) << 12
  *
  * where Base PPN, HHXS, LHXS, HHXW and LHXW come from the 
{m,s}msiaddrcfg[h]
  * registers of the interrupt domain that sends the MSI:
  *
  * XLEN-1           HHXS+24             LHXS+12          12          0
  * |                |                   |                |           |
  * -------------------------------------------------------------------
  * |xxxx|Group Index|xxxxxxxx|Hart Index|xxxx|Guest Index|     0     |
  * -------------------------------------------------------------------
  *
  * - xxxx: the remaining bits of the Base PPN. The specification 
requires the
  *   Base PPN to have zeros in the positions where the indices are OR-ed.
  * - Group Index (g): placed at bit (HHXS + 24) of the physical address.
  * - Hart Index (h): placed at bit (LHXS + 12) of the physical address.
  * - Guest Index: selects one of the 4 KiB pages right above the hart's own
  *   supervisor-level file, i.e. it starts at bit 12; LHXS must 
therefore be
  *   at least as large as the number of guest index bits.
  * - Bits 11:0: always zero because IMSIC files are 4 KiB page-aligned.
  *
  * For wired interrupts in MSI delivery mode (domaincfg.DM = 1) the APLIC
  * builds that address itself from the "Hart Index" field (bits 31:18) 
of the
  * corresponding target[i] register. That field holds a hart index 
*number*,
  * in which both indices are packed adjacently:
  *
  * 13          lhxw+hhxw   lhxw       0
  * |           |           |          |
  * ------------------------------------
  * |     0     |Group Index|Hart Index|
  * ------------------------------------
  *
  * - lhxw (Low Hart Index Width): the number of bits used for the hart 
number
  *   within a group.
  * - hhxw (High Hart Index Width): the number of bits used for the group
  *   number; the remaining bits of the field must be zero.
  *
  * The Guest Index isn't a part of it: for a supervisor-level interrupt 
domain
  * it has its own field (bits 17:12) in target[i].
  *
  * Because there are "xxxx" gaps (Base PPN bits) between the indices in the
  * physical address (depending on HHXS and LHXS), software must extract the
  * group and hart components separately and pack them into the 
APLIC-defined
  * Hart Index format to ensure correct MSI targeting.
  */

Does it answer your question?

>>       unsigned int lhxs = imsic->guest_index_bits;
>>       unsigned int lhxw = imsic->hart_index_bits;
>>       unsigned int hhxw = imsic->group_index_bits;
>>       unsigned int hhxs =
>>           imsic->group_index_shift - APLIC_xMSICFGADDR_PPN_SHIFT * 2;
>>       /*
>>        * msi->base_addr is the base of the MMIO regset this CPU's interrupt
>>        * files live in, and one regset can cover several harts; msi->offset
>>        * selects this CPU's block inside it.  The hart index bits are part of
>>        * that offset, so both indexes have to be derived from the full
>> address.
>>        */
>>       paddr_t target_addr = msi->base_addr + msi->offset;
>>       unsigned long tppn = target_addr >> APLIC_xMSICFGADDR_PPN_SHIFT;
>>       unsigned long group_index =
>>           (tppn >> APLIC_xMSICFGADDR_PPN_HHX_SHIFT(hhxs)) &
>>           APLIC_xMSICFGADDR_PPN_HHX_MASK(hhxw);
>>       unsigned long hart_index =
>>           (tppn >> APLIC_xMSICFGADDR_PPN_LHX_SHIFT(lhxs)) &
>>           APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw);
>>
>>       return (group_index << lhxw) | hart_index;
>> }
>>
>> (note that during writing that I found an issue, it should be really
>> passed Xen cpu id, not hartid as msi[] is iterated through Xen cpu id so
>> I've taken that into account when wrote an implementation mentioned above)
>>
>> Generally I think I am okay with both version of how to get hart_index
>> (or pass it by an argument or extract it).
>>
>>>>
>>>> For systems that use IMSIC groups, the IMSIC address layout is defined
>>>> by the following parameters:
>>>>
>>>> * `lhxw` (Low Hart Index Width, or *k*): the number of bits used for the
>>>> hart number within a group.
>>>> * `hhxw` (High Hart Index Width, or *j*): the number of bits used for
>>>> the group number.
>>> Is group number appelation equivalent to group index?
>>>
>>> I think with if what I wrote above is correct, the proper definition for
>>> `hhxw` and `hhxs` should be:
>>> * `hhxw` (High Hart Index Width, or *j*): the number of bits used for
>>> the `Hart Index` field within the physical address.
>>>> * `hhxs` (High Hart Index Shift): the bit offset of the combined
>>>> hart/group index field within the physical address.
>>> * `hhxs` (High Hart Index Shift): the bit offset of the `Hart Index`
>>> field within the physical address.
>>>> To extract the group index, we first shift the address by `hhxs` so that
>>>> the group index bits are aligned, and then apply a mask derived from
>>>> `hhxw` to isolate those bits.
>>>>
>>>> The hardware performs the same operation to extract the hart index from
>>>> the MSI address. However, in our case we already know which hart should
>>>> receive the interrupt (`hartid`), so there is no need to extract the
>>>> hart index from the base address. We only need to recover the group
>>>> index and combine it with `hartid` to construct the value expected by
>>>> the `target` register.
>>>
>>> Why don't we direclty extract the Hart Index as target directly needs it
>>> as explained in the 4.5.16.2 point of the AIA spec:
>>> target[31:18] = Hart Index
>>> target[17:12] = Guest Index
>>> target[10:0] = EEID
>>> It'd be easier as we just have to do shift from HHXS and apply HHXW.
>>
>>   From IMSIC's DT-binding description we have:
>>
>>     XLEN-1            > (HART Index MSB)                  12    0
>>     |                  |                                  |     |
>>     -------------------------------------------------------------
>>     |xxxxxx|Group Index|xxxxxxxxxxx|HART Index|Guest Index|  0  |
>>     -------------------------------------------------------------
>>
>> If you see there is a set of "xxxxxx" between HART and Group Indexes
> I think I'm missunderstanding the spec, as I wrote before I thought that
> [1] `Hart index` = group_idx << LHXW | hart_idx_within_the_group so,
> does the Hart Index in the schema refer to hart_idx_within_the_group or
> to [1]? The naming makes me a bit confuse.

Could you please check my comment above and if it doesn't provide answer 
to your questions I will try to explain it differently.


>> that is the reason why we have to extract HART and Group Index
>> separately as when h/w will work with target register it doesn't know
>> about "xxxxx" at all so from h/w point of view target's register hart
>> field looks like |Group Index|Hart Index|. In other words, h/w will do
>> the following with TARGET's hart index field:
>>       group_idx = hart_idx >> lhxw;
>>       hart_idx &= APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw);
>>
>> and then embed group_idx and hart_idx into the structure above.
>>
>> Does it make sense?
>>
~ Oleksii

Re: [PATCH v1 05/17] xen/riscv: implement virtual APLIC MMIO emulation

[Baptiste Le Duc]

On 2026-08-11 18:24 +0200, Oleksii Kurochko wrote:
> 
> 
> On 8/11/26 5:29 PM, Baptiste Le Duc wrote:
> > On 2026-08-11 16:36 +0200, Oleksii Kurochko wrote:
> >>
> >>
> >> On 8/11/26 11:21 AM, Baptiste Le Duc wrote:
> >>> On 2026-08-07 18:08:21+02:00, Oleksii Kurochko wrote:
> >>>> On 8/6/26 4:28 PM, Jan Beulich wrote:
> >>>>
> >>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
> >>>>>
> >>>>> For this tag to have any meaning, it should move ahead of the --- above;
> >>>>> the explanations ...
> >>>>>
> >>>>>
> >>>>> ... here rather explain the restriction on the R-b, not its odd placement.
> >>>>>
> >>>>>
> >>>>> As this looks to be recurring - please get versioning of your series right.
> >>>>> The series is supposedly v1, but here you give the impression of it being
> >>>>> v3. If there really was an earlier v2 posting, why isn't the entire series
> >>>>> here v3?
> >>>>
> >>>> It is v3 before before it was a part of another patch series connected
> >>>> to dom0less config enablement.
> >>>>
> >>>> Would it be better to just write in "Change in v3" that it is moved from
> >>>> another patch series + link to that patch series? Or it will be enough
> >>>> just to drop "Changes in v2 and v1" and just start from v1?
> >>>>
> >>>>> PLease can you, before submitting, self-review your patches? I'm really
> >>>>> getting tired of having to repeatedly point out basic style issues, like
> >>>>> the overlong line here.
> >>>>
> >>>> Sorry for that, I will write an extra checker for such cases to not miss
> >>>> them.
> >>>>
> >>>>> It extends to the other local variables here, but I'll use these two to
> >>>>> try to make my point: I'm struggling to associate the names with the
> >>>>> values they are set to. Likely "hxw" is an abbreviation of hart index
> >>>>> width, but (a) what's the leading 'l' then and (b) why is there no 'g'
> >>>>> in "hhxw"? By using hard to grasp names, you make it hard to actually
> >>>>> understand the subsequent expressions, in particular ...
> >>>>
> >>>> The names it taken directly from AIA spec:
> >>>>
> >>>> The use of this value and fields HHXS (High Hart Index Shift), LHXS (Low
> >>>> Hart Index Shift), HHXW (High Hart Index Width), and LHXW (Low Hart
> >>>> Index Width) for determining target addresses for MSIs is described
> >>>> later, in Section 4.9.1.
> >>>>
> >>>> The AIA specification interprets the machine-level hart index as a
> >>>> combination of the **group index** (`g`) and the **hart index within the
> >>>> group** (`h`), according to the following formulas:
> >>>>
> >>>> ```
> >>>> (1) g = (machine-level hart index >> LHXW) & (2^HHXW − 1)
> >>>> (2) h = machine-level hart index & (2^LHXW − 1)
> >>>> ```
> >>>>
> >>>> (In our case, the machine-level hart index is equal to `mhartid`, i.e.
> >>>> the hart index.)
> >>> Therefore, if I understand correclty, if we take the Hart Index as
> >>> defined in the AIA spec, we should have:
> >>> Hart Index = (g << LHXW) | h
> >>> Is it correct?
> >>
> >> Yes.
> >>
> >> But note that in the current version of aplic_hart_field(), hart_id is
> >> passed directly, so there is no need to extract h as described in the
> >> AIA specification. We only need to concatenate it with the group index
> >> that we have already extracted.
> >>
> >> This is partly because aplic_hart_field() uses only .base_addr, which
> >> does not contain hart_index.
> >>
> >> If we want to follow the AIA specification fully, using its terminology,
> >> the code should look something like:
> >>
> >> static unsigned long aplic_hart_field(unsigned int cpu)
> >> {
> >>       const struct imsic_config *imsic = imsic_get_config();
> >>       const struct imsic_msi *msi = &imsic->msi[cpu];
> > Could you please specify how this function will be used and when? It's
> > hard for me to understand how imsic->msi[cpu] is filled.
> 
> imsic->msi[] is filled during IMSIC initialization in imsic_init(), 
> based on the MMIO regset specified in the IMSIC node’s reg property and 
> the number of parents specified in the interrupts-extended property. 
> This is explained to some extent in the comment above local target_addr 
> in aplic_hart_field() (a little further down).
> 
> I am not 100% sure that I fully understand the connection between your 
> question and the sentence after it, but I planned to write the following 
> above the function declaration:
> 
> 
> /*
>   * The arrangement of IMSIC interrupt files in MMIO space follows a 
> topology
>   * defined by the RISC-V AIA specification. An IMSIC group is a set of
>   * interrupt files (e.g., in a cluster or socket) co-located in memory.
>   *
>   * The physical address of an outgoing MSI is calculated by bitwise ORing a
>   * Base Physical Page Number (Base PPN) with the Group Index (g), the Hart
>   * Index (h) and, for a supervisor-level interrupt domain, the Guest Index:
>   *
>   *   ( Base PPN | (g << (HHXS + 12)) | (h << LHXS) | guest ) << 12
>   *
>   * where Base PPN, HHXS, LHXS, HHXW and LHXW come from the 
> {m,s}msiaddrcfg[h]
>   * registers of the interrupt domain that sends the MSI:
>   *
>   * XLEN-1           HHXS+24             LHXS+12          12          0
>   * |                |                   |                |           |
>   * -------------------------------------------------------------------
>   * |xxxx|Group Index|xxxxxxxx|Hart Index|xxxx|Guest Index|     0     |
>   * -------------------------------------------------------------------
>   *
>   * - xxxx: the remaining bits of the Base PPN. The specification 
> requires the
>   *   Base PPN to have zeros in the positions where the indices are OR-ed.
>   * - Group Index (g): placed at bit (HHXS + 24) of the physical address.
>   * - Hart Index (h): placed at bit (LHXS + 12) of the physical address.
>   * - Guest Index: selects one of the 4 KiB pages right above the hart's own
>   *   supervisor-level file, i.e. it starts at bit 12; LHXS must 
> therefore be
>   *   at least as large as the number of guest index bits.

I think the name `Hart Index` is confusing here. In fact, you previously
confirmed it refers to target[i] bits 31:18, i.e. the packed number
(g << LHXW) | h, but here you say `Hart Index` is equivalent to h, which
makes no sense.

I know this diagram came from Linux (Anup Patel, Nov 2022,
https://lore.kernel.org/all/20240307140307.646078-3-apatel@ventanamicro.com/),
where "HART Index" is simply the name of the riscv,hart-index-bits DT
property. Linux's own APLIC driver then reuses a single hart_index
variable for h and for (g << LHXW) | h in consecutive lines, without a
comment, which is confusing - if I understand correctly, obviously :)

I think this diagram could be better aligned with the AIA spec:

    * XLEN-1       HHXS+24          LHXS+12          12          0
    * |            |                |                |           |
    * ------------------------------------------------------------
    * |xxxx|   g   |xxxxxxxx|   h   |xxxx|Guest Index|     0     |
    * ------------------------------------------------------------
    *
    * - g: group number
    * - h: hart number relative to the group
    * - xxxx: remaining Base PPN bits; each gap may be zero-width.

What do you think? It would allow us to keep a single meaning for the
`Hart Index` field, the same one as target[i] bits 31:18 i.e. (g <<
LHXW) | h.

>   * - Bits 11:0: always zero because IMSIC files are 4 KiB page-aligned.
>   *
>   * For wired interrupts in MSI delivery mode (domaincfg.DM = 1) the APLIC
>   * builds that address itself from the "Hart Index" field (bits 31:18) 
> of the
>   * corresponding target[i] register. That field holds a hart index 
> *number*,
>   * in which both indices are packed adjacently:
>   *
>   * 13          lhxw+hhxw   lhxw       0
>   * |           |           |          |
>   * ------------------------------------
>   * |     0     |Group Index|Hart Index|
>   * ------------------------------------
>   *
>   * - lhxw (Low Hart Index Width): the number of bits used for the hart 
> number
>   *   within a group.
>   * - hhxw (High Hart Index Width): the number of bits used for the group
>   *   number; the remaining bits of the field must be zero.
>   *
>   * The Guest Index isn't a part of it: for a supervisor-level interrupt 
> domain
>   * it has its own field (bits 17:12) in target[i].
>   *
>   * Because there are "xxxx" gaps (Base PPN bits) between the indices in the
>   * physical address (depending on HHXS and LHXS), software must extract the
>   * group and hart components separately and pack them into the 
> APLIC-defined
>   * Hart Index format to ensure correct MSI targeting.
>   */
> 
> Does it answer your question?
> 
> >>       unsigned int lhxs = imsic->guest_index_bits;
> >>       unsigned int lhxw = imsic->hart_index_bits;
> >>       unsigned int hhxw = imsic->group_index_bits;
> >>       unsigned int hhxs =
> >>           imsic->group_index_shift - APLIC_xMSICFGADDR_PPN_SHIFT * 2;
> >>       /*
> >>        * msi->base_addr is the base of the MMIO regset this CPU's interrupt
> >>        * files live in, and one regset can cover several harts; msi->offset
> >>        * selects this CPU's block inside it.  The hart index bits are part of
> >>        * that offset, so both indexes have to be derived from the full
> >> address.
> >>        */
> >>       paddr_t target_addr = msi->base_addr + msi->offset;
> >>       unsigned long tppn = target_addr >> APLIC_xMSICFGADDR_PPN_SHIFT;
> >>       unsigned long group_index =
> >>           (tppn >> APLIC_xMSICFGADDR_PPN_HHX_SHIFT(hhxs)) &
> >>           APLIC_xMSICFGADDR_PPN_HHX_MASK(hhxw);
> >>       unsigned long hart_index =
> >>           (tppn >> APLIC_xMSICFGADDR_PPN_LHX_SHIFT(lhxs)) &
> >>           APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw);
> >>
> >>       return (group_index << lhxw) | hart_index;
> >> }
> >>
> >> (note that during writing that I found an issue, it should be really
> >> passed Xen cpu id, not hartid as msi[] is iterated through Xen cpu id so
> >> I've taken that into account when wrote an implementation mentioned above)
> >>
> >> Generally I think I am okay with both version of how to get hart_index
> >> (or pass it by an argument or extract it).
> >>
> >>>>
> >>>> For systems that use IMSIC groups, the IMSIC address layout is defined
> >>>> by the following parameters:
> >>>>
> >>>> * `lhxw` (Low Hart Index Width, or *k*): the number of bits used for the
> >>>> hart number within a group.
> >>>> * `hhxw` (High Hart Index Width, or *j*): the number of bits used for
> >>>> the group number.
> >>> Is group number appelation equivalent to group index?
> >>>
> >>> I think with if what I wrote above is correct, the proper definition for
> >>> `hhxw` and `hhxs` should be:
> >>> * `hhxw` (High Hart Index Width, or *j*): the number of bits used for
> >>> the `Hart Index` field within the physical address.
> >>>> * `hhxs` (High Hart Index Shift): the bit offset of the combined
> >>>> hart/group index field within the physical address.
> >>> * `hhxs` (High Hart Index Shift): the bit offset of the `Hart Index`
> >>> field within the physical address.
> >>>> To extract the group index, we first shift the address by `hhxs` so that
> >>>> the group index bits are aligned, and then apply a mask derived from
> >>>> `hhxw` to isolate those bits.
> >>>>
> >>>> The hardware performs the same operation to extract the hart index from
> >>>> the MSI address. However, in our case we already know which hart should
> >>>> receive the interrupt (`hartid`), so there is no need to extract the
> >>>> hart index from the base address. We only need to recover the group
> >>>> index and combine it with `hartid` to construct the value expected by
> >>>> the `target` register.
> >>>
> >>> Why don't we direclty extract the Hart Index as target directly needs it
> >>> as explained in the 4.5.16.2 point of the AIA spec:
> >>> target[31:18] = Hart Index
> >>> target[17:12] = Guest Index
> >>> target[10:0] = EEID
> >>> It'd be easier as we just have to do shift from HHXS and apply HHXW.
> >>
> >>   From IMSIC's DT-binding description we have:
> >>
> >>     XLEN-1            > (HART Index MSB)                  12    0
> >>     |                  |                                  |     |
> >>     -------------------------------------------------------------
> >>     |xxxxxx|Group Index|xxxxxxxxxxx|HART Index|Guest Index|  0  |
> >>     -------------------------------------------------------------
> >>
> >> If you see there is a set of "xxxxxx" between HART and Group Indexes
> > I think I'm missunderstanding the spec, as I wrote before I thought that
> > [1] `Hart index` = group_idx << LHXW | hart_idx_within_the_group so,
> > does the Hart Index in the schema refer to hart_idx_within_the_group or
> > to [1]? The naming makes me a bit confuse.
> 
> Could you please check my comment above and if it doesn't provide answer 
> to your questions I will try to explain it differently.
>
>
> 
> >> that is the reason why we have to extract HART and Group Index
> >> separately as when h/w will work with target register it doesn't know
> >> about "xxxxx" at all so from h/w point of view target's register hart
> >> field looks like |Group Index|Hart Index|. In other words, h/w will do
> >> the following with TARGET's hart index field:
> >>       group_idx = hart_idx >> lhxw;
> >>       hart_idx &= APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw);
> >>
> >> and then embed group_idx and hart_idx into the structure above.
> >>
> >> Does it make sense?
> >>
> ~ Oleksii
> 
> 

Re: [PATCH v1 05/17] xen/riscv: implement virtual APLIC MMIO emulation

[Oleksii Kurochko]

On 8/12/26 11:47 AM, Baptiste Le Duc wrote:
> On 2026-08-11 18:24 +0200, Oleksii Kurochko wrote:
>>
>>
>> On 8/11/26 5:29 PM, Baptiste Le Duc wrote:
>>> On 2026-08-11 16:36 +0200, Oleksii Kurochko wrote:
>>>>
>>>>
>>>> On 8/11/26 11:21 AM, Baptiste Le Duc wrote:
>>>>> On 2026-08-07 18:08:21+02:00, Oleksii Kurochko wrote:
>>>>>> On 8/6/26 4:28 PM, Jan Beulich wrote:
>>>>>>
>>>>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>>>>
>>>>>>> For this tag to have any meaning, it should move ahead of the --- above;
>>>>>>> the explanations ...
>>>>>>>
>>>>>>>
>>>>>>> ... here rather explain the restriction on the R-b, not its odd placement.
>>>>>>>
>>>>>>>
>>>>>>> As this looks to be recurring - please get versioning of your series right.
>>>>>>> The series is supposedly v1, but here you give the impression of it being
>>>>>>> v3. If there really was an earlier v2 posting, why isn't the entire series
>>>>>>> here v3?
>>>>>>
>>>>>> It is v3 before before it was a part of another patch series connected
>>>>>> to dom0less config enablement.
>>>>>>
>>>>>> Would it be better to just write in "Change in v3" that it is moved from
>>>>>> another patch series + link to that patch series? Or it will be enough
>>>>>> just to drop "Changes in v2 and v1" and just start from v1?
>>>>>>
>>>>>>> PLease can you, before submitting, self-review your patches? I'm really
>>>>>>> getting tired of having to repeatedly point out basic style issues, like
>>>>>>> the overlong line here.
>>>>>>
>>>>>> Sorry for that, I will write an extra checker for such cases to not miss
>>>>>> them.
>>>>>>
>>>>>>> It extends to the other local variables here, but I'll use these two to
>>>>>>> try to make my point: I'm struggling to associate the names with the
>>>>>>> values they are set to. Likely "hxw" is an abbreviation of hart index
>>>>>>> width, but (a) what's the leading 'l' then and (b) why is there no 'g'
>>>>>>> in "hhxw"? By using hard to grasp names, you make it hard to actually
>>>>>>> understand the subsequent expressions, in particular ...
>>>>>>
>>>>>> The names it taken directly from AIA spec:
>>>>>>
>>>>>> The use of this value and fields HHXS (High Hart Index Shift), LHXS (Low
>>>>>> Hart Index Shift), HHXW (High Hart Index Width), and LHXW (Low Hart
>>>>>> Index Width) for determining target addresses for MSIs is described
>>>>>> later, in Section 4.9.1.
>>>>>>
>>>>>> The AIA specification interprets the machine-level hart index as a
>>>>>> combination of the **group index** (`g`) and the **hart index within the
>>>>>> group** (`h`), according to the following formulas:
>>>>>>
>>>>>> ```
>>>>>> (1) g = (machine-level hart index >> LHXW) & (2^HHXW − 1)
>>>>>> (2) h = machine-level hart index & (2^LHXW − 1)
>>>>>> ```
>>>>>>
>>>>>> (In our case, the machine-level hart index is equal to `mhartid`, i.e.
>>>>>> the hart index.)
>>>>> Therefore, if I understand correclty, if we take the Hart Index as
>>>>> defined in the AIA spec, we should have:
>>>>> Hart Index = (g << LHXW) | h
>>>>> Is it correct?
>>>>
>>>> Yes.
>>>>
>>>> But note that in the current version of aplic_hart_field(), hart_id is
>>>> passed directly, so there is no need to extract h as described in the
>>>> AIA specification. We only need to concatenate it with the group index
>>>> that we have already extracted.
>>>>
>>>> This is partly because aplic_hart_field() uses only .base_addr, which
>>>> does not contain hart_index.
>>>>
>>>> If we want to follow the AIA specification fully, using its terminology,
>>>> the code should look something like:
>>>>
>>>> static unsigned long aplic_hart_field(unsigned int cpu)
>>>> {
>>>>        const struct imsic_config *imsic = imsic_get_config();
>>>>        const struct imsic_msi *msi = &imsic->msi[cpu];
>>> Could you please specify how this function will be used and when? It's
>>> hard for me to understand how imsic->msi[cpu] is filled.
>>
>> imsic->msi[] is filled during IMSIC initialization in imsic_init(),
>> based on the MMIO regset specified in the IMSIC node’s reg property and
>> the number of parents specified in the interrupts-extended property.
>> This is explained to some extent in the comment above local target_addr
>> in aplic_hart_field() (a little further down).
>>
>> I am not 100% sure that I fully understand the connection between your
>> question and the sentence after it, but I planned to write the following
>> above the function declaration:
>>
>>
>> /*
>>    * The arrangement of IMSIC interrupt files in MMIO space follows a
>> topology
>>    * defined by the RISC-V AIA specification. An IMSIC group is a set of
>>    * interrupt files (e.g., in a cluster or socket) co-located in memory.
>>    *
>>    * The physical address of an outgoing MSI is calculated by bitwise ORing a
>>    * Base Physical Page Number (Base PPN) with the Group Index (g), the Hart
>>    * Index (h) and, for a supervisor-level interrupt domain, the Guest Index:
>>    *
>>    *   ( Base PPN | (g << (HHXS + 12)) | (h << LHXS) | guest ) << 12
>>    *
>>    * where Base PPN, HHXS, LHXS, HHXW and LHXW come from the
>> {m,s}msiaddrcfg[h]
>>    * registers of the interrupt domain that sends the MSI:
>>    *
>>    * XLEN-1           HHXS+24             LHXS+12          12          0
>>    * |                |                   |                |           |
>>    * -------------------------------------------------------------------
>>    * |xxxx|Group Index|xxxxxxxx|Hart Index|xxxx|Guest Index|     0     |
>>    * -------------------------------------------------------------------
>>    *
>>    * - xxxx: the remaining bits of the Base PPN. The specification
>> requires the
>>    *   Base PPN to have zeros in the positions where the indices are OR-ed.
>>    * - Group Index (g): placed at bit (HHXS + 24) of the physical address.
>>    * - Hart Index (h): placed at bit (LHXS + 12) of the physical address.
>>    * - Guest Index: selects one of the 4 KiB pages right above the hart's own
>>    *   supervisor-level file, i.e. it starts at bit 12; LHXS must
>> therefore be
>>    *   at least as large as the number of guest index bits.
> 
> I think the name `Hart Index` is confusing here. In fact, you previously
> confirmed it refers to target[i] bits 31:18, i.e. the packed number
> (g << LHXW) | h, but here you say `Hart Index` is equivalent to h, which
> makes no sense.
> 
> I know this diagram came from Linux (Anup Patel, Nov 2022,

Not really, this diagram was created from scratch. I think you are 
referring to that one in struct imsic_config but the idea is the same 
and the comment in struct imsic_config should be fixed too. I will 
re-use what we agreed here.

> https://lore.kernel.org/all/20240307140307.646078-3-apatel@ventanamicro.com/),
> where "HART Index" is simply the name of the riscv,hart-index-bits DT
> property. Linux's own APLIC driver then reuses a single hart_index
> variable for h and for (g << LHXW) | h in consecutive lines, without a
> comment, which is confusing - if I understand correctly, obviously :)
> 
> I think this diagram could be better aligned with the AIA spec:
> 
>      * XLEN-1       HHXS+24          LHXS+12          12          0
>      * |            |                |                |           |
>      * ------------------------------------------------------------
>      * |xxxx|   g   |xxxxxxxx|   h   |xxxx|Guest Index|     0     |
>      * ------------------------------------------------------------
>      *
>      * - g: group number
>      * - h: hart number relative to the group
>      * - xxxx: remaining Base PPN bits; each gap may be zero-width.
> 
> What do you think? It would allow us to keep a single meaning for the
> `Hart Index` field, the same one as target[i] bits 31:18 i.e. (g <<
> LHXW) | h.

I agree g and h better describes AIA spec and probably will be easier to 
do a grep in AIA spec.

Thanks.

~ Oleksii

Re: [PATCH v1 05/17] xen/riscv: implement virtual APLIC MMIO emulation

[Jan Beulich]

On 07.08.2026 18:08, Oleksii Kurochko wrote:
> On 8/6/26 4:28 PM, Jan Beulich wrote:
>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>> Guests running under Xen program interrupt routing by writing to APLIC
>>> MMIO registers. Xen must intercept these accesses to enforce interrupt
>>> isolation between domains and to translate guest routing intent into the
>>> underlying physical MSI topology.
>>>
>>> Writes are gated by the domain's authorised interrupt bitmap so that a
>>> guest cannot affect interrupts it does not own. TARGET register writes
>>> additionally require translation of the hart and IMSIC guest-file
>>> indices from virtual to physical, as the APLIC uses these fields
>>> directly to compute the MSI delivery address.
>>>
>>> Delegation (APLIC_SOURCECFG_D) is not yet supported.
>>>
>>> Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>> ---
>>> Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech> # vaplic_mmio_{read,write}
>>
>> For this tag to have any meaning, it should move ahead of the --- above;
>> the explanations ...
>>
>>> The downstream changes related to `vaplic_mmio_{read,write}` were originally
>>> in a separate patch (which was reviewed by Baptiste). However, before
>>> upstreaming, it was decided to merge them into the current patch.
>>> I added `Reviewed-by: Baptiste` in this form for now, but Baptiste will
>>> probably review the remaining changes as well.
>>> Once that happens, I'll simply move the `Reviewed-by` tag up and
>>> remove the `#`.
>>
>> ... here rather explain the restriction on the R-b, not its odd placement.
>>
>>> ---
>>> Changes in v3:
>>
>> As this looks to be recurring - please get versioning of your series right.
>> The series is supposedly v1, but here you give the impression of it being
>> v3. If there really was an earlier v2 posting, why isn't the entire series
>> here v3?
> 
> It is v3 before before it was a part of another patch series connected 
> to dom0less config enablement.
> 
> Would it be better to just write in "Change in v3" that it is moved from 
> another patch series + link to that patch series? Or it will be enough 
> just to drop "Changes in v2 and v1" and just start from v1?

Which part of "never have versions go backwards" was unclear in my earlier
reply?

>>> --- a/xen/arch/riscv/aplic-priv.h
>>> +++ b/xen/arch/riscv/aplic-priv.h
>>> @@ -48,4 +48,6 @@ struct aplic_priv {
>>>    */
>>>   extern unsigned int guest_aplic_num_sources;
>>>   
>>> +uint32_t aplic_msi_target_gen(const struct vcpu *target_vcpu, uint32_t base_val);
>>
>> PLease can you, before submitting, self-review your patches? I'm really
>> getting tired of having to repeatedly point out basic style issues, like
>> the overlong line here.
> 
> Sorry for that, I will write an extra checker for such cases to not miss 
> them.

Well, if there was a checker, many more people would like to use it.

>>> @@ -38,6 +39,60 @@ static struct intc_info __ro_after_init aplic_info = {
>>>       .hw_variant = INTC_APLIC,
>>>   };
>>>   
>>> +static unsigned long aplic_hart_field(unsigned long hartid)
>>> +{
>>> +    const struct imsic_config *imsic = imsic_get_config();
>>> +    unsigned int lhxw = imsic->hart_index_bits;
>>> +    unsigned int hhxw = imsic->group_index_bits;
>>
>> It extends to the other local variables here, but I'll use these two to
>> try to make my point: I'm struggling to associate the names with the
>> values they are set to. Likely "hxw" is an abbreviation of hart index
>> width, but (a) what's the leading 'l' then and (b) why is there no 'g'
>> in "hhxw"? By using hard to grasp names, you make it hard to actually
>> understand the subsequent expressions, in particular ...
> 
> The names it taken directly from AIA spec:
> 
> The use of this value and fields HHXS (High Hart Index Shift), LHXS (Low 
> Hart Index Shift), HHXW (High Hart Index Width), and LHXW (Low Hart 
> Index Width) for determining target addresses for MSIs is described 
> later, in Section 4.9.1.
> 
> The AIA specification interprets the machine-level hart index as a 
> combination of the **group index** (`g`) and the **hart index within the 
> group** (`h`), according to the following formulas:
> 
> ```
> (1) g = (machine-level hart index >> LHXW) & (2^HHXW − 1)
> (2) h = machine-level hart index & (2^LHXW − 1)
> ```
> 
> (In our case, the machine-level hart index is equal to `mhartid`, i.e. 
> the hart index.)
> 
> For systems that use IMSIC groups, the IMSIC address layout is defined 
> by the following parameters:
> 
> * `lhxw` (Low Hart Index Width, or *k*): the number of bits used for the 
> hart number within a group.
> * `hhxw` (High Hart Index Width, or *j*): the number of bits used for 
> the group number.
> * `hhxs` (High Hart Index Shift): the bit offset of the combined 
> hart/group index field within the physical address.
> 
> To extract the group index, we first shift the address by `hhxs` so that 
> the group index bits are aligned, and then apply a mask derived from 
> `hhxw` to isolate those bits.
> 
> The hardware performs the same operation to extract the hart index from 
> the MSI address. However, in our case we already know which hart should 
> receive the interrupt (`hartid`), so there is no need to extract the 
> hart index from the base address. We only need to recover the group 
> index and combine it with `hartid` to construct the value expected by 
> the `target` register.
> 
>>
>>> +    unsigned int hhxs =
>>> +        imsic->group_index_shift - APLIC_xMSICFGADDR_PPN_SHIFT * 2;
>>> +    unsigned long tppn =
>>> +        imsic->msi[hartid].base_addr >> APLIC_xMSICFGADDR_PPN_SHIFT;
>>> +    unsigned long group_index =
>>> +        (tppn >> APLIC_xMSICFGADDR_PPN_HHX_SHIFT(hhxs)) &
>>> +        APLIC_xMSICFGADDR_PPN_HHX_MASK(hhxw);
>>> +
>>> +    return (group_index << lhxw) | hartid;
>>
>> ... these last two. As it stands, they may be easier to understand if
>> you didn't have the local variables at all, despite them then getting
>> textually longer.
> 
> With the explanation above, do the variable names make sense?

Yes and ...

> To be closer to AIA spec I think it would be better to rename 
> group_index to g and hart_id to h. Does it make sense to you?

... yes. Question is whether you want to help readers who aren't that
familiar with the AIA spec. If so, maybe add [brief] comments making 
clear what the names say? E.g.

    /* High Hart Index Shift */
    unsigned int hhxs =
        imsic->group_index_shift - APLIC_xMSICFGADDR_PPN_SHIFT * 2;

>>> --- a/xen/arch/riscv/include/asm/aplic.h
>>> +++ b/xen/arch/riscv/include/asm/aplic.h
>>> @@ -28,6 +28,8 @@
>>>   #define APLIC_DOMAINCFG_BE      BIT(0, U)
>>>   
>>>   /* sourcecfg register fields */
>>> +#define APLIC_SOURCECFG_D       BIT(10, U)
>>
>> As to the comment - this indeed looks to be a field, but ...
>>
>>>   #define APLIC_SOURCECFG_SM_INACTIVE     0x0
>>>   #define APLIC_SOURCECFG_SM_DETACH       0x1
>>>   #define APLIC_SOURCECFG_SM_EDGE_RISE    0x4
>>
>> ... these look to be values of some other field which isn't described. Please
>> may I (again) ask that definitions are their commentary at the very least not
>> misguide readers?
> 
> Thanks for pointing this out. You're right, the comment is misleading as 
> written. APLIC_SOURCECFG_D is a field, whereas the APLIC_SOURCECFG_SM_* 
> definitions are values for the source mode (SM) field, and the comment 
> doesn't make that distinction.
> 
> I'll update the comments to describe the fields more accurately:
> 
> #define APLIC_SOURCECFG_BASE            0x0004
> #define APLIC_SOURCECFG_LAST            0x0ffc
> /*
>   * sourcecfg[] register fields:
>   *  - bit 10 (D) selects the layout of the remaining bits;
>   *  - D = 1: bits [9:0] hold the Child Index, i.e. the source is delegated
>   *           to a child domain (unsupported by Xen);
>   *  - D = 0: bits [2:0] hold the source mode SM (WARL).
>   */
> #define  APLIC_SOURCECFG_D              BIT(10, U)
> /* SM field values (0x2 and 0x3 are reserved): */
> #define   APLIC_SOURCECFG_SM_INACTIVE   0x0
> #define   APLIC_SOURCECFG_SM_DETACH     0x1
> #define   APLIC_SOURCECFG_SM_EDGE_RISE  0x4
> #define   APLIC_SOURCECFG_SM_EDGE_FALL  0x5
> #define   APLIC_SOURCECFG_SM_LEVEL_HIGH 0x6
> #define   APLIC_SOURCECFG_SM_LEVEL_LOW  0x7
> 
> Does it look better? Probably there is not sense for two extra spaces 
> for APLIC_SOURCECFG_SM_*. I want to show by such identation that it is 
> values for SM field of APLIC_SOURCECFG.

Which is fine. All you need to add then is a field definition for the SM
field. Then the extra padding blank will also start to make sense.

>>> --- a/xen/arch/riscv/include/asm/imsic.h
>>> +++ b/xen/arch/riscv/include/asm/imsic.h
>>> @@ -40,6 +40,16 @@ struct imsic_config {
>>>       /* Base address */
>>>       paddr_t base_addr;
>>>   
>>> +    /*
>>> +     * MSI Target Address Scheme
>>> +     *
>>> +     * XLEN-1                                                12     0
>>> +     * |                                                     |     |
>>> +     * -------------------------------------------------------------
>>> +     * |xxxxxx|Group Index|xxxxxxxxxxx|HART Index|Guest Index|  0  |
>>> +     * -------------------------------------------------------------
>>> +     */
>>
>> And the xxx-es in here mean what exactly? Don't care? Some other, unrelated
>> values? Yet something else?
> 
>   The `x` bits denote address bits that are constant across all IMSIC 
> interrupt files. They are not used to encode the group, HART, or guest 
> index; instead, they correspond to the fixed portion of the IMSIC 
> address determined by the platform's memory map.
> 
> For example, consider the IMSIC DT binding:
> 
>      interrupt-controller@28000000 {
>        compatible = "qemu,imsics", "riscv,imsics";
>        interrupts-extended = <&cpu1_intc 9>,
>                              <&cpu2_intc 9>,
>                              <&cpu3_intc 9>,
>                              <&cpu4_intc 9>;
>        reg = <0x28000000 0x2000>, /* Group0 IMSICs */
>              <0x29000000 0x2000>; /* Group1 IMSICs */
>        interrupt-controller;
>        #interrupt-cells = <0>;
>        msi-controller;
>        #msi-cells = <0>;
>        riscv,num-ids = <127>;
>        riscv,group-index-bits = <1>;
>        riscv,group-index-shift = <24>;
>      };
> 
> 
> Here, `hart_index_bits = 2` (4 CPUs) and `guest_index_bits = 0`, so the 
> address layout becomes:
> 
> 31          25 24 23         14 13 12 11          0
> +-------------+-+-------------+-----+-------------+
> | constant    |G|  constant   |HART |    zeros    |
> +-------------+-+-------------+-----+-------------+
> 
> 
> I can update the comment to say:
> "x denotes bits that are constant across all interrupt file addresses."
> 
> or, if you think it's clearer: "x denotes bits whose values are 
> platform-defined and common to all interrupt file addresses."
> 
> Does it make sense any of suggested options?

Either comment is fine imo. What I'd like to suggest is to not use 'x' then,
but e.g. 'c'.

>>> --- a/xen/arch/riscv/vaplic.c
>>> +++ b/xen/arch/riscv/vaplic.c
>>> @@ -17,6 +17,7 @@
>>>   #include <asm/aia.h>
>>>   #include <asm/imsic.h>
>>>   #include <asm/intc.h>
>>> +#include <asm/mmio.h>
>>>   #include <asm/vaplic.h>
>>>   
>>>   #include "aplic-priv.h"
>>> @@ -27,6 +28,256 @@ unsigned int __ro_after_init guest_aplic_num_sources;
>>>   
>>>   #define FDT_VAPLIC_INT_CELLS 2
>>>   
>>> +#define AUTH_IRQ_BIT(d, irqn) ( \
>>> +    ((irqn) < (d)->arch.vintc->nr_virqs) && \
>>> +    test_bit(irqn, (d)->arch.vintc->used_irqs) )
>>
>> Nit: Indentation.
> 
> I will use the following indentation:
> 
> ... (((irqn) < (d)->arch.vintc->nr_virqs) && \
>       test_bit(irqn, (d)->arch.vintc->used_irqs))

Which as written still doesn't look right. What I can't tell is whether
that's merely because of the use of "...".

Of the three opening prarens on the first line, two have their closing
counterparts on the same line. There's thus one pending closing paren,
meaning there should be one extra indenting blank.

>>> +/*
>>> + * Convert a byte offset (within a SETIP/CLRIP/SETIE/CLRIE register group) to
>>> + * a 32-bit word index into the allocated_irqs bitmap.  Each word covers 32
>>> + * interrupt sources.  For SOURCECFG and TARGET groups the same division also
>>> + * yields the interrupt number directly, because those arrays store one 32-bit
>>> + * register per source.
>>> + */
>>> +#define regoffset_to_word_idx(reg_val) ((reg_val) / sizeof(uint32_t))
>>> +
>>> +static inline uint32_t generate_auth_mask(const struct domain *d,
>>> +                                          unsigned int word_idx)
>>> +{
>>> +    unsigned int first_bit = word_idx * sizeof(uint32_t) * BITS_PER_BYTE;
>>> +
>>> +    if ( word_idx >= DIV_ROUND_UP(d->arch.vintc->nr_virqs,
>>> +                                  sizeof(uint32_t) * BITS_PER_BYTE) )
>>> +    {
>>> +        dprintk(XENLOG_DEBUG, "incorrect word_idx(%u) is passed\n", word_idx);
>>
>> Is this really meant to stay?
> 
> For debug purpose it could be useful, so I prefer to have it with 
> changing it to gprintk(XENLOG_DEBUG, ...) to understand which domain is 
> trying to access something wrong.

gdprintk() implies you're on the vCPU that's the subject of the operation.
If that's always the case here, the function parameter wants to reflect
that as far as possible: "currd" instead of "d".

>>> +        return 0U;
>>> +    }
>>> +
>>> +    return (uint32_t)(d->arch.vintc->used_irqs[first_bit / BITS_PER_LONG] >>
>>> +                      (first_bit % BITS_PER_LONG));
>>
>> I don't quite understand the need for the cast.
> 
> Functionally it isn't need but it documents that it is expected that 
> translation from unsinged long  to uint32_t will happen. I will drop the 
> cast.

Thanks. If you really wanted such doc, casts would need adding in many
more places across the code base.

>>> +        if ( !target_vcpu )
>>> +        {
>>> +            dprintk(XENLOG_ERR, "Invalid vCPU id in target register\n");
>>> +
>>> +            /* Ignore such writings */
>>> +            return 0;
>>> +        }
>>> +
>>> +        value = aplic_msi_target_gen(target_vcpu, value);
>>> +
>>> +        break;
>>> +    }
>>> +
>>> +    case APLIC_SETIPNUM:
>>> +    case APLIC_SETIPNUM_LE:
>>> +    case APLIC_CLRIPNUM:
>>> +    case APLIC_SETIENUM:
>>> +    case APLIC_CLRIENUM:
>>> +        if ( !value || !AUTH_IRQ_BIT(d, value) )
>>> +            return 0;
>>> +
>>> +        break;
>>> +
>>> +    case APLIC_DOMAINCFG:
>>> +    {
>>> +        struct vaplic *vaplic = to_vaplic(v->domain);
>>> +
>>> +        /*
>>> +         * The domaincfg register has this format:
>>> +         * bits 31:24 read-only 0x80
>>> +         * bit 8      IE
>>> +         * bit 7      read-only 0
>>> +         * bit 2      DM (WARL)
>>> +         * bit 0      BE (WARL)
>>> +         *
>>> +         * The most interesting bit for us is IE(Interrupt Enable) bit.
>>> +         * At the moment, at least, Linux doesn't use domaincfg.IE bit to
>>> +         * disable interrupts globally, but if one day someone will use it
>>> +         * then extra actions should be done.
>>> +         *
>>> +         * Only DM (bit 2) and IE (bit 8) are writable here. They are assigned
>>> +         * (not OR-ed) so that a write of 0 can also clear them (WARL), and the
>>> +         * read-only high byte (0x80) is always kept set on read-back.
>>> +         */
>>> +        if ( value & ~(APLIC_DOMAINCFG_RO | APLIC_DOMAINCFG_DM |
>>> +                       APLIC_DOMAINCFG_IE) )
>>> +            printk_once("%s: Ignore writes to non-writable domaincfg bits as "
>>> +                        "they are set by aplic during initialization in Xen\n",
>>> +                        __func__);
>>> +
>>> +        vaplic->regs.domaincfg = APLIC_DOMAINCFG_RO |
>>> +                                 (value & (APLIC_DOMAINCFG_DM |
>>> +                                           APLIC_DOMAINCFG_IE));
>>> +
>>> +        return 0;
>>> +    }
>>> +
>>> +    default:
>>> +        goto fail;
>>
>> Instead of this goto, I think you simply want to move the label here.
>> That'll also make the function more similar to its load counterpart.
> 
> Good point. I am curious how fail label should be aligned:
> 
>      default:
>   fail:
>          gdprintk(XENLOG_WARNING,
>                   "Unhandled APLIC write at offset %#x (value %#x)\n", 
> offset,
>                   value);
> 
>          return rc;
>      }
> 
> or default:
>      fail:
> 
> ?

Neither. Labels inside switch() should be indented to same as the
case labels there.

>>> @@ -105,6 +356,50 @@ static const struct vintc_init_ops __initconstrel init_ops = {
>>>       .make_domu_dt_node = vaplic_make_domu_dt_node,
>>>   };
>>>   
>>> +static enum io_state cf_check vaplic_mmio_read(struct vcpu *v, mmio_info_t *info,
>>> +                                               register_t *r)
>>> +{
>>> +    uint32_t data = 0;
>>> +
>>> +    if ( info->len != sizeof(uint32_t) ||
>>> +         !IS_ALIGNED(info->gpa, sizeof(uint32_t)) )
>>> +    {
>>> +        gdprintk(XENLOG_DEBUG,
>>> +                 "VAPLIC: unaligned/wrong-width read gpa=%"PRIpaddr" len=%u\n",
>>> +                 info->gpa, info->len);
>>
>> You have v passed in here, but you'd log current. If passing in v is
>> necessary (i.e. here or elsewhere it may be other than current), then you
>> need to either ASSERT(v == current) at the top of the funciton or otherwise
>> handle v != current correctly.
> 
> It makes sense. I will add ASSERT(v == current) here and for 
> vaplic_mmio_write().

And then further rename the parameter to "curr", please.

Jan

Re: [PATCH v1 05/17] xen/riscv: implement virtual APLIC MMIO emulation

[Oleksii Kurochko]

On 8/12/26 11:10 AM, Jan Beulich wrote:
> On 07.08.2026 18:08, Oleksii Kurochko wrote:
>> On 8/6/26 4:28 PM, Jan Beulich wrote:
>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>> Guests running under Xen program interrupt routing by writing to APLIC
>>>> MMIO registers. Xen must intercept these accesses to enforce interrupt
>>>> isolation between domains and to translate guest routing intent into the
>>>> underlying physical MSI topology.
>>>>
>>>> Writes are gated by the domain's authorised interrupt bitmap so that a
>>>> guest cannot affect interrupts it does not own. TARGET register writes
>>>> additionally require translation of the hart and IMSIC guest-file
>>>> indices from virtual to physical, as the APLIC uses these fields
>>>> directly to compute the MSI delivery address.
>>>>
>>>> Delegation (APLIC_SOURCECFG_D) is not yet supported.
>>>>
>>>> Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
>>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>>> ---
>>>> Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech> # vaplic_mmio_{read,write}
>>>
>>> For this tag to have any meaning, it should move ahead of the --- above;
>>> the explanations ...
>>>
>>>> The downstream changes related to `vaplic_mmio_{read,write}` were originally
>>>> in a separate patch (which was reviewed by Baptiste). However, before
>>>> upstreaming, it was decided to merge them into the current patch.
>>>> I added `Reviewed-by: Baptiste` in this form for now, but Baptiste will
>>>> probably review the remaining changes as well.
>>>> Once that happens, I'll simply move the `Reviewed-by` tag up and
>>>> remove the `#`.
>>>
>>> ... here rather explain the restriction on the R-b, not its odd placement.
>>>
>>>> ---
>>>> Changes in v3:
>>>
>>> As this looks to be recurring - please get versioning of your series right.
>>> The series is supposedly v1, but here you give the impression of it being
>>> v3. If there really was an earlier v2 posting, why isn't the entire series
>>> here v3?
>>
>> It is v3 before before it was a part of another patch series connected
>> to dom0less config enablement.
>>
>> Would it be better to just write in "Change in v3" that it is moved from
>> another patch series + link to that patch series? Or it will be enough
>> just to drop "Changes in v2 and v1" and just start from v1?
> 
> Which part of "never have versions go backwards" was unclear in my earlier
> reply?

Sorry but from your initail reponse it wasn't clear that "never have 
versions go backwards". Now it is clear, thanks for clarification.

> 
>>>> --- a/xen/arch/riscv/aplic-priv.h
>>>> +++ b/xen/arch/riscv/aplic-priv.h
>>>> @@ -48,4 +48,6 @@ struct aplic_priv {
>>>>     */
>>>>    extern unsigned int guest_aplic_num_sources;
>>>>    
>>>> +uint32_t aplic_msi_target_gen(const struct vcpu *target_vcpu, uint32_t base_val);
>>>
>>> PLease can you, before submitting, self-review your patches? I'm really
>>> getting tired of having to repeatedly point out basic style issues, like
>>> the overlong line here.
>>
>> Sorry for that, I will write an extra checker for such cases to not miss
>> them.
> 
> Well, if there was a checker, many more people would like to use it.>
>>>> @@ -38,6 +39,60 @@ static struct intc_info __ro_after_init aplic_info = {
>>>>        .hw_variant = INTC_APLIC,
>>>>    };
>>>>    
>>>> +static unsigned long aplic_hart_field(unsigned long hartid)
>>>> +{
>>>> +    const struct imsic_config *imsic = imsic_get_config();
>>>> +    unsigned int lhxw = imsic->hart_index_bits;
>>>> +    unsigned int hhxw = imsic->group_index_bits;
>>>
>>> It extends to the other local variables here, but I'll use these two to
>>> try to make my point: I'm struggling to associate the names with the
>>> values they are set to. Likely "hxw" is an abbreviation of hart index
>>> width, but (a) what's the leading 'l' then and (b) why is there no 'g'
>>> in "hhxw"? By using hard to grasp names, you make it hard to actually
>>> understand the subsequent expressions, in particular ...
>>
>> The names it taken directly from AIA spec:
>>
>> The use of this value and fields HHXS (High Hart Index Shift), LHXS (Low
>> Hart Index Shift), HHXW (High Hart Index Width), and LHXW (Low Hart
>> Index Width) for determining target addresses for MSIs is described
>> later, in Section 4.9.1.
>>
>> The AIA specification interprets the machine-level hart index as a
>> combination of the **group index** (`g`) and the **hart index within the
>> group** (`h`), according to the following formulas:
>>
>> ```
>> (1) g = (machine-level hart index >> LHXW) & (2^HHXW − 1)
>> (2) h = machine-level hart index & (2^LHXW − 1)
>> ```
>>
>> (In our case, the machine-level hart index is equal to `mhartid`, i.e.
>> the hart index.)
>>
>> For systems that use IMSIC groups, the IMSIC address layout is defined
>> by the following parameters:
>>
>> * `lhxw` (Low Hart Index Width, or *k*): the number of bits used for the
>> hart number within a group.
>> * `hhxw` (High Hart Index Width, or *j*): the number of bits used for
>> the group number.
>> * `hhxs` (High Hart Index Shift): the bit offset of the combined
>> hart/group index field within the physical address.
>>
>> To extract the group index, we first shift the address by `hhxs` so that
>> the group index bits are aligned, and then apply a mask derived from
>> `hhxw` to isolate those bits.
>>
>> The hardware performs the same operation to extract the hart index from
>> the MSI address. However, in our case we already know which hart should
>> receive the interrupt (`hartid`), so there is no need to extract the
>> hart index from the base address. We only need to recover the group
>> index and combine it with `hartid` to construct the value expected by
>> the `target` register.
>>
>>>
>>>> +    unsigned int hhxs =
>>>> +        imsic->group_index_shift - APLIC_xMSICFGADDR_PPN_SHIFT * 2;
>>>> +    unsigned long tppn =
>>>> +        imsic->msi[hartid].base_addr >> APLIC_xMSICFGADDR_PPN_SHIFT;
>>>> +    unsigned long group_index =
>>>> +        (tppn >> APLIC_xMSICFGADDR_PPN_HHX_SHIFT(hhxs)) &
>>>> +        APLIC_xMSICFGADDR_PPN_HHX_MASK(hhxw);
>>>> +
>>>> +    return (group_index << lhxw) | hartid;
>>>
>>> ... these last two. As it stands, they may be easier to understand if
>>> you didn't have the local variables at all, despite them then getting
>>> textually longer.
>>
>> With the explanation above, do the variable names make sense?
> 
> Yes and ...
> 
>> To be closer to AIA spec I think it would be better to rename
>> group_index to g and hart_id to h. Does it make sense to you?
> 
> ... yes. Question is whether you want to help readers who aren't that
> familiar with the AIA spec. If so, maybe add [brief] comments making
> clear what the names say? E.g.
> 
>      /* High Hart Index Shift */
>      unsigned int hhxs =
>          imsic->group_index_shift - APLIC_xMSICFGADDR_PPN_SHIFT * 2;

Good idea with comments. Also, I will apply the comment I suggested in 
reply to one of Baptiste questions which will also provide extra 
information which should help.

> 
>>>> --- a/xen/arch/riscv/include/asm/aplic.h
>>>> +++ b/xen/arch/riscv/include/asm/aplic.h
>>>> @@ -28,6 +28,8 @@
>>>>    #define APLIC_DOMAINCFG_BE      BIT(0, U)
>>>>    
>>>>    /* sourcecfg register fields */
>>>> +#define APLIC_SOURCECFG_D       BIT(10, U)
>>>
>>> As to the comment - this indeed looks to be a field, but ...
>>>
>>>>    #define APLIC_SOURCECFG_SM_INACTIVE     0x0
>>>>    #define APLIC_SOURCECFG_SM_DETACH       0x1
>>>>    #define APLIC_SOURCECFG_SM_EDGE_RISE    0x4
>>>
>>> ... these look to be values of some other field which isn't described. Please
>>> may I (again) ask that definitions are their commentary at the very least not
>>> misguide readers?
>>
>> Thanks for pointing this out. You're right, the comment is misleading as
>> written. APLIC_SOURCECFG_D is a field, whereas the APLIC_SOURCECFG_SM_*
>> definitions are values for the source mode (SM) field, and the comment
>> doesn't make that distinction.
>>
>> I'll update the comments to describe the fields more accurately:
>>
>> #define APLIC_SOURCECFG_BASE            0x0004
>> #define APLIC_SOURCECFG_LAST            0x0ffc
>> /*
>>    * sourcecfg[] register fields:
>>    *  - bit 10 (D) selects the layout of the remaining bits;
>>    *  - D = 1: bits [9:0] hold the Child Index, i.e. the source is delegated
>>    *           to a child domain (unsupported by Xen);
>>    *  - D = 0: bits [2:0] hold the source mode SM (WARL).
>>    */
>> #define  APLIC_SOURCECFG_D              BIT(10, U)
>> /* SM field values (0x2 and 0x3 are reserved): */
>> #define   APLIC_SOURCECFG_SM_INACTIVE   0x0
>> #define   APLIC_SOURCECFG_SM_DETACH     0x1
>> #define   APLIC_SOURCECFG_SM_EDGE_RISE  0x4
>> #define   APLIC_SOURCECFG_SM_EDGE_FALL  0x5
>> #define   APLIC_SOURCECFG_SM_LEVEL_HIGH 0x6
>> #define   APLIC_SOURCECFG_SM_LEVEL_LOW  0x7
>>
>> Does it look better? Probably there is not sense for two extra spaces
>> for APLIC_SOURCECFG_SM_*. I want to show by such identation that it is
>> values for SM field of APLIC_SOURCECFG.
> 
> Which is fine. All you need to add then is a field definition for the SM
> field. Then the extra padding blank will also start to make sense.

Sure, I will do then that.

>>>> --- a/xen/arch/riscv/vaplic.c
>>>> +++ b/xen/arch/riscv/vaplic.c
>>>> @@ -17,6 +17,7 @@
>>>>    #include <asm/aia.h>
>>>>    #include <asm/imsic.h>
>>>>    #include <asm/intc.h>
>>>> +#include <asm/mmio.h>
>>>>    #include <asm/vaplic.h>
>>>>    
>>>>    #include "aplic-priv.h"
>>>> @@ -27,6 +28,256 @@ unsigned int __ro_after_init guest_aplic_num_sources;
>>>>    
>>>>    #define FDT_VAPLIC_INT_CELLS 2
>>>>    
>>>> +#define AUTH_IRQ_BIT(d, irqn) ( \
>>>> +    ((irqn) < (d)->arch.vintc->nr_virqs) && \
>>>> +    test_bit(irqn, (d)->arch.vintc->used_irqs) )
>>>
>>> Nit: Indentation.
>>
>> I will use the following indentation:
>>
>> ... (((irqn) < (d)->arch.vintc->nr_virqs) && \
>>        test_bit(irqn, (d)->arch.vintc->used_irqs))
> 
> Which as written still doesn't look right. What I can't tell is whether
> that's merely because of the use of "...".
> 
> Of the three opening prarens on the first line, two have their closing
> counterparts on the same line. There's thus one pending closing paren,
> meaning there should be one extra indenting blank.

To be more precise:

#define AUTH_IRQ_BIT(d, irqn) \
     (((irqn) < (d)->arch.vintc->nr_virqs) && \
      test_bit(irqn, (d)->arch.vintc->used_irqs))

so test_bit(...) is shifted by one indenting blank to be inisde the first (.

> 
>>>> +/*
>>>> + * Convert a byte offset (within a SETIP/CLRIP/SETIE/CLRIE register group) to
>>>> + * a 32-bit word index into the allocated_irqs bitmap.  Each word covers 32
>>>> + * interrupt sources.  For SOURCECFG and TARGET groups the same division also
>>>> + * yields the interrupt number directly, because those arrays store one 32-bit
>>>> + * register per source.
>>>> + */
>>>> +#define regoffset_to_word_idx(reg_val) ((reg_val) / sizeof(uint32_t))
>>>> +
>>>> +static inline uint32_t generate_auth_mask(const struct domain *d,
>>>> +                                          unsigned int word_idx)
>>>> +{
>>>> +    unsigned int first_bit = word_idx * sizeof(uint32_t) * BITS_PER_BYTE;
>>>> +
>>>> +    if ( word_idx >= DIV_ROUND_UP(d->arch.vintc->nr_virqs,
>>>> +                                  sizeof(uint32_t) * BITS_PER_BYTE) )
>>>> +    {
>>>> +        dprintk(XENLOG_DEBUG, "incorrect word_idx(%u) is passed\n", word_idx);
>>>
>>> Is this really meant to stay?
>>
>> For debug purpose it could be useful, so I prefer to have it with
>> changing it to gprintk(XENLOG_DEBUG, ...) to understand which domain is
>> trying to access something wrong.
> 
> gdprintk() implies you're on the vCPU that's the subject of the operation.
> If that's always the case here, the function parameter wants to reflect
> that as far as possible: "currd" instead of "d".

I will use currd. Then it also makes sense to add ASSERT(v == current) 
in vaplic_emulate_{store,load}().

> 
>>>> +        return 0U;
>>>> +    }
>>>> +
>>>> +    return (uint32_t)(d->arch.vintc->used_irqs[first_bit / BITS_PER_LONG] >>
>>>> +                      (first_bit % BITS_PER_LONG));
>>>
>>> I don't quite understand the need for the cast.
>>
>> Functionally it isn't need but it documents that it is expected that
>> translation from unsinged long  to uint32_t will happen. I will drop the
>> cast.
> 
> Thanks. If you really wanted such doc, casts would need adding in many
> more places across the code base.
> 
>>>> +        if ( !target_vcpu )
>>>> +        {
>>>> +            dprintk(XENLOG_ERR, "Invalid vCPU id in target register\n");
>>>> +
>>>> +            /* Ignore such writings */
>>>> +            return 0;
>>>> +        }
>>>> +
>>>> +        value = aplic_msi_target_gen(target_vcpu, value);
>>>> +
>>>> +        break;
>>>> +    }
>>>> +
>>>> +    case APLIC_SETIPNUM:
>>>> +    case APLIC_SETIPNUM_LE:
>>>> +    case APLIC_CLRIPNUM:
>>>> +    case APLIC_SETIENUM:
>>>> +    case APLIC_CLRIENUM:
>>>> +        if ( !value || !AUTH_IRQ_BIT(d, value) )
>>>> +            return 0;
>>>> +
>>>> +        break;
>>>> +
>>>> +    case APLIC_DOMAINCFG:
>>>> +    {
>>>> +        struct vaplic *vaplic = to_vaplic(v->domain);
>>>> +
>>>> +        /*
>>>> +         * The domaincfg register has this format:
>>>> +         * bits 31:24 read-only 0x80
>>>> +         * bit 8      IE
>>>> +         * bit 7      read-only 0
>>>> +         * bit 2      DM (WARL)
>>>> +         * bit 0      BE (WARL)
>>>> +         *
>>>> +         * The most interesting bit for us is IE(Interrupt Enable) bit.
>>>> +         * At the moment, at least, Linux doesn't use domaincfg.IE bit to
>>>> +         * disable interrupts globally, but if one day someone will use it
>>>> +         * then extra actions should be done.
>>>> +         *
>>>> +         * Only DM (bit 2) and IE (bit 8) are writable here. They are assigned
>>>> +         * (not OR-ed) so that a write of 0 can also clear them (WARL), and the
>>>> +         * read-only high byte (0x80) is always kept set on read-back.
>>>> +         */
>>>> +        if ( value & ~(APLIC_DOMAINCFG_RO | APLIC_DOMAINCFG_DM |
>>>> +                       APLIC_DOMAINCFG_IE) )
>>>> +            printk_once("%s: Ignore writes to non-writable domaincfg bits as "
>>>> +                        "they are set by aplic during initialization in Xen\n",
>>>> +                        __func__);
>>>> +
>>>> +        vaplic->regs.domaincfg = APLIC_DOMAINCFG_RO |
>>>> +                                 (value & (APLIC_DOMAINCFG_DM |
>>>> +                                           APLIC_DOMAINCFG_IE));
>>>> +
>>>> +        return 0;
>>>> +    }
>>>> +
>>>> +    default:
>>>> +        goto fail;
>>>
>>> Instead of this goto, I think you simply want to move the label here.
>>> That'll also make the function more similar to its load counterpart.
>>
>> Good point. I am curious how fail label should be aligned:
>>
>>       default:
>>    fail:
>>           gdprintk(XENLOG_WARNING,
>>                    "Unhandled APLIC write at offset %#x (value %#x)\n",
>> offset,
>>                    value);
>>
>>           return rc;
>>       }
>>
>> or default:
>>       fail:
>>
>> ?
> 
> Neither. Labels inside switch() should be indented to same as the
> case labels there.

thanks for clarifying that.

> 
>>>> @@ -105,6 +356,50 @@ static const struct vintc_init_ops __initconstrel init_ops = {
>>>>        .make_domu_dt_node = vaplic_make_domu_dt_node,
>>>>    };
>>>>    
>>>> +static enum io_state cf_check vaplic_mmio_read(struct vcpu *v, mmio_info_t *info,
>>>> +                                               register_t *r)
>>>> +{
>>>> +    uint32_t data = 0;
>>>> +
>>>> +    if ( info->len != sizeof(uint32_t) ||
>>>> +         !IS_ALIGNED(info->gpa, sizeof(uint32_t)) )
>>>> +    {
>>>> +        gdprintk(XENLOG_DEBUG,
>>>> +                 "VAPLIC: unaligned/wrong-width read gpa=%"PRIpaddr" len=%u\n",
>>>> +                 info->gpa, info->len);
>>>
>>> You have v passed in here, but you'd log current. If passing in v is
>>> necessary (i.e. here or elsewhere it may be other than current), then you
>>> need to either ASSERT(v == current) at the top of the funciton or otherwise
>>> handle v != current correctly.
>>
>> It makes sense. I will add ASSERT(v == current) here and for
>> vaplic_mmio_write().
> 
> And then further rename the parameter to "curr", please.

Applied this.

Thanks.

~ Oleksii

Re: [PATCH v1 05/17] xen/riscv: implement virtual APLIC MMIO emulation

[Jan Beulich]

On 12.08.2026 13:51, Oleksii Kurochko wrote:
> On 8/12/26 11:10 AM, Jan Beulich wrote:
>> On 07.08.2026 18:08, Oleksii Kurochko wrote:
>>> On 8/6/26 4:28 PM, Jan Beulich wrote:
>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>> +/*
>>>>> + * Convert a byte offset (within a SETIP/CLRIP/SETIE/CLRIE register group) to
>>>>> + * a 32-bit word index into the allocated_irqs bitmap.  Each word covers 32
>>>>> + * interrupt sources.  For SOURCECFG and TARGET groups the same division also
>>>>> + * yields the interrupt number directly, because those arrays store one 32-bit
>>>>> + * register per source.
>>>>> + */
>>>>> +#define regoffset_to_word_idx(reg_val) ((reg_val) / sizeof(uint32_t))
>>>>> +
>>>>> +static inline uint32_t generate_auth_mask(const struct domain *d,
>>>>> +                                          unsigned int word_idx)
>>>>> +{
>>>>> +    unsigned int first_bit = word_idx * sizeof(uint32_t) * BITS_PER_BYTE;
>>>>> +
>>>>> +    if ( word_idx >= DIV_ROUND_UP(d->arch.vintc->nr_virqs,
>>>>> +                                  sizeof(uint32_t) * BITS_PER_BYTE) )
>>>>> +    {
>>>>> +        dprintk(XENLOG_DEBUG, "incorrect word_idx(%u) is passed\n", word_idx);
>>>>
>>>> Is this really meant to stay?
>>>
>>> For debug purpose it could be useful, so I prefer to have it with
>>> changing it to gprintk(XENLOG_DEBUG, ...) to understand which domain is
>>> trying to access something wrong.
>>
>> gdprintk() implies you're on the vCPU that's the subject of the operation.
>> If that's always the case here, the function parameter wants to reflect
>> that as far as possible: "currd" instead of "d".
> 
> I will use currd. Then it also makes sense to add ASSERT(v == current) 
> in vaplic_emulate_{store,load}().

ASSERT(curr == current), that is.

Jan

Re: [PATCH v1 05/17] xen/riscv: implement virtual APLIC MMIO emulation

[Baptiste Le Duc]

> Guests running under Xen program interrupt routing by writing to APLIC
> MMIO registers. Xen must intercept these accesses to enforce interrupt
> isolation between domains and to translate guest routing intent into the
> underlying physical MSI topology.
> 
> Writes are gated by the domain's authorised interrupt bitmap so that a
> guest cannot affect interrupts it does not own. TARGET register writes
> additionally require translation of the hart and IMSIC guest-file


> indices from virtual to physical, as the APLIC uses these fields
> directly to compute the MSI delivery address.
> 
> Delegation (APLIC_SOURCECFG_D) is not yet supported.


> 
> Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>


















>
> diff --git a/xen/arch/riscv/aplic-priv.h b/xen/arch/riscv/aplic-priv.h
> index 1391837f89..96bc56dbe5 100644
> --- a/xen/arch/riscv/aplic-priv.h
> +++ b/xen/arch/riscv/aplic-priv.h
> @@ -48,4 +48,6 @@ struct aplic_priv {
>   */
>  extern unsigned int guest_aplic_num_sources;
>  
> +uint32_t aplic_msi_target_gen(const struct vcpu *target_vcpu, uint32_t base_val);


> +
>  #endif /* ASM_RISCV_APLIC_PRIV_H */
> diff --git a/xen/arch/riscv/aplic.c b/xen/arch/riscv/aplic.c
> index 3681f0669e..87f2134bc5 100644
> --- a/xen/arch/riscv/aplic.c
> +++ b/xen/arch/riscv/aplic.c
> @@ -16,6 +16,7 @@
>  #include <xen/irq.h>
>  #include <xen/mm.h>
>  #include <xen/sections.h>
> +#include <xen/sched.h>
>  #include <xen/spinlock.h>
>  #include <xen/types.h>
>  #include <xen/vmap.h>
> @@ -38,6 +39,60 @@ static struct intc_info __ro_after_init aplic_info = {
>      .hw_variant = INTC_APLIC,
>  };
>  
> +static unsigned long aplic_hart_field(unsigned long hartid)


> +{
> +    const struct imsic_config *imsic = imsic_get_config();
> +    unsigned int lhxw = imsic->hart_index_bits;
> +    unsigned int hhxw = imsic->group_index_bits;
> +    unsigned int hhxs =
> +        imsic->group_index_shift - APLIC_xMSICFGADDR_PPN_SHIFT * 2;
> +    unsigned long tppn =
> +        imsic->msi[hartid].base_addr >> APLIC_xMSICFGADDR_PPN_SHIFT;
> +    unsigned long group_index =
> +        (tppn >> APLIC_xMSICFGADDR_PPN_HHX_SHIFT(hhxs)) &
> +        APLIC_xMSICFGADDR_PPN_HHX_MASK(hhxw);
> +
> +    return (group_index << lhxw) | hartid;
> +}
> +
> +uint32_t aplic_msi_target_gen(const struct vcpu *target_vcpu, uint32_t base_val)
> +{
> +    unsigned int guest_id = vcpu_guest_file_id(target_vcpu);
> +    unsigned long hart_id = cpuid_to_hartid(target_vcpu->processor);
> +    unsigned long hart_field = aplic_hart_field(hart_id);
> +
> +    base_val &= APLIC_TARGET_EIID_MASK;
> +    base_val |= MASK_INSR(guest_id, APLIC_TARGET_GUEST_IDX_MASK);
> +    base_val |= MASK_INSR(hart_field, APLIC_TARGET_HART_IDX_MASK);
> +
> +    return base_val;
> +}
> +
> +uint32_t aplic_hw_read_reg(unsigned int offset, uint32_t mask)
> +{
> +    unsigned long flags;
> +    uint32_t val;
> +
> +    ASSERT((offset < aplic.size) && IS_ALIGNED(offset, sizeof(uint32_t)));
> +
> +    spin_lock_irqsave(&aplic.lock, flags);
> +    val = readl((volatile void __iomem *)aplic.regs + offset) & mask;
> +    spin_unlock_irqrestore(&aplic.lock, flags);
> +
> +    return val;
> +}
> +
> +void aplic_hw_write_reg(unsigned int offset, uint32_t value)
> +{
> +    unsigned long flags;
> +
> +    ASSERT((offset < aplic.size) && IS_ALIGNED(offset, sizeof(uint32_t)));
> +
> +    spin_lock_irqsave(&aplic.lock, flags);
> +    writel(value, (volatile void __iomem *)aplic.regs + offset);
> +    spin_unlock_irqrestore(&aplic.lock, flags);
> +}
> +
>  static void __init aplic_init_hw_interrupts(void)
>  {
>      unsigned int i;
> diff --git a/xen/arch/riscv/include/asm/aplic.h b/xen/arch/riscv/include/asm/aplic.h
> index f22622b9a2..4ae5fb8f26 100644
> --- a/xen/arch/riscv/include/asm/aplic.h
> +++ b/xen/arch/riscv/include/asm/aplic.h
> @@ -28,6 +28,8 @@
>  #define APLIC_DOMAINCFG_BE      BIT(0, U)
>  
>  /* sourcecfg register fields */
> +#define APLIC_SOURCECFG_D       BIT(10, U)
> +
>  #define APLIC_SOURCECFG_SM_INACTIVE     0x0
>  #define APLIC_SOURCECFG_SM_DETACH       0x1
>  #define APLIC_SOURCECFG_SM_EDGE_RISE    0x4
> @@ -38,6 +40,16 @@
>  /* target register fields */
>  #define APLIC_TARGET_HART_IDX_SHIFT 18
>  #define APLIC_TARGET_EIID_MASK      0x7ff
> +#define APLIC_TARGET_HART_IDX_MASK  0xfffc0000
> +#define APLIC_TARGET_GUEST_IDX_MASK 0x3f000
> +
> +/* xmsicfgaddr/h register fields */
> +#define APLIC_xMSICFGADDR_PPN_SHIFT IMSIC_MMIO_PAGE_SHIFT
> +
> +#define APLIC_xMSICFGADDR_PPN_HHX_MASK(hhxw) \
> +    (BIT(hhxw, UL) - 1)
> +#define APLIC_xMSICFGADDR_PPN_HHX_SHIFT(hhxs) \
> +    ((hhxs) + APLIC_xMSICFGADDR_PPN_SHIFT)
>  
>  #define APLIC_DOMAINCFG         0x0000
>  #define APLIC_SOURCECFG_BASE    0x0004
> @@ -77,6 +89,15 @@
>  #define APLIC_SIZE(nr_cpus)     (APLIC_MIN_SIZE + \
>                                   APLIC_SIZE_ALIGN(APLIC_IDC_SIZE * (nr_cpus)))
>  
> +/*
> + * Using setip is fine here, as all SET* and CLR* register groups consist of 32
> + * registers and therefore have identical sizes.
> + *
> + * Lowest 2 bits are always zero for SET* and CLR* registers.
> + */
> +#define APLIC_SETCLR_OFFSET_MASK \
> +    (sizeof_field(struct aplic_regs, setip) - sizeof(uint32_t))
> +
>  struct aplic_regs {
>      uint32_t domaincfg;         /* 0x0000 */
>      uint32_t sourcecfg[1023];   /* 0x0004 */
> @@ -120,4 +141,7 @@ struct aplic_regs {
>      uint32_t target[1023];      /* 0x3008 */
>  };
>  
> +uint32_t aplic_hw_read_reg(unsigned int offset, uint32_t mask);
> +void aplic_hw_write_reg(unsigned int offset, uint32_t value);
> +
>  #endif /* ASM_RISCV_APLIC_H */
> diff --git a/xen/arch/riscv/include/asm/imsic.h b/xen/arch/riscv/include/asm/imsic.h
> index e1ec3d03c4..612f503b57 100644
> --- a/xen/arch/riscv/include/asm/imsic.h
> +++ b/xen/arch/riscv/include/asm/imsic.h
> @@ -40,6 +40,16 @@ struct imsic_config {
>      /* Base address */
>      paddr_t base_addr;
>  
> +    /*
> +     * MSI Target Address Scheme
> +     *
> +     * XLEN-1                                                12     0
> +     * |                                                     |     |
> +     * -------------------------------------------------------------
> +     * |xxxxxx|Group Index|xxxxxxxxxxx|HART Index|Guest Index|  0  |
> +     * -------------------------------------------------------------
> +     */
> +
>      /* Bits representing Guest index, HART index, and Group index */
>      unsigned int guest_index_bits;
>      unsigned int hart_index_bits;
> diff --git a/xen/arch/riscv/include/asm/vaplic.h b/xen/arch/riscv/include/asm/vaplic.h
> index 96080bfbc2..7bf9247f4e 100644
> --- a/xen/arch/riscv/include/asm/vaplic.h
> +++ b/xen/arch/riscv/include/asm/vaplic.h
> @@ -26,6 +26,9 @@ struct vaplic_regs {
>  struct vaplic {
>      struct vintc vintc;
>      struct vaplic_regs regs;
> +
> +    paddr_t regs_start;
> +    unsigned int regs_size;
>  };
>  
>  int domain_vaplic_init(struct domain *d);
> diff --git a/xen/arch/riscv/vaplic.c b/xen/arch/riscv/vaplic.c
> index b07b4aa4d3..a09a720d68 100644
> --- a/xen/arch/riscv/vaplic.c
> +++ b/xen/arch/riscv/vaplic.c
> @@ -17,6 +17,7 @@
>  #include <asm/aia.h>
>  #include <asm/imsic.h>
>  #include <asm/intc.h>
> +#include <asm/mmio.h>
>  #include <asm/vaplic.h>
>  
>  #include "aplic-priv.h"
> @@ -27,6 +28,256 @@ unsigned int __ro_after_init guest_aplic_num_sources;
>  
>  #define FDT_VAPLIC_INT_CELLS 2
>  
> +#define AUTH_IRQ_BIT(d, irqn) ( \
> +    ((irqn) < (d)->arch.vintc->nr_virqs) && \
> +    test_bit(irqn, (d)->arch.vintc->used_irqs) )
> +
> +/*
> + * Convert a byte offset (within a SETIP/CLRIP/SETIE/CLRIE register group) to
> + * a 32-bit word index into the allocated_irqs bitmap.  Each word covers 32
> + * interrupt sources.  For SOURCECFG and TARGET groups the same division also
> + * yields the interrupt number directly, because those arrays store one 32-bit
> + * register per source.
> + */
> +#define regoffset_to_word_idx(reg_val) ((reg_val) / sizeof(uint32_t))
> +
> +static inline uint32_t generate_auth_mask(const struct domain *d,
> +                                          unsigned int word_idx)
> +{
> +    unsigned int first_bit = word_idx * sizeof(uint32_t) * BITS_PER_BYTE;
> +
> +    if ( word_idx >= DIV_ROUND_UP(d->arch.vintc->nr_virqs,
> +                                  sizeof(uint32_t) * BITS_PER_BYTE) )
> +    {
> +        dprintk(XENLOG_DEBUG, "incorrect word_idx(%u) is passed\n", word_idx);
> +
> +        return 0U;
> +    }
> +
> +    return (uint32_t)(d->arch.vintc->used_irqs[first_bit / BITS_PER_LONG] >>
> +                      (first_bit % BITS_PER_LONG));
> +}
> +
> +static int cf_check vaplic_emulate_load(const struct vcpu *v,


> +                                        const unsigned long addr,
> +                                        uint32_t *out)
> +{
> +    const struct domain *d = v->domain;
> +    const struct vaplic *vaplic = to_vaplic(d);
> +    const unsigned int offset = addr & APLIC_REG_OFFSET_MASK;


> +    uint32_t auth_mask;
> +    unsigned int i;
> +
> +    switch ( offset )
> +    {
> +    case APLIC_DOMAINCFG:
> +        *out = vaplic->regs.domaincfg;
> +
> +        return 0;
> +
> +    case APLIC_SETIPNUM:
> +    case APLIC_SETIPNUM_LE:
> +    case APLIC_CLRIPNUM:
> +    case APLIC_SETIENUM:
> +    case APLIC_CLRIENUM:
> +    case APLIC_CLRIE_BASE ... APLIC_CLRIE_LAST:
> +        /*
> +         * Based on the RISC-V AIA spec a read of these registers
> +         * always returns zero
> +         */
> +        *out = 0;
> +
> +        return 0;
> +
> +    case APLIC_SETIP_BASE ... APLIC_SETIP_LAST:
> +    case APLIC_CLRIP_BASE ... APLIC_CLRIP_LAST:
> +    case APLIC_SETIE_BASE ... APLIC_SETIE_LAST:
> +        i = regoffset_to_word_idx(offset & APLIC_SETCLR_OFFSET_MASK);
> +        auth_mask = generate_auth_mask(d, i);
> +
> +        break;
> +
> +    case APLIC_TARGET_BASE ... APLIC_TARGET_LAST:
> +        /*
> +         * As target registers start from 1:
> +         *  0x3000 genmsi
> +         *  0x3004 target[1]
> +         *  0x3008 target[2]
> +         *   ...
> +         *  0x3FFC target[1023]
> +         * It is necessary to calculate an interrupt number by subtracting
> +         * APLIC_GENMSI instead of APLIC_TARGET_BASE.
> +         */
> +        i = regoffset_to_word_idx(offset - APLIC_GENMSI);
> +
> +        if ( !AUTH_IRQ_BIT(d, i) )
> +        {
> +            *out = 0;
> +
> +            return 0;
> +        }
> +
> +        auth_mask = ~0U;
> +
> +        break;
> +
> +    default:
> +        gdprintk(XENLOG_WARNING, "Unhandled APLIC read at offset %#x\n",
> +                 offset);
> +
> +        return -EINVAL;
> +    }
> +
> +    *out = aplic_hw_read_reg(offset, auth_mask);

I think there is a problem here for the target registers: a read does not
return what the guest wrote.

Consider domU calling request_irq() for source 10, with the interrupt
affinity to vCPU1:

      writel(0x0004000A, GUEST_APLIC_BASE + 0x3028)
      /* hart_idx = 1 (vCPU1), guest_idx = 0, EIID = 10 */

vaplic_emulate_store() passes this through aplic_msi_target_gen(), which
keeps only the EIID and substitutes the physical hart field and the
vCPU's guest interrupt file index, so we write target[10] = 0x001C100A

Therefore, a readl() of the same address returns that raw value (0x001C100A) instead of 0x0004000A, since
auth_mask is ~0U here.

-- 
Baptiste Le Duc <baptiste.le-duc@vates.tech>

Re: [PATCH v1 05/17] xen/riscv: implement virtual APLIC MMIO emulation

[Oleksii Kurochko]

On 8/12/26 4:03 PM, Baptiste Le Duc wrote:
>> +
>> +static int cf_check vaplic_emulate_load(const struct vcpu *v,
> 
> 
>> +                                        const unsigned long addr,
>> +                                        uint32_t *out)
>> +{
>> +    const struct domain *d = v->domain;
>> +    const struct vaplic *vaplic = to_vaplic(d);
>> +    const unsigned int offset = addr & APLIC_REG_OFFSET_MASK;
> 
> 
>> +    uint32_t auth_mask;
>> +    unsigned int i;
>> +
>> +    switch ( offset )
>> +    {
>> +    case APLIC_DOMAINCFG:
>> +        *out = vaplic->regs.domaincfg;
>> +
>> +        return 0;
>> +
>> +    case APLIC_SETIPNUM:
>> +    case APLIC_SETIPNUM_LE:
>> +    case APLIC_CLRIPNUM:
>> +    case APLIC_SETIENUM:
>> +    case APLIC_CLRIENUM:
>> +    case APLIC_CLRIE_BASE ... APLIC_CLRIE_LAST:
>> +        /*
>> +         * Based on the RISC-V AIA spec a read of these registers
>> +         * always returns zero
>> +         */
>> +        *out = 0;
>> +
>> +        return 0;
>> +
>> +    case APLIC_SETIP_BASE ... APLIC_SETIP_LAST:
>> +    case APLIC_CLRIP_BASE ... APLIC_CLRIP_LAST:
>> +    case APLIC_SETIE_BASE ... APLIC_SETIE_LAST:
>> +        i = regoffset_to_word_idx(offset & APLIC_SETCLR_OFFSET_MASK);
>> +        auth_mask = generate_auth_mask(d, i);
>> +
>> +        break;
>> +
>> +    case APLIC_TARGET_BASE ... APLIC_TARGET_LAST:
>> +        /*
>> +         * As target registers start from 1:
>> +         *  0x3000 genmsi
>> +         *  0x3004 target[1]
>> +         *  0x3008 target[2]
>> +         *   ...
>> +         *  0x3FFC target[1023]
>> +         * It is necessary to calculate an interrupt number by subtracting
>> +         * APLIC_GENMSI instead of APLIC_TARGET_BASE.
>> +         */
>> +        i = regoffset_to_word_idx(offset - APLIC_GENMSI);
>> +
>> +        if ( !AUTH_IRQ_BIT(d, i) )
>> +        {
>> +            *out = 0;
>> +
>> +            return 0;
>> +        }
>> +
>> +        auth_mask = ~0U;
>> +
>> +        break;
>> +
>> +    default:
>> +        gdprintk(XENLOG_WARNING, "Unhandled APLIC read at offset %#x\n",
>> +                 offset);
>> +
>> +        return -EINVAL;
>> +    }
>> +
>> +    *out = aplic_hw_read_reg(offset, auth_mask);
> 
> I think there is a problem here for the target registers: a read does not
> return what the guest wrote.
> 
> Consider domU calling request_irq() for source 10, with the interrupt
> affinity to vCPU1:
> 
>        writel(0x0004000A, GUEST_APLIC_BASE + 0x3028)
>        /* hart_idx = 1 (vCPU1), guest_idx = 0, EIID = 10 */
> 
> vaplic_emulate_store() passes this through aplic_msi_target_gen(), which
> keeps only the EIID and substitutes the physical hart field and the
> vCPU's guest interrupt file index, so we write target[10] = 0x001C100A
> 
> Therefore, a readl() of the same address returns that raw value (0x001C100A) instead of 0x0004000A, since
> auth_mask is ~0U here.
> 

I found this issue while working on support for the IMSIC software 
interrupt file. I already have a fix that I need to port to this code. 
However, I completely missed that this was already an issue and that the 
fix should have been ported earlier.

Thanks!

~ Oleksii

[PATCH v1 06/17] xen/riscv: map IMSIC interrupt file for vCPUs

[Oleksii Kurochko]

A guest running in VS-mode expects its own IMSIC S-file at offset 0 of its
guest-physical IMSIC block. Physically, the guest-file (G-file) assigned to
this vCPU lives at a hart-relative offset given by guest_file_id (assigned
via the vGEIN allocator). Therefore, imsic_map_guest_file() uses stage-2
translation to redirect the guest's fixed per-vCPU GPA page (offset 0) to
the specific physical guest-file page.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
The corresponding unmap of the IMSIC interrupt file will be introduced
separately when the need arises.
---
---
 xen/arch/riscv/imsic.c             | 63 ++++++++++++++++++++++++++++++
 xen/arch/riscv/include/asm/imsic.h |  2 +
 2 files changed, 65 insertions(+)

diff --git a/xen/arch/riscv/imsic.c b/xen/arch/riscv/imsic.c
index ffce77209c26..c5ae74e456e8 100644
--- a/xen/arch/riscv/imsic.c
+++ b/xen/arch/riscv/imsic.c
@@ -25,7 +25,9 @@
 #include <xen/spinlock.h>
 #include <xen/xvmalloc.h>
 
+#include <asm/aia.h>
 #include <asm/imsic.h>
+#include <asm/p2m.h>
 
 #define IMSIC_HART_SIZE(guest_bits) (BIT(guest_bits, U) * IMSIC_MMIO_PAGE_SZ)
 
@@ -342,6 +344,67 @@ static int __init imsic_parse_node(const struct dt_device_node *node,
     return 0;
 }
 
+/*
+ * Map the physical IMSIC guest interrupt file (G-file) assigned to vCPU v
+ * into the domain's stage-2 guest-physical address space.
+ *
+ * In the machine's physical address space (SPA), each hart's IMSIC
+ * supervisor-level file (S-file) is located at offset 0 of its address block,
+ * followed contiguously by GEILEN guest files at offsets of 1, 2, ..., N pages.
+ *
+ * Because a guest OS running in VS-mode expects its own supervisor-level
+ * interrupt file to be at offset 0 of its guest-physical IMSIC block, the
+ * hypervisor must use stage-2 address translation to map the vCPU's
+ * guest-physical "supervisor" page (GPA offset 0) to the specific
+ * physical guest file page (SPA offset guest_file_id) on the physical hart.
+ *
+ * Xen pins each vCPU to a pCPU (v->processor) and assigns it a physical
+ * guest file index (guest_file_id) from the vGEIN allocator. A guest_file_id
+ * of 0 indicates that no hardware guest file is selected (matching the
+ * architectural behavior where vGEIN = 0 in the hstatus CSR selects no
+ * guest external interrupt source), requiring the VS-file to be emulated
+ * in software.
+ *
+ * The base guest-physical address advertised to the guest in the device
+ * tree matches offset 0 of the vCPU's virtual IMSIC block. Stage-2
+ * translation ensures that guest supervisor accesses to this page are
+ * transparently routed to the real hardware VS-file granted to it on
+ * the current pCPU.
+ */
+int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id)
+{
+    int res = 0;
+    struct domain *d = v->domain;
+    unsigned int cpu = v->processor;
+    vaddr_t gaddr = imsic_cfg.base_addr + (IMSIC_MMIO_PAGE_SZ * v->vcpu_id);
+    paddr_t paddr;
+    unsigned long guest_stride;
+
+    /* Nothing to map in the case of sw interrupt file. */
+    if ( !vsfile_id )
+        return res;
+
+    guest_stride = vsfile_id * IMSIC_MMIO_PAGE_SZ;
+
+    paddr = imsic_cfg.msi[cpu].base_addr + imsic_cfg.msi[cpu].offset +
+            guest_stride;
+
+#ifdef IMSIC_DEBUG
+    printk("%s: %pv: ga(%#lx) -> pa(%#lx), cpu(%#x), guest_file_id(%d) "
+           "base_addr(%#lx) offset(%#lx)\n", __func__, v, gaddr, paddr, cpu,
+           vsfile_id, imsic_cfg.msi[cpu].base_addr, imsic_cfg.msi[cpu].offset);
+#endif
+
+    res = map_regions_p2mt(d, gaddr_to_gfn(gaddr),
+                           PFN_DOWN(IMSIC_MMIO_PAGE_SZ), maddr_to_mfn(paddr),
+                           arch_dt_passthrough_p2m_type());
+    if ( res )
+        printk("%s: Failed to map %#lx to the guest at %#lx\n",
+               __func__, paddr, gaddr);
+
+    return res;
+}
+
 int vcpu_imsic_init(struct vcpu *v)
 {
     struct vimsic_state *imsic_state;
diff --git a/xen/arch/riscv/include/asm/imsic.h b/xen/arch/riscv/include/asm/imsic.h
index 612f503b5799..f2c649517fd1 100644
--- a/xen/arch/riscv/include/asm/imsic.h
+++ b/xen/arch/riscv/include/asm/imsic.h
@@ -106,4 +106,6 @@ unsigned int vcpu_guest_file_id(const struct vcpu *v);
 
 int vimsic_make_domu_dt_node(struct kernel_info *kinfo, unsigned int *phandle);
 
+int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id);
+
 #endif /* ASM_RISCV_IMSIC_H */
-- 
2.54.0

Re: [PATCH v1 06/17] xen/riscv: map IMSIC interrupt file for vCPUs

[Jan Beulich]

On 20.07.2026 18:02, Oleksii Kurochko wrote:
> A guest running in VS-mode expects its own IMSIC S-file at offset 0 of its
> guest-physical IMSIC block. Physically, the guest-file (G-file) assigned to
> this vCPU lives at a hart-relative offset given by guest_file_id (assigned
> via the vGEIN allocator). Therefore, imsic_map_guest_file() uses stage-2
> translation to redirect the guest's fixed per-vCPU GPA page (offset 0) to
> the specific physical guest-file page.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
> The corresponding unmap of the IMSIC interrupt file will be introduced
> separately when the need arises.

Doesn't the need exist right away? There is ...

> @@ -342,6 +344,67 @@ static int __init imsic_parse_node(const struct dt_device_node *node,
>      return 0;
>  }
>  
> +/*
> + * Map the physical IMSIC guest interrupt file (G-file) assigned to vCPU v
> + * into the domain's stage-2 guest-physical address space.
> + *
> + * In the machine's physical address space (SPA), each hart's IMSIC
> + * supervisor-level file (S-file) is located at offset 0 of its address block,
> + * followed contiguously by GEILEN guest files at offsets of 1, 2, ..., N pages.
> + *
> + * Because a guest OS running in VS-mode expects its own supervisor-level
> + * interrupt file to be at offset 0 of its guest-physical IMSIC block, the
> + * hypervisor must use stage-2 address translation to map the vCPU's
> + * guest-physical "supervisor" page (GPA offset 0) to the specific
> + * physical guest file page (SPA offset guest_file_id) on the physical hart.
> + *
> + * Xen pins each vCPU to a pCPU (v->processor) and assigns it a physical

... an apparently wrong assumption here: Xen doesn't normally pin vCPU-s.
When a vCPU migrates between pCPU-s, clearly the mapping referencing the
page associated with the old hart needs tearing down again.

That said, since the new mapping will appear at the same GFN, the original
mapping may simply end up being replaced. If such direct replacement is
legitimate to do, maybe this could actually be mentioned here?

> + * guest file index (guest_file_id) from the vGEIN allocator. A guest_file_id
> + * of 0 indicates that no hardware guest file is selected (matching the
> + * architectural behavior where vGEIN = 0 in the hstatus CSR selects no
> + * guest external interrupt source), requiring the VS-file to be emulated
> + * in software.
> + *
> + * The base guest-physical address advertised to the guest in the device
> + * tree matches offset 0 of the vCPU's virtual IMSIC block. Stage-2
> + * translation ensures that guest supervisor accesses to this page are
> + * transparently routed to the real hardware VS-file granted to it on
> + * the current pCPU.
> + */
> +int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id)
> +{
> +    int res = 0;
> +    struct domain *d = v->domain;
> +    unsigned int cpu = v->processor;
> +    vaddr_t gaddr = imsic_cfg.base_addr + (IMSIC_MMIO_PAGE_SZ * v->vcpu_id);
> +    paddr_t paddr;
> +    unsigned long guest_stride;
> +
> +    /* Nothing to map in the case of sw interrupt file. */
> +    if ( !vsfile_id )
> +        return res;
> +
> +    guest_stride = vsfile_id * IMSIC_MMIO_PAGE_SZ;

To me "stride" feels the wrong term here, as there's nothing that repeats.
"offset" likely would be better, assuming the use of this local variable is
really deemed worth it, as it's used ...

> +    paddr = imsic_cfg.msi[cpu].base_addr + imsic_cfg.msi[cpu].offset +
> +            guest_stride;

... only here.

> +#ifdef IMSIC_DEBUG
> +    printk("%s: %pv: ga(%#lx) -> pa(%#lx), cpu(%#x), guest_file_id(%d) "
> +           "base_addr(%#lx) offset(%#lx)\n", __func__, v, gaddr, paddr, cpu,
> +           vsfile_id, imsic_cfg.msi[cpu].base_addr, imsic_cfg.msi[cpu].offset);
> +#endif
> +
> +    res = map_regions_p2mt(d, gaddr_to_gfn(gaddr),
> +                           PFN_DOWN(IMSIC_MMIO_PAGE_SZ), maddr_to_mfn(paddr),
> +                           arch_dt_passthrough_p2m_type());
> +    if ( res )
> +        printk("%s: Failed to map %#lx to the guest at %#lx\n",
> +               __func__, paddr, gaddr);

I think you mean to use PRIpaddr with paddr_t (oddly enough there's no
PRIgaddr).

Jan

Re: [PATCH v1 06/17] xen/riscv: map IMSIC interrupt file for vCPUs

[Oleksii Kurochko]

On 8/6/26 4:48 PM, Jan Beulich wrote:
> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>> A guest running in VS-mode expects its own IMSIC S-file at offset 0 of its
>> guest-physical IMSIC block. Physically, the guest-file (G-file) assigned to
>> this vCPU lives at a hart-relative offset given by guest_file_id (assigned
>> via the vGEIN allocator). Therefore, imsic_map_guest_file() uses stage-2
>> translation to redirect the guest's fixed per-vCPU GPA page (offset 0) to
>> the specific physical guest-file page.
>>
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>> ---
>> The corresponding unmap of the IMSIC interrupt file will be introduced
>> separately when the need arises.
> 
> Doesn't the need exist right away? There is ...
> 
>> @@ -342,6 +344,67 @@ static int __init imsic_parse_node(const struct dt_device_node *node,
>>       return 0;
>>   }
>>   
>> +/*
>> + * Map the physical IMSIC guest interrupt file (G-file) assigned to vCPU v
>> + * into the domain's stage-2 guest-physical address space.
>> + *
>> + * In the machine's physical address space (SPA), each hart's IMSIC
>> + * supervisor-level file (S-file) is located at offset 0 of its address block,
>> + * followed contiguously by GEILEN guest files at offsets of 1, 2, ..., N pages.
>> + *
>> + * Because a guest OS running in VS-mode expects its own supervisor-level
>> + * interrupt file to be at offset 0 of its guest-physical IMSIC block, the
>> + * hypervisor must use stage-2 address translation to map the vCPU's
>> + * guest-physical "supervisor" page (GPA offset 0) to the specific
>> + * physical guest file page (SPA offset guest_file_id) on the physical hart.
>> + *
>> + * Xen pins each vCPU to a pCPU (v->processor) and assigns it a physical
> 
> ... an apparently wrong assumption here: Xen doesn't normally pin vCPU-s.
> When a vCPU migrates between pCPU-s, clearly the mapping referencing the
> page associated with the old hart needs tearing down again.

The word “pin” was incorrect to use here. What I meant is that a vCPU is 
assigned to a pCPU by scheduler and of course it could be re-scheduled 
by a scheduler to another pCPU (maybe for NULL scheduler such 
re-scheduling don't happen...), and after this assignment happens, the 
IMSIC interrupt file mapping needs to be recalculated.

> 
> That said, since the new mapping will appear at the same GFN, the original
> mapping may simply end up being replaced. If such direct replacement is
> legitimate to do, maybe this could actually be mentioned here?

Yes, the GFN isn’t changed for a vCPU. The plan was for 
map_regions_p2mt() to simply replace the corresponding PTE for the GFN, 
which is why imsic_unmap_guest_file() isn’t really needed now.

I will re-phrase this paragraph to:
  * A vCPU runs on the pCPU the scheduler picked for it (v->processor), and
  * the guest file it is given (guest_file_id, from the vGEIN allocator)
  * belongs to that very pCPU's IMSIC. A guest_file_id of 0 indicates 
that no
  * hardware guest file is selected (matching the architectural behavior 
where
  * vGEIN = 0 in the hstatus CSR selects no guest external interrupt 
source),
  * requiring the VS-file to be emulated in software.
  *
  * Consequently the mapping installed here is only valid as long as the 
vCPU
  * stays on that pCPU. When it migrates, a VS-file is acquired on the new
  * pCPU and mapped at the very same GFN, so the stale mapping needs no
  * explicit tear-down: it is simply replaced.

> 
>> + * guest file index (guest_file_id) from the vGEIN allocator. A guest_file_id
>> + * of 0 indicates that no hardware guest file is selected (matching the
>> + * architectural behavior where vGEIN = 0 in the hstatus CSR selects no
>> + * guest external interrupt source), requiring the VS-file to be emulated
>> + * in software.
>> + *
>> + * The base guest-physical address advertised to the guest in the device
>> + * tree matches offset 0 of the vCPU's virtual IMSIC block. Stage-2
>> + * translation ensures that guest supervisor accesses to this page are
>> + * transparently routed to the real hardware VS-file granted to it on
>> + * the current pCPU.
>> + */
>> +int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id)
>> +{
>> +    int res = 0;
>> +    struct domain *d = v->domain;
>> +    unsigned int cpu = v->processor;
>> +    vaddr_t gaddr = imsic_cfg.base_addr + (IMSIC_MMIO_PAGE_SZ * v->vcpu_id);

I just noticed that imsic_cfg.base_addr isn't really good to use here. 
It should be GUEST_IMSIC_S_BASE instead.

>> +    paddr_t paddr;
>> +    unsigned long guest_stride;
>> +
>> +    /* Nothing to map in the case of sw interrupt file. */
>> +    if ( !vsfile_id )
>> +        return res;
>> +
>> +    guest_stride = vsfile_id * IMSIC_MMIO_PAGE_SZ;
> 
> To me "stride" feels the wrong term here, as there's nothing that repeats.
> "offset" likely would be better, assuming the use of this local variable is
> really deemed worth it, as it's used ...
> 
>> +    paddr = imsic_cfg.msi[cpu].base_addr + imsic_cfg.msi[cpu].offset +
>> +            guest_stride;
> 
> ... only here.

I will apply your suggestion.

> 
>> +#ifdef IMSIC_DEBUG
>> +    printk("%s: %pv: ga(%#lx) -> pa(%#lx), cpu(%#x), guest_file_id(%d) "
>> +           "base_addr(%#lx) offset(%#lx)\n", __func__, v, gaddr, paddr, cpu,
>> +           vsfile_id, imsic_cfg.msi[cpu].base_addr, imsic_cfg.msi[cpu].offset);
>> +#endif
>> +
>> +    res = map_regions_p2mt(d, gaddr_to_gfn(gaddr),
>> +                           PFN_DOWN(IMSIC_MMIO_PAGE_SZ), maddr_to_mfn(paddr),
>> +                           arch_dt_passthrough_p2m_type());
>> +    if ( res )
>> +        printk("%s: Failed to map %#lx to the guest at %#lx\n",
>> +               __func__, paddr, gaddr);
> 
> I think you mean to use PRIpaddr with paddr_t (oddly enough there's no
> PRIgaddr).

I’m wondering if it wouldn’t be better to use paddr_t for gaddr as well, 
since technically it is a guest *physical address*. In that case, 
PRIpaddr could be used to print both paddr and gaddr variables.

Also, could this be the reason why PRIgaddr doesn’t exist? Basically, a 
GPA could be considered a physical address, while for a GVA there is 
already PRIvaddr.

Thanks!

Best regards,
  Oleksii

Re: [PATCH v1 06/17] xen/riscv: map IMSIC interrupt file for vCPUs

[Jan Beulich]

On 10.08.2026 10:50, Oleksii Kurochko wrote:
> On 8/6/26 4:48 PM, Jan Beulich wrote:
>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>> +#ifdef IMSIC_DEBUG
>>> +    printk("%s: %pv: ga(%#lx) -> pa(%#lx), cpu(%#x), guest_file_id(%d) "
>>> +           "base_addr(%#lx) offset(%#lx)\n", __func__, v, gaddr, paddr, cpu,
>>> +           vsfile_id, imsic_cfg.msi[cpu].base_addr, imsic_cfg.msi[cpu].offset);
>>> +#endif
>>> +
>>> +    res = map_regions_p2mt(d, gaddr_to_gfn(gaddr),
>>> +                           PFN_DOWN(IMSIC_MMIO_PAGE_SZ), maddr_to_mfn(paddr),
>>> +                           arch_dt_passthrough_p2m_type());
>>> +    if ( res )
>>> +        printk("%s: Failed to map %#lx to the guest at %#lx\n",
>>> +               __func__, paddr, gaddr);
>>
>> I think you mean to use PRIpaddr with paddr_t (oddly enough there's no
>> PRIgaddr).
> 
> I’m wondering if it wouldn’t be better to use paddr_t for gaddr as well, 
> since technically it is a guest *physical address*. In that case, 
> PRIpaddr could be used to print both paddr and gaddr variables.
> 
> Also, could this be the reason why PRIgaddr doesn’t exist? Basically, a 
> GPA could be considered a physical address, while for a GVA there is 
> already PRIvaddr.

Well. There's nothing wrong with guest {physical,virtual} addresses to be
a different width compared to the host's. They could be both smaller and
(in principle) larger. As long as higher-bitness guests can't be run on a
smaller-bitness hypervisor, using paddr_t for gaddr-s is okay(ish).

Jan

Re: [PATCH v1 06/17] xen/riscv: map IMSIC interrupt file for vCPUs

[Baptiste Le Duc]

> A guest running in VS-mode expects its own IMSIC S-file at offset 0 of its
> guest-physical IMSIC block. Physically, the guest-file (G-file) assigned to
> this vCPU lives at a hart-relative offset given by guest_file_id (assigned
> via the vGEIN allocator). Therefore, imsic_map_guest_file() uses stage-2
> translation to redirect the guest's fixed per-vCPU GPA page (offset 0) to
> the specific physical guest-file page.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>


>
> diff --git a/xen/arch/riscv/imsic.c b/xen/arch/riscv/imsic.c
> index ffce77209c..c5ae74e456 100644
> --- a/xen/arch/riscv/imsic.c
> +++ b/xen/arch/riscv/imsic.c
> @@ -25,7 +25,9 @@
>  #include <xen/spinlock.h>
>  #include <xen/xvmalloc.h>
>  
> +#include <asm/aia.h>
>  #include <asm/imsic.h>
> +#include <asm/p2m.h>
>  
>  #define IMSIC_HART_SIZE(guest_bits) (BIT(guest_bits, U) * IMSIC_MMIO_PAGE_SZ)
>  
> @@ -342,6 +344,67 @@ static int __init imsic_parse_node(const struct dt_device_node *node,
>      return 0;
>  }
>  
> +/*
> + * Map the physical IMSIC guest interrupt file (G-file) assigned to vCPU v
Nit: What is this `v`?
> + * into the domain's stage-2 guest-physical address space.
> + *
> + * In the machine's physical address space (SPA), each hart's IMSIC
> + * supervisor-level file (S-file) is located at offset 0 of its address block,
> + * followed contiguously by GEILEN guest files at offsets of 1, 2, ..., N pages.
> + *
> + * Because a guest OS running in VS-mode expects its own supervisor-level
> + * interrupt file to be at offset 0 of its guest-physical IMSIC block, the
> + * hypervisor must use stage-2 address translation to map the vCPU's
> + * guest-physical "supervisor" page (GPA offset 0) to the specific
> + * physical guest file page (SPA offset guest_file_id) on the physical hart.
> + *
> + * Xen pins each vCPU to a pCPU (v->processor) and assigns it a physical


> + * guest file index (guest_file_id) from the vGEIN allocator. A guest_file_id
> + * of 0 indicates that no hardware guest file is selected (matching the
> + * architectural behavior where vGEIN = 0 in the hstatus CSR selects no
> + * guest external interrupt source), requiring the VS-file to be emulated
> + * in software.
> + *
> + * The base guest-physical address advertised to the guest in the device
> + * tree matches offset 0 of the vCPU's virtual IMSIC block. Stage-2
> + * translation ensures that guest supervisor accesses to this page are
> + * transparently routed to the real hardware VS-file granted to it on
> + * the current pCPU.
> + */
> +int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id)
> +{
> +    int res = 0;
> +    struct domain *d = v->domain;
> +    unsigned int cpu = v->processor;
> +    vaddr_t gaddr = imsic_cfg.base_addr + (IMSIC_MMIO_PAGE_SZ * v->vcpu_id);

The variable holds a guest-physical address, so vaddr_t is the wrong
type should be either paddr_t or gaddr_t.

> +    paddr_t paddr;
> +    unsigned long guest_stride;
> +
> +    /* Nothing to map in the case of sw interrupt file. */

There is no software interrupt file implementation in this series, patch 11
turns the non-MSI path into a BUG_ON(). So "vsfile_id == 0" today means "this
vCPU gets no external interrupts at all and nothing tells anybody". Worth
saying so plainly here rather than implying a fallback exists.

> +    if ( !vsfile_id )
> +        return res;
> +
> +    guest_stride = vsfile_id * IMSIC_MMIO_PAGE_SZ;


> +
> +    paddr = imsic_cfg.msi[cpu].base_addr + imsic_cfg.msi[cpu].offset +
> +            guest_stride;


> +
> +#ifdef IMSIC_DEBUG

> +    printk("%s: %pv: ga(%#lx) -> pa(%#lx), cpu(%#x), guest_file_id(%d) "
> +           "base_addr(%#lx) offset(%#lx)\n", __func__, v, gaddr, paddr, cpu,
> +           vsfile_id, imsic_cfg.msi[cpu].base_addr, imsic_cfg.msi[cpu].offset);
> +#endif
> +
> +    res = map_regions_p2mt(d, gaddr_to_gfn(gaddr),
> +                           PFN_DOWN(IMSIC_MMIO_PAGE_SZ), maddr_to_mfn(paddr),
> +                           arch_dt_passthrough_p2m_type());
> +    if ( res )
> +        printk("%s: Failed to map %#lx to the guest at %#lx\n",

Maybe a use of dprintk() would be more appropriate?

-- 
Baptiste Le Duc <baptiste.le-duc@vates.tech>

Re: [PATCH v1 06/17] xen/riscv: map IMSIC interrupt file for vCPUs

[Oleksii Kurochko]

On 8/13/26 11:06 AM, Baptiste Le Duc wrote:
>> A guest running in VS-mode expects its own IMSIC S-file at offset 0 of its
>> guest-physical IMSIC block. Physically, the guest-file (G-file) assigned to
>> this vCPU lives at a hart-relative offset given by guest_file_id (assigned
>> via the vGEIN allocator). Therefore, imsic_map_guest_file() uses stage-2
>> translation to redirect the guest's fixed per-vCPU GPA page (offset 0) to
>> the specific physical guest-file page.
>>
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> 
> 
>>
>> diff --git a/xen/arch/riscv/imsic.c b/xen/arch/riscv/imsic.c
>> index ffce77209c..c5ae74e456 100644
>> --- a/xen/arch/riscv/imsic.c
>> +++ b/xen/arch/riscv/imsic.c
>> @@ -25,7 +25,9 @@
>>   #include <xen/spinlock.h>
>>   #include <xen/xvmalloc.h>
>>   
>> +#include <asm/aia.h>
>>   #include <asm/imsic.h>
>> +#include <asm/p2m.h>
>>   
>>   #define IMSIC_HART_SIZE(guest_bits) (BIT(guest_bits, U) * IMSIC_MMIO_PAGE_SZ)
>>   
>> @@ -342,6 +344,67 @@ static int __init imsic_parse_node(const struct dt_device_node *node,
>>       return 0;
>>   }
>>   
>> +/*
>> + * Map the physical IMSIC guest interrupt file (G-file) assigned to vCPU v
> Nit: What is this `v`?

A function argument. But I will just drop v from the comment.

>> + * into the domain's stage-2 guest-physical address space.
>> + *
>> + * In the machine's physical address space (SPA), each hart's IMSIC
>> + * supervisor-level file (S-file) is located at offset 0 of its address block,
>> + * followed contiguously by GEILEN guest files at offsets of 1, 2, ..., N pages.
>> + *
>> + * Because a guest OS running in VS-mode expects its own supervisor-level
>> + * interrupt file to be at offset 0 of its guest-physical IMSIC block, the
>> + * hypervisor must use stage-2 address translation to map the vCPU's
>> + * guest-physical "supervisor" page (GPA offset 0) to the specific
>> + * physical guest file page (SPA offset guest_file_id) on the physical hart.
>> + *
>> + * Xen pins each vCPU to a pCPU (v->processor) and assigns it a physical
> 
> 
>> + * guest file index (guest_file_id) from the vGEIN allocator. A guest_file_id
>> + * of 0 indicates that no hardware guest file is selected (matching the
>> + * architectural behavior where vGEIN = 0 in the hstatus CSR selects no
>> + * guest external interrupt source), requiring the VS-file to be emulated
>> + * in software.
>> + *
>> + * The base guest-physical address advertised to the guest in the device
>> + * tree matches offset 0 of the vCPU's virtual IMSIC block. Stage-2
>> + * translation ensures that guest supervisor accesses to this page are
>> + * transparently routed to the real hardware VS-file granted to it on
>> + * the current pCPU.
>> + */
>> +int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id)
>> +{
>> +    int res = 0;
>> +    struct domain *d = v->domain;
>> +    unsigned int cpu = v->processor;
>> +    vaddr_t gaddr = imsic_cfg.base_addr + (IMSIC_MMIO_PAGE_SZ * v->vcpu_id);
> 
> The variable holds a guest-physical address, so vaddr_t is the wrong
> type should be either paddr_t or gaddr_t.

Agree, paddr_t will be better what was mentioned in thread with Jan B.

> 
>> +    paddr_t paddr;
>> +    unsigned long guest_stride;
>> +
>> +    /* Nothing to map in the case of sw interrupt file. */
> 
> There is no software interrupt file implementation in this series, patch 11
> turns the non-MSI path into a BUG_ON(). So "vsfile_id == 0" today means "this
> vCPU gets no external interrupts at all and nothing tells anybody". Worth
> saying so plainly here rather than implying a fallback exists.

I would ask then different question will this function change when IMSIC 
interrupt file support will be added? I think - no as in the case of 
IMSIC interrupt file we don't need any stage-2 mapping. So here it is 
just a check that nothing should be mapped for non-hw-assisted interrupt 
files.

> 
>> +    if ( !vsfile_id )
>> +        return res;
>> +
>> +    guest_stride = vsfile_id * IMSIC_MMIO_PAGE_SZ;
> 
> 
>> +
>> +    paddr = imsic_cfg.msi[cpu].base_addr + imsic_cfg.msi[cpu].offset +
>> +            guest_stride;
> 
> 
>> +
>> +#ifdef IMSIC_DEBUG
> 
>> +    printk("%s: %pv: ga(%#lx) -> pa(%#lx), cpu(%#x), guest_file_id(%d) "
>> +           "base_addr(%#lx) offset(%#lx)\n", __func__, v, gaddr, paddr, cpu,
>> +           vsfile_id, imsic_cfg.msi[cpu].base_addr, imsic_cfg.msi[cpu].offset);
>> +#endif
>> +
>> +    res = map_regions_p2mt(d, gaddr_to_gfn(gaddr),
>> +                           PFN_DOWN(IMSIC_MMIO_PAGE_SZ), maddr_to_mfn(paddr),
>> +                           arch_dt_passthrough_p2m_type());
>> +    if ( res )
>> +        printk("%s: Failed to map %#lx to the guest at %#lx\n",
> 
> Maybe a use of dprintk() would be more appropriate?
> 
Agree, dprintk() will be better.

Thanks.

~ Oleksii

Re: [PATCH v1 06/17] xen/riscv: map IMSIC interrupt file for vCPUs

[Baptiste Le Duc]

On 2026-08-13 11:42 +0200, Oleksii Kurochko wrote:
> 
> 
> On 8/13/26 11:06 AM, Baptiste Le Duc wrote:
> >> A guest running in VS-mode expects its own IMSIC S-file at offset 0 of its
> >> guest-physical IMSIC block. Physically, the guest-file (G-file) assigned to
> >> this vCPU lives at a hart-relative offset given by guest_file_id (assigned
> >> via the vGEIN allocator). Therefore, imsic_map_guest_file() uses stage-2
> >> translation to redirect the guest's fixed per-vCPU GPA page (offset 0) to
> >> the specific physical guest-file page.
> >>
> >> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > 
> > 
> >>
> >> diff --git a/xen/arch/riscv/imsic.c b/xen/arch/riscv/imsic.c
> >> index ffce77209c..c5ae74e456 100644
> >> --- a/xen/arch/riscv/imsic.c
> >> +++ b/xen/arch/riscv/imsic.c
> >> @@ -25,7 +25,9 @@
> >>   #include <xen/spinlock.h>
> >>   #include <xen/xvmalloc.h>
> >>   
> >> +#include <asm/aia.h>
> >>   #include <asm/imsic.h>
> >> +#include <asm/p2m.h>
> >>   
> >>   #define IMSIC_HART_SIZE(guest_bits) (BIT(guest_bits, U) * IMSIC_MMIO_PAGE_SZ)
> >>   
> >> @@ -342,6 +344,67 @@ static int __init imsic_parse_node(const struct dt_device_node *node,
> >>       return 0;
> >>   }
> >>   
> >> +/*
> >> + * Map the physical IMSIC guest interrupt file (G-file) assigned to vCPU v
> > Nit: What is this `v`?
> 
> A function argument. But I will just drop v from the comment.
> 
> >> + * into the domain's stage-2 guest-physical address space.
> >> + *
> >> + * In the machine's physical address space (SPA), each hart's IMSIC
> >> + * supervisor-level file (S-file) is located at offset 0 of its address block,
> >> + * followed contiguously by GEILEN guest files at offsets of 1, 2, ..., N pages.
> >> + *
> >> + * Because a guest OS running in VS-mode expects its own supervisor-level
> >> + * interrupt file to be at offset 0 of its guest-physical IMSIC block, the
> >> + * hypervisor must use stage-2 address translation to map the vCPU's
> >> + * guest-physical "supervisor" page (GPA offset 0) to the specific
> >> + * physical guest file page (SPA offset guest_file_id) on the physical hart.
> >> + *
> >> + * Xen pins each vCPU to a pCPU (v->processor) and assigns it a physical
> > 
> > 
> >> + * guest file index (guest_file_id) from the vGEIN allocator. A guest_file_id
> >> + * of 0 indicates that no hardware guest file is selected (matching the
> >> + * architectural behavior where vGEIN = 0 in the hstatus CSR selects no
> >> + * guest external interrupt source), requiring the VS-file to be emulated
> >> + * in software.
> >> + *
> >> + * The base guest-physical address advertised to the guest in the device
> >> + * tree matches offset 0 of the vCPU's virtual IMSIC block. Stage-2
> >> + * translation ensures that guest supervisor accesses to this page are
> >> + * transparently routed to the real hardware VS-file granted to it on
> >> + * the current pCPU.
> >> + */
> >> +int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id)
> >> +{
> >> +    int res = 0;
> >> +    struct domain *d = v->domain;
> >> +    unsigned int cpu = v->processor;
> >> +    vaddr_t gaddr = imsic_cfg.base_addr + (IMSIC_MMIO_PAGE_SZ * v->vcpu_id);
> > 
> > The variable holds a guest-physical address, so vaddr_t is the wrong
> > type should be either paddr_t or gaddr_t.
> 
> Agree, paddr_t will be better what was mentioned in thread with Jan B.
> 
> > 
> >> +    paddr_t paddr;
> >> +    unsigned long guest_stride;
> >> +
> >> +    /* Nothing to map in the case of sw interrupt file. */
> > 
> > There is no software interrupt file implementation in this series, patch 11
> > turns the non-MSI path into a BUG_ON(). So "vsfile_id == 0" today means "this
> > vCPU gets no external interrupts at all and nothing tells anybody". Worth
> > saying so plainly here rather than implying a fallback exists.
> 
> I would ask then different question will this function change when IMSIC 
> interrupt file support will be added? I think - no as in the case of 
Did you forget s/w word? If not it's weird as IMSIC interrupt file is
the current topic of this patch series.
> IMSIC interrupt file we don't need any stage-2 mapping. So here it is 
here too.
> just a check that nothing should be mapped for non-hw-assisted interrupt 
> files.
> 
> > 
> >> +    if ( !vsfile_id )
> >> +        return res;
> >> +
> >> +    guest_stride = vsfile_id * IMSIC_MMIO_PAGE_SZ;
> > 
> > 
> >> +
> >> +    paddr = imsic_cfg.msi[cpu].base_addr + imsic_cfg.msi[cpu].offset +
> >> +            guest_stride;
> > 
> > 
> >> +
> >> +#ifdef IMSIC_DEBUG
> > 
> >> +    printk("%s: %pv: ga(%#lx) -> pa(%#lx), cpu(%#x), guest_file_id(%d) "
> >> +           "base_addr(%#lx) offset(%#lx)\n", __func__, v, gaddr, paddr, cpu,
> >> +           vsfile_id, imsic_cfg.msi[cpu].base_addr, imsic_cfg.msi[cpu].offset);
> >> +#endif
> >> +
> >> +    res = map_regions_p2mt(d, gaddr_to_gfn(gaddr),
> >> +                           PFN_DOWN(IMSIC_MMIO_PAGE_SZ), maddr_to_mfn(paddr),
> >> +                           arch_dt_passthrough_p2m_type());
> >> +    if ( res )
> >> +        printk("%s: Failed to map %#lx to the guest at %#lx\n",
> > 
> > Maybe a use of dprintk() would be more appropriate?
> > 
> Agree, dprintk() will be better.
> 
> Thanks.
> 
> ~ Oleksii
> 
> 
> 

Re: [PATCH v1 06/17] xen/riscv: map IMSIC interrupt file for vCPUs

[Oleksii Kurochko]

On 8/13/26 11:49 AM, Baptiste Le Duc wrote:
> On 2026-08-13 11:42 +0200, Oleksii Kurochko wrote:
>>>> +/*
>>>> + * Map the physical IMSIC guest interrupt file (G-file) assigned to vCPU v
>>> Nit: What is this `v`?
>>
>> A function argument. But I will just drop v from the comment.
>>
>>>> + * into the domain's stage-2 guest-physical address space.
>>>> + *
>>>> + * In the machine's physical address space (SPA), each hart's IMSIC
>>>> + * supervisor-level file (S-file) is located at offset 0 of its address block,
>>>> + * followed contiguously by GEILEN guest files at offsets of 1, 2, ..., N pages.
>>>> + *
>>>> + * Because a guest OS running in VS-mode expects its own supervisor-level
>>>> + * interrupt file to be at offset 0 of its guest-physical IMSIC block, the
>>>> + * hypervisor must use stage-2 address translation to map the vCPU's
>>>> + * guest-physical "supervisor" page (GPA offset 0) to the specific
>>>> + * physical guest file page (SPA offset guest_file_id) on the physical hart.
>>>> + *
>>>> + * Xen pins each vCPU to a pCPU (v->processor) and assigns it a physical
>>>
>>>
>>>> + * guest file index (guest_file_id) from the vGEIN allocator. A guest_file_id
>>>> + * of 0 indicates that no hardware guest file is selected (matching the
>>>> + * architectural behavior where vGEIN = 0 in the hstatus CSR selects no
>>>> + * guest external interrupt source), requiring the VS-file to be emulated
>>>> + * in software.
>>>> + *
>>>> + * The base guest-physical address advertised to the guest in the device
>>>> + * tree matches offset 0 of the vCPU's virtual IMSIC block. Stage-2
>>>> + * translation ensures that guest supervisor accesses to this page are
>>>> + * transparently routed to the real hardware VS-file granted to it on
>>>> + * the current pCPU.
>>>> + */
>>>> +int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id)
>>>> +{
>>>> +    int res = 0;
>>>> +    struct domain *d = v->domain;
>>>> +    unsigned int cpu = v->processor;
>>>> +    vaddr_t gaddr = imsic_cfg.base_addr + (IMSIC_MMIO_PAGE_SZ * v->vcpu_id);
>>>
>>> The variable holds a guest-physical address, so vaddr_t is the wrong
>>> type should be either paddr_t or gaddr_t.
>>
>> Agree, paddr_t will be better what was mentioned in thread with Jan B.
>>
>>>
>>>> +    paddr_t paddr;
>>>> +    unsigned long guest_stride;
>>>> +
>>>> +    /* Nothing to map in the case of sw interrupt file. */
>>>
>>> There is no software interrupt file implementation in this series, patch 11
>>> turns the non-MSI path into a BUG_ON(). So "vsfile_id == 0" today means "this
>>> vCPU gets no external interrupts at all and nothing tells anybody". Worth
>>> saying so plainly here rather than implying a fallback exists.
>>
>> I would ask then different question will this function change when IMSIC
>> interrupt file support will be added? I think - no as in the case of
> Did you forget s/w word? If not it's weird as IMSIC interrupt file is
> the current topic of this patch series.
>> IMSIC interrupt file we don't need any stage-2 mapping. So here it is
> here too.

Yes, sorry, I missed s/w word.

>> just a check that nothing should be mapped for non-hw-assisted interrupt
>> files.
>>
>>>
>>>> +    if ( !vsfile_id )
>>>> +        return res;
>>>> +
>>>> +    guest_stride = vsfile_id * IMSIC_MMIO_PAGE_SZ;
>>>
>>>
>>>> +
>>>> +    paddr = imsic_cfg.msi[cpu].base_addr + imsic_cfg.msi[cpu].offset +
>>>> +            guest_stride;
>>>
>>>
>>>> +
>>>> +#ifdef IMSIC_DEBUG
>>>
>>>> +    printk("%s: %pv: ga(%#lx) -> pa(%#lx), cpu(%#x), guest_file_id(%d) "
>>>> +           "base_addr(%#lx) offset(%#lx)\n", __func__, v, gaddr, paddr, cpu,
>>>> +           vsfile_id, imsic_cfg.msi[cpu].base_addr, imsic_cfg.msi[cpu].offset);
>>>> +#endif
>>>> +
>>>> +    res = map_regions_p2mt(d, gaddr_to_gfn(gaddr),
>>>> +                           PFN_DOWN(IMSIC_MMIO_PAGE_SZ), maddr_to_mfn(paddr),
>>>> +                           arch_dt_passthrough_p2m_type());
>>>> +    if ( res )
>>>> +        printk("%s: Failed to map %#lx to the guest at %#lx\n",
>>>

~ Oleksii

[PATCH v1 07/17] xen/riscv: introduce vCPU AIA initialization

[Oleksii Kurochko]

Introduce vcpu_aia_init() to initialize the AIA-related state needed
for a vCPU to have a working guest interrupt file.

A guest (VS) interrupt file must be mapped to one of a pCPU's
hardware interrupt files (if they exist), so the pCPU a vCPU will actually
run on needs to be known first. arch_vcpu_create() is therefore not a
suitable place to call vcpu_aia_init(), since the pCPU assigned to a
vCPU can still change before it is first scheduled. To avoid
reassigning the VS interrupt file id and remapping it to a different
pCPU's hardware interrupt file, vcpu_aia_init() will instead be
called from a later point in the scheduling path (e.g.
continue_to_new_vcpu()), to be introduced in a follow-up patch. Since
it will end up being called from a non-__init context, it is not
itself marked __init.

Introduce imsic_update_state() to update a vCPU's guest IMSIC state
(the guest interrupt file id and the pCPU whose hardware interrupt
file it is mapped to) as a single consistent unit. This state can be
read concurrently, e.g. by a future helper that checks whether a
vCPU has a pending IMSIC interrupt, though no such consumer exists
yet at this stage - so it is protected by a lock.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 xen/arch/riscv/aia.c               | 30 ++++++++++++++++++++++++++++++
 xen/arch/riscv/imsic.c             | 13 +++++++++++++
 xen/arch/riscv/include/asm/aia.h   |  2 ++
 xen/arch/riscv/include/asm/imsic.h |  2 ++
 4 files changed, 47 insertions(+)

diff --git a/xen/arch/riscv/aia.c b/xen/arch/riscv/aia.c
index 4f7f46f58f0b..ed19600d4644 100644
--- a/xen/arch/riscv/aia.c
+++ b/xen/arch/riscv/aia.c
@@ -14,6 +14,7 @@
 #include <asm/cpufeature.h>
 #include <asm/csr.h>
 #include <asm/current.h>
+#include <asm/imsic.h>
 
 struct vgein_ctrl {
     unsigned long bmp;
@@ -36,6 +37,35 @@ bool aia_usable(void)
     return _aia_usable;
 }
 
+void vcpu_aia_init(struct vcpu *v)
+{
+    unsigned int new_vsfile_id;
+    int rc;
+
+    if ( !aia_usable() )
+        return;
+
+    new_vsfile_id = vgein_assign(v);
+
+    /*
+     * vgein_assign() returns 0 when no free h/w guest interrupt file is
+     * available (including GEILEN == 0); imsic_map_guest_file() maps nothing
+     * in that case.
+     */
+    rc = imsic_map_guest_file(v, new_vsfile_id);
+    if ( rc )
+    {
+        /* Can't continue w/o correctly mapped IMSIC interrupt file */
+        domain_crash(v->domain);
+        return;
+    }
+
+    vcpu_guest_cpu_user_regs(v)->hstatus |=
+        MASK_INSR(new_vsfile_id, HSTATUS_VGEIN);
+
+    imsic_update_state(v, new_vsfile_id);
+}
+
 static int vgein_init(unsigned int cpu)
 {
     struct vgein_ctrl *vgein = &per_cpu(vgein, cpu);
diff --git a/xen/arch/riscv/imsic.c b/xen/arch/riscv/imsic.c
index c5ae74e456e8..2a792e756c4e 100644
--- a/xen/arch/riscv/imsic.c
+++ b/xen/arch/riscv/imsic.c
@@ -83,6 +83,19 @@ unsigned int vcpu_guest_file_id(const struct vcpu *v)
     return ACCESS_ONCE(v->arch.vimsic_state->guest_file_id);
 }
 
+void imsic_update_state(struct vcpu *v, unsigned int guest_file_id)
+{
+    unsigned long flags;
+    struct vimsic_state *vimsic_state = v->arch.vimsic_state;
+    unsigned long pcpu = ( !guest_file_id ) ?
+                         NR_CPUS : cpuid_to_hartid(v->processor);
+
+    write_lock_irqsave(&vimsic_state->vsfile_lock, flags);
+    vimsic_state->guest_file_id = guest_file_id;
+    vimsic_state->vsfile_pcpu = pcpu;
+    write_unlock_irqrestore(&vimsic_state->vsfile_lock, flags);
+}
+
 void __init imsic_ids_local_delivery(bool enable)
 {
     if ( enable )
diff --git a/xen/arch/riscv/include/asm/aia.h b/xen/arch/riscv/include/asm/aia.h
index c67be0069a1d..2aef24cad4c0 100644
--- a/xen/arch/riscv/include/asm/aia.h
+++ b/xen/arch/riscv/include/asm/aia.h
@@ -15,4 +15,6 @@ void aia_init(void);
 unsigned int vgein_assign(struct vcpu *v);
 void vgein_release(struct vcpu *v, unsigned int vgein_id);
 
+void vcpu_aia_init(struct vcpu *v);
+
 #endif /* RISCV_AIA_H */
diff --git a/xen/arch/riscv/include/asm/imsic.h b/xen/arch/riscv/include/asm/imsic.h
index f2c649517fd1..8e3797690f99 100644
--- a/xen/arch/riscv/include/asm/imsic.h
+++ b/xen/arch/riscv/include/asm/imsic.h
@@ -104,6 +104,8 @@ int vcpu_imsic_init(struct vcpu *v);
 void vcpu_imsic_deinit(struct vcpu *v);
 unsigned int vcpu_guest_file_id(const struct vcpu *v);
 
+void imsic_update_state(struct vcpu *v, unsigned int guest_file_id);
+
 int vimsic_make_domu_dt_node(struct kernel_info *kinfo, unsigned int *phandle);
 
 int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id);
-- 
2.54.0

Re: [PATCH v1 07/17] xen/riscv: introduce vCPU AIA initialization

[Jan Beulich]

On 20.07.2026 18:02, Oleksii Kurochko wrote:
> Introduce vcpu_aia_init() to initialize the AIA-related state needed
> for a vCPU to have a working guest interrupt file.
> 
> A guest (VS) interrupt file must be mapped to one of a pCPU's
> hardware interrupt files (if they exist), so the pCPU a vCPU will actually
> run on needs to be known first. arch_vcpu_create() is therefore not a
> suitable place to call vcpu_aia_init(), since the pCPU assigned to a
> vCPU can still change before it is first scheduled. To avoid
> reassigning the VS interrupt file id and remapping it to a different
> pCPU's hardware interrupt file, vcpu_aia_init() will instead be
> called from a later point in the scheduling path (e.g.
> continue_to_new_vcpu()), to be introduced in a follow-up patch. Since
> it will end up being called from a non-__init context, it is not
> itself marked __init.

If it's called during scheduling, perhaps vcpu_aia_init() simply isn't
an appropriate name, and that issue is then also reflected in a
misleading patch subject?

> @@ -36,6 +37,35 @@ bool aia_usable(void)
>      return _aia_usable;
>  }
>  
> +void vcpu_aia_init(struct vcpu *v)
> +{
> +    unsigned int new_vsfile_id;
> +    int rc;
> +
> +    if ( !aia_usable() )
> +        return;
> +
> +    new_vsfile_id = vgein_assign(v);
> +
> +    /*
> +     * vgein_assign() returns 0 when no free h/w guest interrupt file is
> +     * available (including GEILEN == 0); imsic_map_guest_file() maps nothing
> +     * in that case.
> +     */
> +    rc = imsic_map_guest_file(v, new_vsfile_id);
> +    if ( rc )
> +    {
> +        /* Can't continue w/o correctly mapped IMSIC interrupt file */
> +        domain_crash(v->domain);
> +        return;
> +    }
> +
> +    vcpu_guest_cpu_user_regs(v)->hstatus |=
> +        MASK_INSR(new_vsfile_id, HSTATUS_VGEIN);

Looks like you're assuming that no other ID was previously stored in that
field. That can't be quite right when the function is called after the
vCPU moved to a different pCPU.

> --- a/xen/arch/riscv/imsic.c
> +++ b/xen/arch/riscv/imsic.c
> @@ -83,6 +83,19 @@ unsigned int vcpu_guest_file_id(const struct vcpu *v)
>      return ACCESS_ONCE(v->arch.vimsic_state->guest_file_id);
>  }
>  
> +void imsic_update_state(struct vcpu *v, unsigned int guest_file_id)
> +{
> +    unsigned long flags;
> +    struct vimsic_state *vimsic_state = v->arch.vimsic_state;
> +    unsigned long pcpu = ( !guest_file_id ) ?
> +                         NR_CPUS : cpuid_to_hartid(v->processor);

"pcpu" as a name is misleading when what you store is a hart ID. NR_CPUS
then also isn't a suitable sentinel.

Also, style nit: The parentheses aren't really needed around the conditional.
But what's definitely wrong are the blanks immediately inside them.

> +    write_lock_irqsave(&vimsic_state->vsfile_lock, flags);
> +    vimsic_state->guest_file_id = guest_file_id;
> +    vimsic_state->vsfile_pcpu = pcpu;

By implication from the remark above, the field name stored into then also
is misnamed.

Jan

Re: [PATCH v1 07/17] xen/riscv: introduce vCPU AIA initialization

[Oleksii Kurochko]

On 8/6/26 4:56 PM, Jan Beulich wrote:
> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>> Introduce vcpu_aia_init() to initialize the AIA-related state needed
>> for a vCPU to have a working guest interrupt file.
>>
>> A guest (VS) interrupt file must be mapped to one of a pCPU's
>> hardware interrupt files (if they exist), so the pCPU a vCPU will actually
>> run on needs to be known first. arch_vcpu_create() is therefore not a
>> suitable place to call vcpu_aia_init(), since the pCPU assigned to a
>> vCPU can still change before it is first scheduled. To avoid
>> reassigning the VS interrupt file id and remapping it to a different
>> pCPU's hardware interrupt file, vcpu_aia_init() will instead be
>> called from a later point in the scheduling path (e.g.
>> continue_to_new_vcpu()), to be introduced in a follow-up patch. Since
>> it will end up being called from a non-__init context, it is not
>> itself marked __init.
> 
> If it's called during scheduling, perhaps vcpu_aia_init() simply isn't
> an appropriate name, and that issue is then also reflected in a
> misleading patch subject?z
I also thought about that while working on the IMSIC interrupt file 
support, but I was thinking of moving it to imsic.c.

Regarding the function name, a better name would be 
vcpu_imsic_hw_vsfile_attach(). Alternatively, we could use a slightly 
more architectural term, such as HGEI/VGEIN, and call it 
vcpu_imsic_hgei_attach(). I think I prefer vcpu_imsic_hw_vsfile_attach().

Considering your observation and question, it could also be placed where 
it will actually be called from continue_new() in the future, so 
riscv/domain.c might be the right place for it but at the moment I think 
it will be better to put it in imsic.c closer to other IMSIC functionality.

> 
>> @@ -36,6 +37,35 @@ bool aia_usable(void)
>>       return _aia_usable;
>>   }
>>   
>> +void vcpu_aia_init(struct vcpu *v)
>> +{
>> +    unsigned int new_vsfile_id;
>> +    int rc;
>> +
>> +    if ( !aia_usable() )
>> +        return;
>> +
>> +    new_vsfile_id = vgein_assign(v);

I will add here also the check that if new_vsfile_id = 0 then we don't 
need to map guest file.

>> +
>> +    /*
>> +     * vgein_assign() returns 0 when no free h/w guest interrupt file is
>> +     * available (including GEILEN == 0); imsic_map_guest_file() maps nothing
>> +     * in that case.
>> +     */
>> +    rc = imsic_map_guest_file(v, new_vsfile_id);
>> +    if ( rc )
>> +    {

I missed here vgein_release().

>> +        /* Can't continue w/o correctly mapped IMSIC interrupt file */
>> +        domain_crash(v->domain);
>> +        return;
>> +    }
>> +
>> +    vcpu_guest_cpu_user_regs(v)->hstatus |=
>> +        MASK_INSR(new_vsfile_id, HSTATUS_VGEIN);
> 
> Looks like you're assuming that no other ID was previously stored in that
> field. That can't be quite right when the function is called after the
> vCPU moved to a different pCPU.

I don't use it during the migration process as during migration it is a 
little bit different sequence of how all of that inside the function is 
called; I use it only jumping to new vCPU (continue_new_cpu()), where I 
expect ->hstatus.vgein to be zero because of how the area for the vCPU 
registers is allocated, via vzalloc().

Probably I should consider to rework that and make it re-usable for both 
creating/jumping_to_new_vcpu and migration process.

> 
>> --- a/xen/arch/riscv/imsic.c
>> +++ b/xen/arch/riscv/imsic.c
>> @@ -83,6 +83,19 @@ unsigned int vcpu_guest_file_id(const struct vcpu *v)
>>       return ACCESS_ONCE(v->arch.vimsic_state->guest_file_id);
>>   }
>>   
>> +void imsic_update_state(struct vcpu *v, unsigned int guest_file_id)
>> +{
>> +    unsigned long flags;
>> +    struct vimsic_state *vimsic_state = v->arch.vimsic_state;
>> +    unsigned long pcpu = ( !guest_file_id ) ?
>> +                         NR_CPUS : cpuid_to_hartid(v->processor);
> 
> "pcpu" as a name is misleading when what you store is a hart ID. NR_CPUS
> then also isn't a suitable sentinel.
> 

Agree. I will store here v->processor and NR_CPUS if s/w interrupt file 
is used and then use cpuid_to_hartid() when it will be necessary.

> Also, style nit: The parentheses aren't really needed around the conditional.
> But what's definitely wrong are the blanks immediately inside them.

I will deal with that.

> 
>> +    write_lock_irqsave(&vimsic_state->vsfile_lock, flags);
>> +    vimsic_state->guest_file_id = guest_file_id;
>> +    vimsic_state->vsfile_pcpu = pcpu;
> 
> By implication from the remark above, the field name stored into then also
> is misnamed.

I think with the suggested changed above here everything will be fine.

Thanks.

~ Oleksii

Re: [PATCH v1 07/17] xen/riscv: introduce vCPU AIA initialization

[Baptiste Le Duc]

> Introduce vcpu_aia_init() to initialize the AIA-related state needed
> for a vCPU to have a working guest interrupt file.
> 
> A guest (VS) interrupt file must be mapped to one of a pCPU's
> hardware interrupt files (if they exist), so the pCPU a vCPU will actually
> run on needs to be known first. arch_vcpu_create() is therefore not a
> suitable place to call vcpu_aia_init(), since the pCPU assigned to a
> vCPU can still change before it is first scheduled. To avoid
> reassigning the VS interrupt file id and remapping it to a different
> pCPU's hardware interrupt file, vcpu_aia_init() will instead be
> called from a later point in the scheduling path (e.g.
> continue_to_new_vcpu()), to be introduced in a follow-up patch. Since
> it will end up being called from a non-__init context, it is not
> itself marked __init.


> 
> Introduce imsic_update_state() to update a vCPU's guest IMSIC state
> (the guest interrupt file id and the pCPU whose hardware interrupt
> file it is mapped to) as a single consistent unit. This state can be
> read concurrently, e.g. by a future helper that checks whether a
> vCPU has a pending IMSIC interrupt, though no such consumer exists
> yet at this stage - so it is protected by a lock.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>
> diff --git a/xen/arch/riscv/aia.c b/xen/arch/riscv/aia.c
> index 4f7f46f58f..ed19600d46 100644
> --- a/xen/arch/riscv/aia.c
> +++ b/xen/arch/riscv/aia.c
> @@ -14,6 +14,7 @@
>  #include <asm/cpufeature.h>
>  #include <asm/csr.h>
>  #include <asm/current.h>
> +#include <asm/imsic.h>
>  
>  struct vgein_ctrl {
>      unsigned long bmp;
> @@ -36,6 +37,35 @@ bool aia_usable(void)
>      return _aia_usable;
>  }
>  
> +void vcpu_aia_init(struct vcpu *v)
> +{
> +    unsigned int new_vsfile_id;
> +    int rc;
> +
> +    if ( !aia_usable() )
> +        return;
> +
> +    new_vsfile_id = vgein_assign(v);
> +
> +    /*
> +     * vgein_assign() returns 0 when no free h/w guest interrupt file is
> +     * available (including GEILEN == 0); imsic_map_guest_file() maps nothing
> +     * in that case.
> +     */
> +    rc = imsic_map_guest_file(v, new_vsfile_id);
> +    if ( rc )
> +    {
> +        /* Can't continue w/o correctly mapped IMSIC interrupt file */
> +        domain_crash(v->domain);

The vgein id assigned a few lines up is not released here. The domain is dying
anyway, but the guest interrupt file stays marked in use on that pCPU forever,
since nothing else ever calls vgein_release(). A vgein_release(v,
new_vsfile_id) before the domain_crash() would fix it.

-- 
Baptiste Le Duc <baptiste.le-duc@vates.tech>

Re: [PATCH v1 07/17] xen/riscv: introduce vCPU AIA initialization

[Oleksii Kurochko]

On 8/13/26 11:24 AM, Baptiste Le Duc wrote:
>> Introduce vcpu_aia_init() to initialize the AIA-related state needed
>> for a vCPU to have a working guest interrupt file.
>>
>> A guest (VS) interrupt file must be mapped to one of a pCPU's
>> hardware interrupt files (if they exist), so the pCPU a vCPU will actually
>> run on needs to be known first. arch_vcpu_create() is therefore not a
>> suitable place to call vcpu_aia_init(), since the pCPU assigned to a
>> vCPU can still change before it is first scheduled. To avoid
>> reassigning the VS interrupt file id and remapping it to a different
>> pCPU's hardware interrupt file, vcpu_aia_init() will instead be
>> called from a later point in the scheduling path (e.g.
>> continue_to_new_vcpu()), to be introduced in a follow-up patch. Since
>> it will end up being called from a non-__init context, it is not
>> itself marked __init.
> 
> 
>>
>> Introduce imsic_update_state() to update a vCPU's guest IMSIC state
>> (the guest interrupt file id and the pCPU whose hardware interrupt
>> file it is mapped to) as a single consistent unit. This state can be
>> read concurrently, e.g. by a future helper that checks whether a
>> vCPU has a pending IMSIC interrupt, though no such consumer exists
>> yet at this stage - so it is protected by a lock.
>>
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>
>> diff --git a/xen/arch/riscv/aia.c b/xen/arch/riscv/aia.c
>> index 4f7f46f58f..ed19600d46 100644
>> --- a/xen/arch/riscv/aia.c
>> +++ b/xen/arch/riscv/aia.c
>> @@ -14,6 +14,7 @@
>>   #include <asm/cpufeature.h>
>>   #include <asm/csr.h>
>>   #include <asm/current.h>
>> +#include <asm/imsic.h>
>>   
>>   struct vgein_ctrl {
>>       unsigned long bmp;
>> @@ -36,6 +37,35 @@ bool aia_usable(void)
>>       return _aia_usable;
>>   }
>>   
>> +void vcpu_aia_init(struct vcpu *v)
>> +{
>> +    unsigned int new_vsfile_id;
>> +    int rc;
>> +
>> +    if ( !aia_usable() )
>> +        return;
>> +
>> +    new_vsfile_id = vgein_assign(v);
>> +
>> +    /*
>> +     * vgein_assign() returns 0 when no free h/w guest interrupt file is
>> +     * available (including GEILEN == 0); imsic_map_guest_file() maps nothing
>> +     * in that case.
>> +     */
>> +    rc = imsic_map_guest_file(v, new_vsfile_id);
>> +    if ( rc )
>> +    {
>> +        /* Can't continue w/o correctly mapped IMSIC interrupt file */
>> +        domain_crash(v->domain);
> 
> The vgein id assigned a few lines up is not released here. The domain is dying
> anyway, but the guest interrupt file stays marked in use on that pCPU forever,
> since nothing else ever calls vgein_release(). A vgein_release(v,
> new_vsfile_id) before the domain_crash() would fix it.
> 

Yes, agree with that vgein_release() is missed. I've mentioned that 
before in reply to Jan B.

Thanks!

~ Oleksii

Re: [PATCH v1 07/17] xen/riscv: introduce vCPU AIA initialization

[Jan Beulich]

On 13.08.2026 11:24, Baptiste Le Duc wrote:
>> --- a/xen/arch/riscv/aia.c
>> +++ b/xen/arch/riscv/aia.c
>> @@ -14,6 +14,7 @@
>>  #include <asm/cpufeature.h>
>>  #include <asm/csr.h>
>>  #include <asm/current.h>
>> +#include <asm/imsic.h>
>>  
>>  struct vgein_ctrl {
>>      unsigned long bmp;
>> @@ -36,6 +37,35 @@ bool aia_usable(void)
>>      return _aia_usable;
>>  }
>>  
>> +void vcpu_aia_init(struct vcpu *v)
>> +{
>> +    unsigned int new_vsfile_id;
>> +    int rc;
>> +
>> +    if ( !aia_usable() )
>> +        return;
>> +
>> +    new_vsfile_id = vgein_assign(v);
>> +
>> +    /*
>> +     * vgein_assign() returns 0 when no free h/w guest interrupt file is
>> +     * available (including GEILEN == 0); imsic_map_guest_file() maps nothing
>> +     * in that case.
>> +     */
>> +    rc = imsic_map_guest_file(v, new_vsfile_id);
>> +    if ( rc )
>> +    {
>> +        /* Can't continue w/o correctly mapped IMSIC interrupt file */
>> +        domain_crash(v->domain);
> 
> The vgein id assigned a few lines up is not released here. The domain is dying
> anyway, but the guest interrupt file stays marked in use on that pCPU forever,
> since nothing else ever calls vgein_release(). A vgein_release(v,
> new_vsfile_id) before the domain_crash() would fix it.

Instead of (or in addition to) doing that here, wouldn't releasing better be part
of the normal cleanup path? Whether "instead of" or "in addition to" depends on
the implications of deferring the release. But to guarantee no leak, domain
cleanup will want to either do the release, or have an explicit check that is was
done.

Jan

Re: [PATCH v1 07/17] xen/riscv: introduce vCPU AIA initialization

[Oleksii Kurochko]

On 8/13/26 11:47 AM, Jan Beulich wrote:
> On 13.08.2026 11:24, Baptiste Le Duc wrote:
>>> --- a/xen/arch/riscv/aia.c
>>> +++ b/xen/arch/riscv/aia.c
>>> @@ -14,6 +14,7 @@
>>>   #include <asm/cpufeature.h>
>>>   #include <asm/csr.h>
>>>   #include <asm/current.h>
>>> +#include <asm/imsic.h>
>>>   
>>>   struct vgein_ctrl {
>>>       unsigned long bmp;
>>> @@ -36,6 +37,35 @@ bool aia_usable(void)
>>>       return _aia_usable;
>>>   }
>>>   
>>> +void vcpu_aia_init(struct vcpu *v)
>>> +{
>>> +    unsigned int new_vsfile_id;
>>> +    int rc;
>>> +
>>> +    if ( !aia_usable() )
>>> +        return;
>>> +
>>> +    new_vsfile_id = vgein_assign(v);
>>> +
>>> +    /*
>>> +     * vgein_assign() returns 0 when no free h/w guest interrupt file is
>>> +     * available (including GEILEN == 0); imsic_map_guest_file() maps nothing
>>> +     * in that case.
>>> +     */
>>> +    rc = imsic_map_guest_file(v, new_vsfile_id);
>>> +    if ( rc )
>>> +    {
>>> +        /* Can't continue w/o correctly mapped IMSIC interrupt file */
>>> +        domain_crash(v->domain);
>>
>> The vgein id assigned a few lines up is not released here. The domain is dying
>> anyway, but the guest interrupt file stays marked in use on that pCPU forever,
>> since nothing else ever calls vgein_release(). A vgein_release(v,
>> new_vsfile_id) before the domain_crash() would fix it.
> 
> Instead of (or in addition to) doing that here, wouldn't releasing better be part
> of the normal cleanup path? Whether "instead of" or "in addition to" depends on
> the implications of deferring the release. But to guarantee no leak, domain
> cleanup will want to either do the release, or have an explicit check that is was
> done.

I think vgein_release() should be here, as when the IMSIC software 
interrupt file is supported, it will mean that domain_crash() can 
generally be dropped (there is no need to call imsic_map_guest_file() 
for s/w IMSIC interrupt file) and the vCPU can use the software 
interrupt file instead of killing the domain.

~ Oleksii

[PATCH v1 08/17] xen/riscv: add IMSIC state save/restore

[Oleksii Kurochko]

IMSIC state is currently needed only to track which physical CPU owns a
vCPU's IMSIC interrupt file. This is required because the physical CPU
ID is part of the physical address used to map the IMSIC file.

Add imsic_state_save() to record the current pCPU for a vCPU. When the
vCPU is migrated to a different pCPU, the mapping will need to be updated.

When imsic_state_restore() is called, VGEIN is already assigned to the
vCPU and the guest interrupt file is already mapped, and, as only h/w
interrupt files are used for now, nothing specific needs to be done.
Action is only required when the vCPU is moved to a different pCPU, which
requires recalculating VGEIN and the mapping for the new guest interrupt
file. That will be handled separately by vcpu_move_irqs(), which is
introduced in a follow-up patch; until then this case is guarded by a
BUG_ON(), which is fine.

Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 xen/arch/riscv/imsic.c             | 23 +++++++++++++++++++++++
 xen/arch/riscv/include/asm/imsic.h |  3 +++
 2 files changed, 26 insertions(+)

diff --git a/xen/arch/riscv/imsic.c b/xen/arch/riscv/imsic.c
index 2a792e756c4e..406bc68cbcd1 100644
--- a/xen/arch/riscv/imsic.c
+++ b/xen/arch/riscv/imsic.c
@@ -20,6 +20,7 @@
 #include <xen/init.h>
 #include <xen/libfdt/libfdt.h>
 #include <xen/macros.h>
+#include <xen/rwlock.h>
 #include <xen/sched.h>
 #include <xen/smp.h>
 #include <xen/spinlock.h>
@@ -418,6 +419,28 @@ int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id)
     return res;
 }
 
+void imsic_state_save(struct vcpu *v)
+{
+    struct vimsic_state *imsic_state = v->arch.vimsic_state;
+    unsigned long flags;
+
+    /*
+     * SW interrupt file always has ->vsfile_pcpu = NR_CPUS so nothing specific
+     * should be done in this case.
+     */
+    if ( !vcpu_guest_file_id(v) )
+        return;
+
+    write_lock_irqsave(&imsic_state->vsfile_lock, flags);
+    imsic_state->vsfile_pcpu = cpuid_to_hartid(v->processor);
+    write_unlock_irqrestore(&imsic_state->vsfile_lock, flags);
+}
+
+void imsic_state_restore(struct vcpu *v)
+{
+    /* Nothing to do */
+}
+
 int vcpu_imsic_init(struct vcpu *v)
 {
     struct vimsic_state *imsic_state;
diff --git a/xen/arch/riscv/include/asm/imsic.h b/xen/arch/riscv/include/asm/imsic.h
index 8e3797690f99..cfb968204a08 100644
--- a/xen/arch/riscv/include/asm/imsic.h
+++ b/xen/arch/riscv/include/asm/imsic.h
@@ -110,4 +110,7 @@ int vimsic_make_domu_dt_node(struct kernel_info *kinfo, unsigned int *phandle);
 
 int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id);
 
+void imsic_state_save(struct vcpu *v);
+void imsic_state_restore(struct vcpu *v);
+
 #endif /* ASM_RISCV_IMSIC_H */
-- 
2.54.0

Re: [PATCH v1 08/17] xen/riscv: add IMSIC state save/restore

[Jan Beulich]

On 20.07.2026 18:02, Oleksii Kurochko wrote:
> @@ -418,6 +419,28 @@ int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id)
>      return res;
>  }
>  
> +void imsic_state_save(struct vcpu *v)
> +{
> +    struct vimsic_state *imsic_state = v->arch.vimsic_state;
> +    unsigned long flags;
> +
> +    /*
> +     * SW interrupt file always has ->vsfile_pcpu = NR_CPUS so nothing specific
> +     * should be done in this case.
> +     */
> +    if ( !vcpu_guest_file_id(v) )
> +        return;

How does the ->vsfile_pcpu sentinel value matter here, when you're checking
->guest_file_id?

And anyway, there being dependencies like this one on the other big series
makes it rather hard to review things.

> +    write_lock_irqsave(&imsic_state->vsfile_lock, flags);
> +    imsic_state->vsfile_pcpu = cpuid_to_hartid(v->processor);

As discussed for another patch in this series, this will need to change then
as well.

> +    write_unlock_irqrestore(&imsic_state->vsfile_lock, flags);
> +}
> +
> +void imsic_state_restore(struct vcpu *v)
> +{
> +    /* Nothing to do */
> +}

"save" and "restore" have meaning other than what you intend here, aiui. Once
again without call sites it remains unclear when exactly these functions would
be called. Which makes it close to impossible to suggest better names.

Jan

Re: [PATCH v1 08/17] xen/riscv: add IMSIC state save/restore

[Oleksii Kurochko]

On 8/12/26 3:57 PM, Jan Beulich wrote:
> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>> @@ -418,6 +419,28 @@ int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id)
>>       return res;
>>   }
>>   
>> +void imsic_state_save(struct vcpu *v)
>> +{
>> +    struct vimsic_state *imsic_state = v->arch.vimsic_state;
>> +    unsigned long flags;
>> +
>> +    /*
>> +     * SW interrupt file always has ->vsfile_pcpu = NR_CPUS so nothing specific
>> +     * should be done in this case.
>> +     */
>> +    if ( !vcpu_guest_file_id(v) )
>> +        return;
> 
> How does the ->vsfile_pcpu sentinel value matter here, when you're checking
> ->guest_file_id?

Comment is incorrect. I will fix it.

> 
> And anyway, there being dependencies like this one on the other big series
> makes it rather hard to review things.
> 
>> +    write_lock_irqsave(&imsic_state->vsfile_lock, flags);
>> +    imsic_state->vsfile_pcpu = cpuid_to_hartid(v->processor);
> 
> As discussed for another patch in this series, this will need to change then
> as well.

I will update that properly.

> 
>> +    write_unlock_irqrestore(&imsic_state->vsfile_lock, flags);
>> +}
>> +
>> +void imsic_state_restore(struct vcpu *v)
>> +{
>> +    /* Nothing to do */
>> +}
> 
> "save" and "restore" have meaning other than what you intend here, aiui. Once
> again without call sites it remains unclear when exactly these functions would
> be called. Which makes it close to impossible to suggest better names.

I will add some extra context and/or re-shuffle patches to make it more 
clearer. Anyway as you explained me in another thread a name is really 
incorrect. I will use imsic_ctxt_switch_{to,from}() instead.

Thanks.

~ Oleksii

Re: [PATCH v1 08/17] xen/riscv: add IMSIC state save/restore

[Baptiste Le Duc]

> IMSIC state is currently needed only to track which physical CPU owns a
> vCPU's IMSIC interrupt file. This is required because the physical CPU
> ID is part of the physical address used to map the IMSIC file.
> 
> Add imsic_state_save() to record the current pCPU for a vCPU. When the
> vCPU is migrated to a different pCPU, the mapping will need to be updated.
> 
> When imsic_state_restore() is called, VGEIN is already assigned to the
> vCPU and the guest interrupt file is already mapped, and, as only h/w
> interrupt files are used for now, nothing specific needs to be done.
> Action is only required when the vCPU is moved to a different pCPU, which
> requires recalculating VGEIN and the mapping for the new guest interrupt
> file. That will be handled separately by vcpu_move_irqs(), which is
> introduced in a follow-up patch; until then this case is guarded by a
> BUG_ON(), which is fine.
> 
> Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>
> diff --git a/xen/arch/riscv/imsic.c b/xen/arch/riscv/imsic.c
> index 2a792e756c..406bc68cbc 100644
> --- a/xen/arch/riscv/imsic.c
> +++ b/xen/arch/riscv/imsic.c
> @@ -20,6 +20,7 @@
>  #include <xen/init.h>
>  #include <xen/libfdt/libfdt.h>
>  #include <xen/macros.h>
> +#include <xen/rwlock.h>
>  #include <xen/sched.h>
>  #include <xen/smp.h>
>  #include <xen/spinlock.h>
> @@ -418,6 +419,28 @@ int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id)
>      return res;
>  }
>  
> +void imsic_state_save(struct vcpu *v)
> +{
> +    struct vimsic_state *imsic_state = v->arch.vimsic_state;
> +    unsigned long flags;
> +
> +    /*
> +     * SW interrupt file always has ->vsfile_pcpu = NR_CPUS so nothing specific
> +     * should be done in this case.
> +     */
> +    if ( !vcpu_guest_file_id(v) )
> +        return;


> +
> +    write_lock_irqsave(&imsic_state->vsfile_lock, flags);
> +    imsic_state->vsfile_pcpu = cpuid_to_hartid(v->processor);

How will you detect a migration is needed? Don't you need to first know
if ->vsfile_pcpu is different to cpuid_to_hartid(v->processor)? (I
didn't take a look to other patchs for the moment, so the
explanations might be later.)

-- 
Baptiste Le Duc <baptiste.le-duc@vates.tech>

Re: [PATCH v1 08/17] xen/riscv: add IMSIC state save/restore

[Oleksii Kurochko]

On 8/13/26 11:30 AM, Baptiste Le Duc wrote:
>> IMSIC state is currently needed only to track which physical CPU owns a
>> vCPU's IMSIC interrupt file. This is required because the physical CPU
>> ID is part of the physical address used to map the IMSIC file.
>>
>> Add imsic_state_save() to record the current pCPU for a vCPU. When the
>> vCPU is migrated to a different pCPU, the mapping will need to be updated.
>>
>> When imsic_state_restore() is called, VGEIN is already assigned to the
>> vCPU and the guest interrupt file is already mapped, and, as only h/w
>> interrupt files are used for now, nothing specific needs to be done.
>> Action is only required when the vCPU is moved to a different pCPU, which
>> requires recalculating VGEIN and the mapping for the new guest interrupt
>> file. That will be handled separately by vcpu_move_irqs(), which is
>> introduced in a follow-up patch; until then this case is guarded by a
>> BUG_ON(), which is fine.
>>
>> Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>>
>> diff --git a/xen/arch/riscv/imsic.c b/xen/arch/riscv/imsic.c
>> index 2a792e756c..406bc68cbc 100644
>> --- a/xen/arch/riscv/imsic.c
>> +++ b/xen/arch/riscv/imsic.c
>> @@ -20,6 +20,7 @@
>>   #include <xen/init.h>
>>   #include <xen/libfdt/libfdt.h>
>>   #include <xen/macros.h>
>> +#include <xen/rwlock.h>
>>   #include <xen/sched.h>
>>   #include <xen/smp.h>
>>   #include <xen/spinlock.h>
>> @@ -418,6 +419,28 @@ int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id)
>>       return res;
>>   }
>>   
>> +void imsic_state_save(struct vcpu *v)
>> +{
>> +    struct vimsic_state *imsic_state = v->arch.vimsic_state;
>> +    unsigned long flags;
>> +
>> +    /*
>> +     * SW interrupt file always has ->vsfile_pcpu = NR_CPUS so nothing specific
>> +     * should be done in this case.
>> +     */
>> +    if ( !vcpu_guest_file_id(v) )
>> +        return;
> 
> 
>> +
>> +    write_lock_irqsave(&imsic_state->vsfile_lock, flags);
>> +    imsic_state->vsfile_pcpu = cpuid_to_hartid(v->processor);
> 
> How will you detect a migration is needed? Don't you need to first know
> if ->vsfile_pcpu is different to cpuid_to_hartid(v->processor)? (I
> didn't take a look to other patchs for the moment, so the
> explanations might be later.)
> 

Migration (if you are speaking about migration of vCPU from one pCPU to 
another) is completely different path. Look at sched_move_irqs().

~ Oleksii

Re: [PATCH v1 08/17] xen/riscv: add IMSIC state save/restore

[Jan Beulich]

On 13.08.2026 11:34, Oleksii Kurochko wrote:
> On 8/13/26 11:30 AM, Baptiste Le Duc wrote:
>>> --- a/xen/arch/riscv/imsic.c
>>> +++ b/xen/arch/riscv/imsic.c
>>> @@ -20,6 +20,7 @@
>>>   #include <xen/init.h>
>>>   #include <xen/libfdt/libfdt.h>
>>>   #include <xen/macros.h>
>>> +#include <xen/rwlock.h>
>>>   #include <xen/sched.h>
>>>   #include <xen/smp.h>
>>>   #include <xen/spinlock.h>
>>> @@ -418,6 +419,28 @@ int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id)
>>>       return res;
>>>   }
>>>   
>>> +void imsic_state_save(struct vcpu *v)
>>> +{
>>> +    struct vimsic_state *imsic_state = v->arch.vimsic_state;
>>> +    unsigned long flags;
>>> +
>>> +    /*
>>> +     * SW interrupt file always has ->vsfile_pcpu = NR_CPUS so nothing specific
>>> +     * should be done in this case.
>>> +     */
>>> +    if ( !vcpu_guest_file_id(v) )
>>> +        return;
>>
>>
>>> +
>>> +    write_lock_irqsave(&imsic_state->vsfile_lock, flags);
>>> +    imsic_state->vsfile_pcpu = cpuid_to_hartid(v->processor);
>>
>> How will you detect a migration is needed? Don't you need to first know
>> if ->vsfile_pcpu is different to cpuid_to_hartid(v->processor)? (I
>> didn't take a look to other patchs for the moment, so the
>> explanations might be later.)
> 
> Migration (if you are speaking about migration of vCPU from one pCPU to 
> another) is completely different path. Look at sched_move_irqs().

See how terminology is important. As said elsewhere, "save state" and
"restore state" don't make clear at all in which situation they're to be
used.

Also, can both of you please adjust Roger's email address when replying?

Jan

Re: [PATCH v1 08/17] xen/riscv: add IMSIC state save/restore

[Oleksii Kurochko]

On 8/13/26 11:51 AM, Jan Beulich wrote:
> On 13.08.2026 11:34, Oleksii Kurochko wrote:
>> On 8/13/26 11:30 AM, Baptiste Le Duc wrote:
>>>> --- a/xen/arch/riscv/imsic.c
>>>> +++ b/xen/arch/riscv/imsic.c
>>>> @@ -20,6 +20,7 @@
>>>>    #include <xen/init.h>
>>>>    #include <xen/libfdt/libfdt.h>
>>>>    #include <xen/macros.h>
>>>> +#include <xen/rwlock.h>
>>>>    #include <xen/sched.h>
>>>>    #include <xen/smp.h>
>>>>    #include <xen/spinlock.h>
>>>> @@ -418,6 +419,28 @@ int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id)
>>>>        return res;
>>>>    }
>>>>    
>>>> +void imsic_state_save(struct vcpu *v)
>>>> +{
>>>> +    struct vimsic_state *imsic_state = v->arch.vimsic_state;
>>>> +    unsigned long flags;
>>>> +
>>>> +    /*
>>>> +     * SW interrupt file always has ->vsfile_pcpu = NR_CPUS so nothing specific
>>>> +     * should be done in this case.
>>>> +     */
>>>> +    if ( !vcpu_guest_file_id(v) )
>>>> +        return;
>>>
>>>
>>>> +
>>>> +    write_lock_irqsave(&imsic_state->vsfile_lock, flags);
>>>> +    imsic_state->vsfile_pcpu = cpuid_to_hartid(v->processor);
>>>
>>> How will you detect a migration is needed? Don't you need to first know
>>> if ->vsfile_pcpu is different to cpuid_to_hartid(v->processor)? (I
>>> didn't take a look to other patchs for the moment, so the
>>> explanations might be later.)
>>
>> Migration (if you are speaking about migration of vCPU from one pCPU to
>> another) is completely different path. Look at sched_move_irqs().
> 
> See how terminology is important. As said elsewhere, "save state" and
> "restore state" don't make clear at all in which situation they're to be
> used.

I totally agree that it is important.

Just to clarify it now (before I started to re-shuffle and/or adding 
extra patches to have better context how this functions will be called) 
I will add some information here. So imsic_state_save() and 
imsic_state_restore() is going to be called from context_switch() 
function when one vCPU is de-scheduled and new vCPU is scheduled (so no 
migration here at all, yes it could happen but it is still a separate 
path and so separate question). Considering that my understanding that 
during context_switch() I have to save state of IMSIC which corresponds 
to vCPU which is going to be de-scheduled and restore a state of IMSIC 
of vCPU which is going to be scheduled.

With the current context is imsic_state_save() and imsic_state_restore() 
are correct names?

> 
> Also, can both of you please adjust Roger's email address when replying?

Could you please clarify what is wrong with it? For example, in this 
patch series:
   [PATCH v2 0/2] vpci: allow unaligned accesses by the hardware domain

This one is used: Roger Pau Monne <roger@xenproject.org>

~ Oleksii

Re: [PATCH v1 08/17] xen/riscv: add IMSIC state save/restore

[Jan Beulich]

On 13.08.2026 12:22, Oleksii Kurochko wrote:
> On 8/13/26 11:51 AM, Jan Beulich wrote:
>> On 13.08.2026 11:34, Oleksii Kurochko wrote:
>>> On 8/13/26 11:30 AM, Baptiste Le Duc wrote:
>>>>> --- a/xen/arch/riscv/imsic.c
>>>>> +++ b/xen/arch/riscv/imsic.c
>>>>> @@ -20,6 +20,7 @@
>>>>>    #include <xen/init.h>
>>>>>    #include <xen/libfdt/libfdt.h>
>>>>>    #include <xen/macros.h>
>>>>> +#include <xen/rwlock.h>
>>>>>    #include <xen/sched.h>
>>>>>    #include <xen/smp.h>
>>>>>    #include <xen/spinlock.h>
>>>>> @@ -418,6 +419,28 @@ int imsic_map_guest_file(struct vcpu *v, unsigned int vsfile_id)
>>>>>        return res;
>>>>>    }
>>>>>    
>>>>> +void imsic_state_save(struct vcpu *v)
>>>>> +{
>>>>> +    struct vimsic_state *imsic_state = v->arch.vimsic_state;
>>>>> +    unsigned long flags;
>>>>> +
>>>>> +    /*
>>>>> +     * SW interrupt file always has ->vsfile_pcpu = NR_CPUS so nothing specific
>>>>> +     * should be done in this case.
>>>>> +     */
>>>>> +    if ( !vcpu_guest_file_id(v) )
>>>>> +        return;
>>>>
>>>>
>>>>> +
>>>>> +    write_lock_irqsave(&imsic_state->vsfile_lock, flags);
>>>>> +    imsic_state->vsfile_pcpu = cpuid_to_hartid(v->processor);
>>>>
>>>> How will you detect a migration is needed? Don't you need to first know
>>>> if ->vsfile_pcpu is different to cpuid_to_hartid(v->processor)? (I
>>>> didn't take a look to other patchs for the moment, so the
>>>> explanations might be later.)
>>>
>>> Migration (if you are speaking about migration of vCPU from one pCPU to
>>> another) is completely different path. Look at sched_move_irqs().
>>
>> See how terminology is important. As said elsewhere, "save state" and
>> "restore state" don't make clear at all in which situation they're to be
>> used.
> 
> I totally agree that it is important.
> 
> Just to clarify it now (before I started to re-shuffle and/or adding 
> extra patches to have better context how this functions will be called) 
> I will add some information here. So imsic_state_save() and 
> imsic_state_restore() is going to be called from context_switch() 
> function when one vCPU is de-scheduled and new vCPU is scheduled (so no 
> migration here at all, yes it could happen but it is still a separate 
> path and so separate question). Considering that my understanding that 
> during context_switch() I have to save state of IMSIC which corresponds 
> to vCPU which is going to be de-scheduled and restore a state of IMSIC 
> of vCPU which is going to be scheduled.
> 
> With the current context is imsic_state_save() and imsic_state_restore() 
> are correct names?

No. "save" and "restore" would best be limited to migration paths (migration
of guests between hosts, that is). I can only once again suggest that you
look at existing naming in the code base. You'll find e.g.
svm_ctxt_switch_from() or vmx_ctxt_switch_to() under x86/hvm/.

>> Also, can both of you please adjust Roger's email address when replying?
> 
> Could you please clarify what is wrong with it? For example, in this 
> patch series:
>    [PATCH v2 0/2] vpci: allow unaligned accesses by the hardware domain
> 
> This one is used: Roger Pau Monne <roger@xenproject.org>

Whereas in your mail it was still Roger Pau Monné <roger.pau@citrix.com>.
When you originally posted the series, that was still correct. But in the
meantime it has changed (and I expect sending mail to the old address
wouldn't reach him anymore).

Jan

Re: [PATCH v1 08/17] xen/riscv: add IMSIC state save/restore

[Oleksii Kurochko]

On 8/13/26 12:44 PM, Jan Beulich wrote:
>>> Also, can both of you please adjust Roger's email address when replying?
>> Could you please clarify what is wrong with it? For example, in this
>> patch series:
>>     [PATCH v2 0/2] vpci: allow unaligned accesses by the hardware domain
>>
>> This one is used: Roger Pau Monne<roger@xenproject.org>
> Whereas in your mail it was still Roger Pau Monné<roger.pau@citrix.com>.
> When you originally posted the series, that was still correct. But in the
> meantime it has changed (and I expect sending mail to the old address
> wouldn't reach him anymore).

Thank for claryfing, that e-mail was added by ./add_mainterners.pl so it 
returns incorrect e-mail...


~ Oleksii

Re: [PATCH v1 08/17] xen/riscv: add IMSIC state save/restore

[Jan Beulich]

On 13.08.2026 12:56, Oleksii Kurochko wrote:
> On 8/13/26 12:44 PM, Jan Beulich wrote:
>>>> Also, can both of you please adjust Roger's email address when replying?
>>> Could you please clarify what is wrong with it? For example, in this
>>> patch series:
>>>     [PATCH v2 0/2] vpci: allow unaligned accesses by the hardware domain
>>>
>>> This one is used: Roger Pau Monne<roger@xenproject.org>
>> Whereas in your mail it was still Roger Pau Monné<roger.pau@citrix.com>.
>> When you originally posted the series, that was still correct. But in the
>> meantime it has changed (and I expect sending mail to the old address
>> wouldn't reach him anymore).
> 
> Thank for claryfing, that e-mail was added by ./add_mainterners.pl so it 
> returns incorrect e-mail...

No, I don't think it does. See commit ed3df2522ac7 from 2026-07-21. Your
series was sent a day earlier, so correctly with the Citrix address. But
when replying, people want to try to remember to switch stale email
addresses (in general; Roger merely happens to be affected right now).

Jan

[PATCH v1 09/17] xen/riscv: add helper to check APLIC MSI mode

[Oleksii Kurochko]

This helper can be used outside aplic.c to determine whether MSI mode
is enabled. A follow-up patch uses it to decide whether the guest
IMSIC state should be saved/restored.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 xen/arch/riscv/aplic.c             | 9 +++++++--
 xen/arch/riscv/include/asm/aplic.h | 2 ++
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/xen/arch/riscv/aplic.c b/xen/arch/riscv/aplic.c
index 87f2134bc561..1ce844cd2162 100644
--- a/xen/arch/riscv/aplic.c
+++ b/xen/arch/riscv/aplic.c
@@ -93,6 +93,11 @@ void aplic_hw_write_reg(unsigned int offset, uint32_t value)
     spin_unlock_irqrestore(&aplic.lock, flags);
 }
 
+bool has_msi_support(void)
+{
+    return readl(&aplic.regs->domaincfg) & APLIC_DOMAINCFG_DM;
+}
+
 static void __init aplic_init_hw_interrupts(void)
 {
     unsigned int i;
@@ -185,7 +190,7 @@ static void cf_check aplic_irq_enable(struct irq_desc *desc)
      *       If APLIC without MSI interrupts is required in the future,
      *       this function will need to be updated accordingly.
      */
-    ASSERT(readl(&aplic.regs->domaincfg) & APLIC_DOMAINCFG_DM);
+    ASSERT(has_msi_support());
 
     ASSERT(spin_is_locked(&desc->lock));
 
@@ -216,7 +221,7 @@ static void cf_check aplic_irq_disable(struct irq_desc *desc)
      *       If APLIC without MSI interrupts is required in the future,
      *       this function will need to be updated accordingly.
      */
-    ASSERT(readl(&aplic.regs->domaincfg) & APLIC_DOMAINCFG_DM);
+    ASSERT(has_msi_support());
 
     ASSERT(spin_is_locked(&desc->lock));
 
diff --git a/xen/arch/riscv/include/asm/aplic.h b/xen/arch/riscv/include/asm/aplic.h
index 4ae5fb8f26d1..9a0b23351154 100644
--- a/xen/arch/riscv/include/asm/aplic.h
+++ b/xen/arch/riscv/include/asm/aplic.h
@@ -144,4 +144,6 @@ struct aplic_regs {
 uint32_t aplic_hw_read_reg(unsigned int offset, uint32_t mask);
 void aplic_hw_write_reg(unsigned int offset, uint32_t value);
 
+bool has_msi_support(void);
+
 #endif /* ASM_RISCV_APLIC_H */
-- 
2.54.0

Re: [PATCH v1 09/17] xen/riscv: add helper to check APLIC MSI mode

[Jan Beulich]

On 20.07.2026 18:02, Oleksii Kurochko wrote:
> This helper can be used outside aplic.c to determine whether MSI mode
> is enabled. A follow-up patch uses it to decide whether the guest
> IMSIC state should be saved/restored.

How would this work, when ...

> --- a/xen/arch/riscv/aplic.c
> +++ b/xen/arch/riscv/aplic.c
> @@ -93,6 +93,11 @@ void aplic_hw_write_reg(unsigned int offset, uint32_t value)
>      spin_unlock_irqrestore(&aplic.lock, flags);
>  }
>  
> +bool has_msi_support(void)
> +{
> +    return readl(&aplic.regs->domaincfg) & APLIC_DOMAINCFG_DM;

... you read a global here? To know what state a guest's vAPLIC is in, you'd
need to read its (virtual) register, wouldn't you?

> +}

I think the name is overly ambiguous, the more that there's also no parameter
the type of which would help disambiguation. Judging from title and description,
maybe aplic_msi_mode_enabled() or simpler aplic_msi_mode() could be more to the
point. (Note that neither "has" nor "available" would really express things
correctly, as the DM field can [aiui] in principle be changed.)

Jan

Re: [PATCH v1 09/17] xen/riscv: add helper to check APLIC MSI mode

[Baptiste Le Duc]

> This helper can be used outside aplic.c to determine whether MSI mode
> is enabled. A follow-up patch uses it to decide whether the guest
> IMSIC state should be saved/restored.


> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>
> diff --git a/xen/arch/riscv/aplic.c b/xen/arch/riscv/aplic.c
> index 87f2134bc5..1ce844cd21 100644
> --- a/xen/arch/riscv/aplic.c
> +++ b/xen/arch/riscv/aplic.c
> @@ -93,6 +93,11 @@ void aplic_hw_write_reg(unsigned int offset, uint32_t value)
>      spin_unlock_irqrestore(&aplic.lock, flags);
>  }
>  
> +bool has_msi_support(void)

It does a readl() of the physical APLIC on every call. Patch 11 puts it on
the vCPU context switch path (vaplic_state_save/restore), so this becomes an
uncached MMIO read per switch to read a value that aplic_init_hw_interrupts()
sets once and nothing ever changes. Please cache it at init
time.

-- 
Baptiste Le Duc <baptiste.le-duc@vates.tech>

[PATCH v1 10/17] xen/riscv: introduce vintc_state_{save,restore}()

[Oleksii Kurochko]

Virtual interrupt controller state must be preserved across vCPU context
switches: for AIA, a vCPU's guest interrupt file lives in the IMSIC of
the pCPU it runs on, so the related state has to be saved when the vCPU
is descheduled and re-established when it is scheduled again.

Introduce vintc_state_save()/vintc_state_restore() wrappers around new
store_state()/restore_state() hooks in struct vintc_ops, so that the
context switch path can save/restore this state without knowing which
vINTC variant a domain uses.

No callers are wired up yet: the vAPLIC implementation of the hooks is
added by the follow-up patch, and the calls from the context switch path
will be introduced together with vCPU context switch support.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 xen/arch/riscv/include/asm/intc.h |  9 +++++++++
 xen/arch/riscv/intc.c             | 14 ++++++++++++++
 2 files changed, 23 insertions(+)

diff --git a/xen/arch/riscv/include/asm/intc.h b/xen/arch/riscv/include/asm/intc.h
index 2ee5d1533c8e..f4f0ce365c00 100644
--- a/xen/arch/riscv/include/asm/intc.h
+++ b/xen/arch/riscv/include/asm/intc.h
@@ -64,6 +64,12 @@ struct vintc_ops {
 
     /* Deinitialize some vINTC-related stuff for a vCPU */
     void (*vcpu_deinit)(struct vcpu *v);
+
+    /* Store virtual interrupt controller state */
+    void (*store_state)(struct vcpu *v);
+
+    /* Restore virtual interrupt controller state */
+    void (*restore_state)(struct vcpu *v);
 };
 
 struct vintc {
@@ -91,4 +97,7 @@ void domain_vintc_deinit(struct domain *d);
 
 bool vintc_reserve_virq(const struct domain *d, unsigned int virq);
 
+void vintc_state_save(struct vcpu *vcpu);
+void vintc_state_restore(struct vcpu *vcpu);
+
 #endif /* ASM__RISCV__INTERRUPT_CONTOLLER_H */
diff --git a/xen/arch/riscv/intc.c b/xen/arch/riscv/intc.c
index 372c8d3a20f9..879d51337491 100644
--- a/xen/arch/riscv/intc.c
+++ b/xen/arch/riscv/intc.c
@@ -163,3 +163,17 @@ bool vintc_reserve_virq(const struct domain *d, unsigned int virq)
 
     return !test_and_set_bit(virq, d->arch.vintc->used_irqs);
 }
+
+void vintc_state_save(struct vcpu *vcpu)
+{
+    const struct vintc_ops *ops = vcpu->domain->arch.vintc->ops;
+
+    ops->store_state(vcpu);
+}
+
+void vintc_state_restore(struct vcpu *vcpu)
+{
+    const struct vintc_ops *ops = vcpu->domain->arch.vintc->ops;
+
+    ops->restore_state(vcpu);
+}
-- 
2.54.0

Re: [PATCH v1 10/17] xen/riscv: introduce vintc_state_{save,restore}()

[Jan Beulich]

On 20.07.2026 18:02, Oleksii Kurochko wrote:
> Virtual interrupt controller state must be preserved across vCPU context
> switches: for AIA, a vCPU's guest interrupt file lives in the IMSIC of
> the pCPU it runs on, so the related state has to be saved when the vCPU
> is descheduled and re-established when it is scheduled again.
> 
> Introduce vintc_state_save()/vintc_state_restore() wrappers around new
> store_state()/restore_state() hooks in struct vintc_ops, so that the
> context switch path can save/restore this state without knowing which
> vINTC variant a domain uses.

Same issue with naming as mentioned for patch 08.

> No callers are wired up yet: the vAPLIC implementation of the hooks is
> added by the follow-up patch,

Neither "follow-up patch" nor "patch" alone nor "follow-up commit" should
appear in a description. You simply don't know how many other commits are
going to come between the two.

> --- a/xen/arch/riscv/include/asm/intc.h
> +++ b/xen/arch/riscv/include/asm/intc.h
> @@ -64,6 +64,12 @@ struct vintc_ops {
>  
>      /* Deinitialize some vINTC-related stuff for a vCPU */
>      void (*vcpu_deinit)(struct vcpu *v);
> +
> +    /* Store virtual interrupt controller state */
> +    void (*store_state)(struct vcpu *v);
> +
> +    /* Restore virtual interrupt controller state */
> +    void (*restore_state)(struct vcpu *v);
>  };

The parameters are properly named "v" here. Why ...

> @@ -91,4 +97,7 @@ void domain_vintc_deinit(struct domain *d);
>  
>  bool vintc_reserve_virq(const struct domain *d, unsigned int virq);
>  
> +void vintc_state_save(struct vcpu *vcpu);
> +void vintc_state_restore(struct vcpu *vcpu);

... is it "vcpu" here and ...

> --- a/xen/arch/riscv/intc.c
> +++ b/xen/arch/riscv/intc.c
> @@ -163,3 +163,17 @@ bool vintc_reserve_virq(const struct domain *d, unsigned int virq)
>  
>      return !test_and_set_bit(virq, d->arch.vintc->used_irqs);
>  }
> +
> +void vintc_state_save(struct vcpu *vcpu)
> +{
> +    const struct vintc_ops *ops = vcpu->domain->arch.vintc->ops;
> +
> +    ops->store_state(vcpu);
> +}
> +
> +void vintc_state_restore(struct vcpu *vcpu)
> +{
> +    const struct vintc_ops *ops = vcpu->domain->arch.vintc->ops;
> +
> +    ops->restore_state(vcpu);
> +}

... here? Consistent and predictable naming of parameters / variables _is_
important.

Jan

Re: [PATCH v1 10/17] xen/riscv: introduce vintc_state_{save,restore}()

[Baptiste Le Duc]

> Virtual interrupt controller state must be preserved across vCPU context
> switches: for AIA, a vCPU's guest interrupt file lives in the IMSIC of
> the pCPU it runs on, so the related state has to be saved when the vCPU
> is descheduled and re-established when it is scheduled again.
> 
> Introduce vintc_state_save()/vintc_state_restore() wrappers around new
> store_state()/restore_state() hooks in struct vintc_ops, so that the
> context switch path can save/restore this state without knowing which
> vINTC variant a domain uses.


> 
> No callers are wired up yet: the vAPLIC implementation of the hooks is


> added by the follow-up patch, and the calls from the context switch path
> will be introduced together with vCPU context switch support.
> 
> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>
> diff --git a/xen/arch/riscv/include/asm/intc.h b/xen/arch/riscv/include/asm/intc.h
> index 2ee5d1533c..f4f0ce365c 100644
> --- a/xen/arch/riscv/include/asm/intc.h
> +++ b/xen/arch/riscv/include/asm/intc.h
> @@ -64,6 +64,12 @@ struct vintc_ops {
>  
>      /* Deinitialize some vINTC-related stuff for a vCPU */
>      void (*vcpu_deinit)(struct vcpu *v);
> +
> +    /* Store virtual interrupt controller state */
> +    void (*store_state)(struct vcpu *v);
> +
> +    /* Restore virtual interrupt controller state */
> +    void (*restore_state)(struct vcpu *v);
>  };


>  
>  struct vintc {
> @@ -91,4 +97,7 @@ void domain_vintc_deinit(struct domain *d);
>  
>  bool vintc_reserve_virq(const struct domain *d, unsigned int virq);
>  
> +void vintc_state_save(struct vcpu *vcpu);


> +void vintc_state_restore(struct vcpu *vcpu);


> +
>  #endif /* ASM__RISCV__INTERRUPT_CONTOLLER_H */
> diff --git a/xen/arch/riscv/intc.c b/xen/arch/riscv/intc.c
> index 372c8d3a20..879d513374 100644
> --- a/xen/arch/riscv/intc.c
> +++ b/xen/arch/riscv/intc.c
> @@ -163,3 +163,17 @@ bool vintc_reserve_virq(const struct domain *d, unsigned int virq)
>  
>      return !test_and_set_bit(virq, d->arch.vintc->used_irqs);
>  }
> +
> +void vintc_state_save(struct vcpu *vcpu)
> +{
> +    const struct vintc_ops *ops = vcpu->domain->arch.vintc->ops;
Is there a situation where ops could be NULL? If yes, add a check.
> +
> +    ops->store_state(vcpu);
> +}
> +
> +void vintc_state_restore(struct vcpu *vcpu)
> +{
> +    const struct vintc_ops *ops = vcpu->domain->arch.vintc->ops;
Same as above

-- 
Baptiste Le Duc <baptiste.le-duc@vates.tech>

Re: [PATCH v1 10/17] xen/riscv: introduce vintc_state_{save,restore}()

[Oleksii Kurochko]

On 8/13/26 11:42 AM, Baptiste Le Duc wrote:
>>   #endif /* ASM__RISCV__INTERRUPT_CONTOLLER_H */
>> diff --git a/xen/arch/riscv/intc.c b/xen/arch/riscv/intc.c
>> index 372c8d3a20..879d513374 100644
>> --- a/xen/arch/riscv/intc.c
>> +++ b/xen/arch/riscv/intc.c
>> @@ -163,3 +163,17 @@ bool vintc_reserve_virq(const struct domain *d, unsigned int virq)
>>   
>>       return !test_and_set_bit(virq, d->arch.vintc->used_irqs);
>>   }
>> +
>> +void vintc_state_save(struct vcpu *vcpu)
>> +{
>> +    const struct vintc_ops *ops = vcpu->domain->arch.vintc->ops;
> Is there a situation where ops could be NULL? If yes, add a check.

It is unlikely that there is nothing to do during a context switch for 
vINTC, so vINTC should provide an implementation for saving and 
restoring its context. This also ensures that a NULL pointer dereference 
will lead to a trap, allowing us to catch cases where a 
context-switch/restore handler is missing.

Even if it turns out that vINTC does not need to perform any actions 
during a context switch, it is perfectly fine to provide an empty 
implementation. However, as mentioned above, this is unlikely. 
Therefore, having a NULL pointer dereference here is intentional: it 
helps catch cases where someone adds a new interrupt controller driver 
but forgets to implement the corresponding context switch functionality.

~ Oleksii

Re: [PATCH v1 10/17] xen/riscv: introduce vintc_state_{save,restore}()

[Baptiste Le Duc]

On 2026-08-17 10:31 +0200, Oleksii Kurochko wrote:
> 
> 
> On 8/13/26 11:42 AM, Baptiste Le Duc wrote:
> >>   #endif /* ASM__RISCV__INTERRUPT_CONTOLLER_H */
> >> diff --git a/xen/arch/riscv/intc.c b/xen/arch/riscv/intc.c
> >> index 372c8d3a20..879d513374 100644
> >> --- a/xen/arch/riscv/intc.c
> >> +++ b/xen/arch/riscv/intc.c
> >> @@ -163,3 +163,17 @@ bool vintc_reserve_virq(const struct domain *d, unsigned int virq)
> >>   
> >>       return !test_and_set_bit(virq, d->arch.vintc->used_irqs);
> >>   }
> >> +
> >> +void vintc_state_save(struct vcpu *vcpu)
> >> +{
> >> +    const struct vintc_ops *ops = vcpu->domain->arch.vintc->ops;
> > Is there a situation where ops could be NULL? If yes, add a check.
> 
> It is unlikely that there is nothing to do during a context switch for 
> vINTC, so vINTC should provide an implementation for saving and 
> restoring its context. This also ensures that a NULL pointer dereference 
> will lead to a trap, allowing us to catch cases where a 
> context-switch/restore handler is missing.
> 
> Even if it turns out that vINTC does not need to perform any actions 
> during a context switch, it is perfectly fine to provide an empty 
> implementation. However, as mentioned above, this is unlikely. 
> Therefore, having a NULL pointer dereference here is intentional: it 
> helps catch cases where someone adds a new interrupt controller driver 
> but forgets to implement the corresponding context switch functionality.

Thanks for these explanations. However, wouldn't it be better to have a
dedicated BUG_ON in case of NULL dereference to indicate clean call
trace to people who missed to implement context-switch functionality?

> 
> ~ Oleksii
> 
> 

Re: [PATCH v1 10/17] xen/riscv: introduce vintc_state_{save,restore}()

[Jan Beulich]

On 18.08.2026 10:28, Baptiste Le Duc wrote:
> On 2026-08-17 10:31 +0200, Oleksii Kurochko wrote:
>>
>>
>> On 8/13/26 11:42 AM, Baptiste Le Duc wrote:
>>>>   #endif /* ASM__RISCV__INTERRUPT_CONTOLLER_H */
>>>> diff --git a/xen/arch/riscv/intc.c b/xen/arch/riscv/intc.c
>>>> index 372c8d3a20..879d513374 100644
>>>> --- a/xen/arch/riscv/intc.c
>>>> +++ b/xen/arch/riscv/intc.c
>>>> @@ -163,3 +163,17 @@ bool vintc_reserve_virq(const struct domain *d, unsigned int virq)
>>>>   
>>>>       return !test_and_set_bit(virq, d->arch.vintc->used_irqs);
>>>>   }
>>>> +
>>>> +void vintc_state_save(struct vcpu *vcpu)
>>>> +{
>>>> +    const struct vintc_ops *ops = vcpu->domain->arch.vintc->ops;
>>> Is there a situation where ops could be NULL? If yes, add a check.
>>
>> It is unlikely that there is nothing to do during a context switch for 
>> vINTC, so vINTC should provide an implementation for saving and 
>> restoring its context. This also ensures that a NULL pointer dereference 
>> will lead to a trap, allowing us to catch cases where a 
>> context-switch/restore handler is missing.
>>
>> Even if it turns out that vINTC does not need to perform any actions 
>> during a context switch, it is perfectly fine to provide an empty 
>> implementation. However, as mentioned above, this is unlikely. 
>> Therefore, having a NULL pointer dereference here is intentional: it 
>> helps catch cases where someone adds a new interrupt controller driver 
>> but forgets to implement the corresponding context switch functionality.
> 
> Thanks for these explanations. However, wouldn't it be better to have a
> dedicated BUG_ON in case of NULL dereference to indicate clean call
> trace to people who missed to implement context-switch functionality?

How would BUG_ON() provide any better (or worse) call trace, compared to
a NULL deref?

Jan

Re: [PATCH v1 10/17] xen/riscv: introduce vintc_state_{save,restore}()

[Baptiste Le Duc]

On 2026-08-18 10:31 +0200, Jan Beulich wrote:
> On 18.08.2026 10:28, Baptiste Le Duc wrote:
> > On 2026-08-17 10:31 +0200, Oleksii Kurochko wrote:
> >>
> >>
> >> On 8/13/26 11:42 AM, Baptiste Le Duc wrote:
> >>>>   #endif /* ASM__RISCV__INTERRUPT_CONTOLLER_H */
> >>>> diff --git a/xen/arch/riscv/intc.c b/xen/arch/riscv/intc.c
> >>>> index 372c8d3a20..879d513374 100644
> >>>> --- a/xen/arch/riscv/intc.c
> >>>> +++ b/xen/arch/riscv/intc.c
> >>>> @@ -163,3 +163,17 @@ bool vintc_reserve_virq(const struct domain *d, unsigned int virq)
> >>>>   
> >>>>       return !test_and_set_bit(virq, d->arch.vintc->used_irqs);
> >>>>   }
> >>>> +
> >>>> +void vintc_state_save(struct vcpu *vcpu)
> >>>> +{
> >>>> +    const struct vintc_ops *ops = vcpu->domain->arch.vintc->ops;
> >>> Is there a situation where ops could be NULL? If yes, add a check.
> >>
> >> It is unlikely that there is nothing to do during a context switch for 
> >> vINTC, so vINTC should provide an implementation for saving and 
> >> restoring its context. This also ensures that a NULL pointer dereference 
> >> will lead to a trap, allowing us to catch cases where a 
> >> context-switch/restore handler is missing.
> >>
> >> Even if it turns out that vINTC does not need to perform any actions 
> >> during a context switch, it is perfectly fine to provide an empty 
> >> implementation. However, as mentioned above, this is unlikely. 
> >> Therefore, having a NULL pointer dereference here is intentional: it 
> >> helps catch cases where someone adds a new interrupt controller driver 
> >> but forgets to implement the corresponding context switch functionality.
> > 
> > Thanks for these explanations. However, wouldn't it be better to have a
> > dedicated BUG_ON in case of NULL dereference to indicate clean call
> > trace to people who missed to implement context-switch functionality?
> 
> How would BUG_ON() provide any better (or worse) call trace, compared to
> a NULL deref?

I wanted the file:line and function printed directly, but sepc in the
trap's register dump resolves to the same place, and BUG_ON() ends up in
the same handler anyway. Fair enough, dropping it.

Thanks,
Baptiste

> 
> Jan
> 
> 
> 

[PATCH v1 11/17] xen/riscv: add vAPLIC state save/restore hooks

[Oleksii Kurochko]

vAPLIC state needs to be saved and restored as part of vCPU context
management.

Introduce vaplic_state_save() and vaplic_state_restore() and wire them
to the IMSIC state save/restore helpers when MSI is available. The
functions are currently no-ops on platforms without MSI support and
will lead to BUG_ON() to not miss add support of no-MSI case.

Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 xen/arch/riscv/include/asm/vaplic.h |  3 +++
 xen/arch/riscv/vaplic.c             | 18 ++++++++++++++++++
 2 files changed, 21 insertions(+)

diff --git a/xen/arch/riscv/include/asm/vaplic.h b/xen/arch/riscv/include/asm/vaplic.h
index 7bf9247f4eae..fbd224b9a34a 100644
--- a/xen/arch/riscv/include/asm/vaplic.h
+++ b/xen/arch/riscv/include/asm/vaplic.h
@@ -34,4 +34,7 @@ struct vaplic {
 int domain_vaplic_init(struct domain *d);
 void domain_vaplic_deinit(struct domain *d);
 
+void vaplic_state_save(struct vcpu *v);
+void vaplic_state_restore(struct vcpu *v);
+
 #endif /* ASM__RISCV__VAPLIC_H */
diff --git a/xen/arch/riscv/vaplic.c b/xen/arch/riscv/vaplic.c
index 03240730e344..0723aad9558e 100644
--- a/xen/arch/riscv/vaplic.c
+++ b/xen/arch/riscv/vaplic.c
@@ -400,9 +400,27 @@ static const struct mmio_handler_ops vaplic_mmio_ops = {
     .write = vaplic_mmio_write,
 };
 
+void vaplic_state_save(struct vcpu *v)
+{
+    if ( has_msi_support() )
+        imsic_state_save(v);
+    else
+        BUG_ON("unimplemented");
+}
+
+void vaplic_state_restore(struct vcpu *v)
+{
+    if ( has_msi_support() )
+        imsic_state_restore(v);
+    else
+        BUG_ON("unimplemented");
+}
+
 static const struct vintc_ops vintc_ops = {
     .vcpu_init = vaplic_init,
     .vcpu_deinit = vaplic_deinit,
+    .store_state = vaplic_state_save,
+    .restore_state = vaplic_state_restore,
 };
 
 int domain_vaplic_init(struct domain *d)
-- 
2.54.0

Re: [PATCH v1 11/17] xen/riscv: add vAPLIC state save/restore hooks

[Jan Beulich]

On 20.07.2026 18:02, Oleksii Kurochko wrote:
> --- a/xen/arch/riscv/include/asm/vaplic.h
> +++ b/xen/arch/riscv/include/asm/vaplic.h
> @@ -34,4 +34,7 @@ struct vaplic {
>  int domain_vaplic_init(struct domain *d);
>  void domain_vaplic_deinit(struct domain *d);
>  
> +void vaplic_state_save(struct vcpu *v);
> +void vaplic_state_restore(struct vcpu *v);

Why would these be needed? Can't ...

> --- a/xen/arch/riscv/vaplic.c
> +++ b/xen/arch/riscv/vaplic.c
> @@ -400,9 +400,27 @@ static const struct mmio_handler_ops vaplic_mmio_ops = {
>      .write = vaplic_mmio_write,
>  };
>  
> +void vaplic_state_save(struct vcpu *v)

... both be static? And don't they want to be cf_check?

> +{
> +    if ( has_msi_support() )
> +        imsic_state_save(v);
> +    else
> +        BUG_ON("unimplemented");
> +}
> +
> +void vaplic_state_restore(struct vcpu *v)
> +{
> +    if ( has_msi_support() )
> +        imsic_state_restore(v);
> +    else
> +        BUG_ON("unimplemented");
> +}

If you're merely forwarding the calls, why can't ...

>  static const struct vintc_ops vintc_ops = {
>      .vcpu_init = vaplic_init,
>      .vcpu_deinit = vaplic_deinit,
> +    .store_state = vaplic_state_save,
> +    .restore_state = vaplic_state_restore,

... imsic_state_{save,restore}() be used directly here? And whatever other
pair of handlers for the case when it's not IMSIC?

Jan

Re: [PATCH v1 11/17] xen/riscv: add vAPLIC state save/restore hooks

[Oleksii Kurochko]

On 8/12/26 4:19 PM, Jan Beulich wrote:
> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>> --- a/xen/arch/riscv/include/asm/vaplic.h
>> +++ b/xen/arch/riscv/include/asm/vaplic.h
>> @@ -34,4 +34,7 @@ struct vaplic {
>>   int domain_vaplic_init(struct domain *d);
>>   void domain_vaplic_deinit(struct domain *d);
>>   
>> +void vaplic_state_save(struct vcpu *v);
>> +void vaplic_state_restore(struct vcpu *v);
> 
> Why would these be needed? Can't ...
> 
>> --- a/xen/arch/riscv/vaplic.c
>> +++ b/xen/arch/riscv/vaplic.c
>> @@ -400,9 +400,27 @@ static const struct mmio_handler_ops vaplic_mmio_ops = {
>>       .write = vaplic_mmio_write,
>>   };
>>   
>> +void vaplic_state_save(struct vcpu *v)
> 
> ... both be static? 

They are only needed to cover potentially two cases (w/ MSI and w/o MSI 
support) but I see a sense two follow your suggestion below ...

> And don't they want to be cf_check?

Agree, cf_check should be used here.

> 
>> +{
>> +    if ( has_msi_support() )
>> +        imsic_state_save(v);
>> +    else
>> +        BUG_ON("unimplemented");
>> +}
>> +
>> +void vaplic_state_restore(struct vcpu *v)
>> +{
>> +    if ( has_msi_support() )
>> +        imsic_state_restore(v);
>> +    else
>> +        BUG_ON("unimplemented");
>> +}
> 
> If you're merely forwarding the calls, why can't ...
> 
>>   static const struct vintc_ops vintc_ops = {
>>       .vcpu_init = vaplic_init,
>>       .vcpu_deinit = vaplic_deinit,
>> +    .store_state = vaplic_state_save,
>> +    .restore_state = vaplic_state_restore,
> 
> ... imsic_state_{save,restore}() be used directly here? And whatever other
> pair of handlers for the case when it's not IMSIC?

... It could be done in that way. I will follow it.

Thanks.

~ Oleksii

[PATCH v1 12/17] xen/riscv: extend exception tables with type and data fields

[Oleksii Kurochko]

Extend the RISC-V exception table format to include a type and
auxiliary data field.

The existing format only supports simple fixups. Some use cases require
additional context from the fault (e.g. capturing trap information),
which cannot be expressed with the current EX_TYPE_FIXUP entries.

Introduce a generic ASM_EXTABLE_RAW() helper to describe entries with a
handler type and associated data. Reimplement ASM_EXTABLE() in terms of
it using EX_TYPE_FIXUP for compatibility.

Add EX_TYPE_TRAP_INFO to allow handlers to retrieve trap state
(sepc/scause/stval) and pass it to the fixup path. The data field is
used to encode which GPR contains a pointer to a struct trap_info.

Provide ASM_EXTABLE_TRAP_INFO() as a convenience wrapper for this case.

Also add gpr-num.h, providing symbolic GPR numbers for use in assembly
and inline asm. This is derived from Linux 6.16 with minor adjustments such
as using .irp instead of open-coding the same using a set of .equ.

Update the exception handling code to dispatch based on the entry type.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 xen/arch/riscv/extable.c               | 60 ++++++++++++++++++++++++-
 xen/arch/riscv/include/asm/extable.h   | 62 ++++++++++++++++++--------
 xen/arch/riscv/include/asm/gpr-num.h   | 33 ++++++++++++++
 xen/arch/riscv/include/asm/processor.h |  2 +
 xen/arch/riscv/include/asm/traps.h     |  6 +++
 5 files changed, 143 insertions(+), 20 deletions(-)
 create mode 100644 xen/arch/riscv/include/asm/gpr-num.h

diff --git a/xen/arch/riscv/extable.c b/xen/arch/riscv/extable.c
index 77e5e9e89439..e0ced0537182 100644
--- a/xen/arch/riscv/extable.c
+++ b/xen/arch/riscv/extable.c
@@ -7,8 +7,10 @@
 #include <xen/sort.h>
 #include <xen/virtual_region.h>
 
+#include <asm/csr.h>
 #include <asm/extable.h>
 #include <asm/processor.h>
+#include <asm/traps.h>
 
 #define EX_FIELD(ptr, field) ((unsigned long)&(ptr)->field + (ptr)->field)
 
@@ -33,6 +35,12 @@ static void __init cf_check swap_ex(void *a, void *b)
 
     x->fixup = y->fixup + delta;
     y->fixup = tmp.fixup - delta;
+
+    x->type = y->type;
+    y->type = tmp.type;
+
+    x->data = y->data;
+    y->data = tmp.data;
 }
 
 static int cf_check cmp_ex(const void *a, const void *b)
@@ -60,6 +68,40 @@ static void ex_handler_fixup(const struct exception_table_entry *ex,
     regs->sepc = ex_fixup(ex);
 }
 
+static inline unsigned long regs_get_gpr(struct cpu_user_regs *regs,
+                                         unsigned int offset)
+{
+    /*
+     * The GPR number -> offset arithmetic below relies on x0..x31 being
+     * laid out at the start of struct cpu_user_regs in architectural
+     * order.
+     */
+    BUILD_BUG_ON(offsetof(struct cpu_user_regs, ra) !=
+                 sizeof(unsigned long));
+    BUILD_BUG_ON(offsetof(struct cpu_user_regs, t6) !=
+                 31 * sizeof(unsigned long));
+
+    if ( unlikely(!offset || (offset > MAX_REG_OFFSET)) )
+        return 0;
+
+    return *(unsigned long *)((unsigned long)regs + offset);
+}
+
+static void ex_handler_trap_info(const struct exception_table_entry *ex,
+                                 struct cpu_user_regs *regs)
+{
+    struct trap_info *trap_info =
+        (struct trap_info *)regs_get_gpr(regs, ex->data * sizeof(unsigned long));
+
+    BUG_ON(!trap_info);
+
+    trap_info->sepc = csr_read(CSR_SEPC);
+    trap_info->scause = csr_read(CSR_SCAUSE);
+    trap_info->stval = csr_read(CSR_STVAL);
+
+    regs->sepc = ex_fixup(ex);
+}
+
 bool fixup_exception(struct cpu_user_regs *regs)
 {
     unsigned long pc = regs->sepc;
@@ -78,7 +120,23 @@ bool fixup_exception(struct cpu_user_regs *regs)
     if ( !ex )
         return false;
 
-    ex_handler_fixup(ex, regs);
+    switch ( ex->type )
+    {
+    case EX_TYPE_FIXUP:
+        ex_handler_fixup(ex, regs);
+        break;
+
+    case EX_TYPE_TRAP_INFO:
+        ex_handler_trap_info(ex, regs);
+        break;
+
+    default:
+        printk(XENLOG_ERR
+               "Unsupported exception table entry type %u for pc %#lx\n",
+               ex->type, pc);
+
+        return false;
+    }
 
     return true;
 }
diff --git a/xen/arch/riscv/include/asm/extable.h b/xen/arch/riscv/include/asm/extable.h
index c0128a91818f..287ce962b5cf 100644
--- a/xen/arch/riscv/include/asm/extable.h
+++ b/xen/arch/riscv/include/asm/extable.h
@@ -3,17 +3,24 @@
 #ifndef ASM__RISCV__ASM_EXTABLE_H
 #define ASM__RISCV__ASM_EXTABLE_H
 
+#include <asm/gpr-num.h>
+
+#define EX_TYPE_FIXUP 0
+#define EX_TYPE_TRAP_INFO 1
+
 #ifdef __ASSEMBLER__
 
-#define ASM_EXTABLE(insn, fixup) \
-    .pushsection .ex_table, "a"; \
-    .balign     4;               \
-    .word       (insn) - .;      \
-    .word       (fixup) - .;     \
-    .popsection
+#define ASM_EXTABLE_RAW(insn, fixup, type, data)    \
+    .pushsection .ex_table, "a";                    \
+    .balign     4;                                  \
+    .long       ((insn) - .);                       \
+    .long       ((fixup) - .);                      \
+    .short      (type);                             \
+    .short      (data);                             \
+    .popsection;
 
-.macro asm_extable, insn, fixup
-    ASM_EXTABLE(\insn, \fixup)
+.macro _asm_extable, insn, fixup
+    ASM_EXTABLE_RAW(\insn, \fixup, EX_TYPE_FIXUP, 0)
 .endm
 
 #else /* __ASSEMBLER__ */
@@ -23,20 +30,36 @@
 
 struct cpu_user_regs;
 
-#define ASM_EXTABLE(insn, fixup)      \
-    ".pushsection .ex_table, \"a\"\n" \
-    ".balign    4\n"                  \
-    ".word      (" #insn " - .)\n"    \
-    ".word      (" #fixup " - .)\n"   \
+#define ASM_EXTABLE_RAW(insn, fixup, type, data)    \
+    ".pushsection .ex_table, \"a\"\n"               \
+    ".balign    4\n"                                \
+    ".long      ((" insn ") - .)\n"                 \
+    ".long      ((" fixup ") - .)\n"                \
+    ".short      (" type ")\n"                      \
+    ".short      (" data ")\n"                      \
     ".popsection\n"
 
+#define ASM_EXTABLE(insn, fixup)    \
+    ASM_EXTABLE_RAW(#insn, #fixup, __stringify(EX_TYPE_FIXUP), "0")
+
+#define EX_TRAP_INFO_REG(gpr)   \
+    "(.L_gpr_num_" #gpr ")"
+
+#define ASM_EXTABLE_TRAP_INFO(insn, fixup, data)                    \
+    DEFINE_ASM_GPR_NUMS                                             \
+    ASM_EXTABLE_RAW(#insn, #fixup, __stringify(EX_TYPE_TRAP_INFO),  \
+                    EX_TRAP_INFO_REG(data))
+
 /*
- * The exception table consists of pairs of relative offsets: the first
- * is the relative offset to an instruction that is allowed to fault,
- * and the second is the relative offset at which the program should
- * continue. No general-purpose registers are modified by the exception
- * handling mechanism itself, so it is up to the fixup code to handle
- * any necessary state cleanup.
+ * Each exception table entry consists of two relative offsets and a
+ * handler description: `insn` is the relative offset to an instruction
+ * that is allowed to fault, `fixup` is the relative offset at which the
+ * program should continue, `type` selects how the exception is handled
+ * (EX_TYPE_*), and `data` holds auxiliary information for the handler
+ * (e.g. for EX_TYPE_TRAP_INFO, the number of the GPR that contains a
+ * pointer to a struct trap_info). No general-purpose registers are
+ * modified by the exception handling mechanism itself, so it is up to
+ * the fixup code to handle any necessary state cleanup.
  *
  * The exception table and fixup code live out of line with the main
  * instruction path. This means when everything is well, we don't even
@@ -45,6 +68,7 @@ struct cpu_user_regs;
  */
 struct exception_table_entry {
     int32_t insn, fixup;
+    uint16_t type, data;
 };
 
 extern struct exception_table_entry __start___ex_table[];
diff --git a/xen/arch/riscv/include/asm/gpr-num.h b/xen/arch/riscv/include/asm/gpr-num.h
new file mode 100644
index 000000000000..1578f55cbd97
--- /dev/null
+++ b/xen/arch/riscv/include/asm/gpr-num.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef RISCV_GPR_NUM_H
+#define RISCV_GPR_NUM_H
+
+/* GPR ABI names, in register-number order (x0 .. x31). */
+#define GPR_ABI_NAMES                   \
+    zero, ra, sp, gp, tp, t0, t1, t2,   \
+    s0, s1, a0, a1, a2, a3, a4, a5,     \
+    a6, a7, s2, s3, s4, s5, s6, s7,     \
+    s8, s9, s10, s11, t3, t4, t5, t6
+
+#ifdef __ASSEMBLER__
+
+    .equ    .L_gpr_num, 0
+    .irp    name, GPR_ABI_NAMES
+    .equ    .L_gpr_num_\name, .L_gpr_num
+    .equ    .L_gpr_num, .L_gpr_num + 1
+    .endr
+
+#else /* __ASSEMBLER__ */
+
+#include <xen/stringify.h>
+
+#define DEFINE_ASM_GPR_NUMS                         \
+"   .equ    .L_gpr_num, 0\n"                        \
+"   .irp    name, " __stringify(GPR_ABI_NAMES) "\n" \
+"   .equ    .L_gpr_num_\\name, .L_gpr_num\n"        \
+"   .equ    .L_gpr_num, .L_gpr_num + 1\n"           \
+"   .endr\n"
+
+#endif /* __ASSEMBLER__ */
+
+#endif /* RISCV_GPR_NUM_H */
diff --git a/xen/arch/riscv/include/asm/processor.h b/xen/arch/riscv/include/asm/processor.h
index 6b89df4a2d4f..a5ccfa61bb9f 100644
--- a/xen/arch/riscv/include/asm/processor.h
+++ b/xen/arch/riscv/include/asm/processor.h
@@ -54,6 +54,8 @@ struct cpu_user_regs
     unsigned long pregs;
 };
 
+#define MAX_REG_OFFSET offsetof(struct cpu_user_regs, t6)
+
 /* TODO: need to implement */
 #define cpu_to_core(cpu)   0
 #define cpu_to_socket(cpu) 0
diff --git a/xen/arch/riscv/include/asm/traps.h b/xen/arch/riscv/include/asm/traps.h
index 21fa3c3259b3..8d4ab664bca9 100644
--- a/xen/arch/riscv/include/asm/traps.h
+++ b/xen/arch/riscv/include/asm/traps.h
@@ -7,6 +7,12 @@
 
 #ifndef __ASSEMBLER__
 
+struct trap_info {
+    register_t sepc;
+    register_t scause;
+    register_t stval;
+};
+
 void do_trap(struct cpu_user_regs *cpu_regs);
 void handle_trap(void);
 void trap_init(void);
-- 
2.54.0

Re: [PATCH v1 12/17] xen/riscv: extend exception tables with type and data fields

[Jan Beulich]

On 20.07.2026 18:02, Oleksii Kurochko wrote:
> @@ -60,6 +68,40 @@ static void ex_handler_fixup(const struct exception_table_entry *ex,
>      regs->sepc = ex_fixup(ex);
>  }
>  
> +static inline unsigned long regs_get_gpr(struct cpu_user_regs *regs,
> +                                         unsigned int offset)
> +{
> +    /*
> +     * The GPR number -> offset arithmetic below relies on x0..x31 being
> +     * laid out at the start of struct cpu_user_regs in architectural
> +     * order.
> +     */
> +    BUILD_BUG_ON(offsetof(struct cpu_user_regs, ra) !=
> +                 sizeof(unsigned long));
> +    BUILD_BUG_ON(offsetof(struct cpu_user_regs, t6) !=
> +                 31 * sizeof(unsigned long));
> +
> +    if ( unlikely(!offset || (offset > MAX_REG_OFFSET)) )
> +        return 0;

And an offset not divisible by sizeof(unsigned long) is okay?

Returning 0 as error indicator also feels fragile.

> +    return *(unsigned long *)((unsigned long)regs + offset);
> +}
> +
> +static void ex_handler_trap_info(const struct exception_table_entry *ex,
> +                                 struct cpu_user_regs *regs)
> +{
> +    struct trap_info *trap_info =
> +        (struct trap_info *)regs_get_gpr(regs, ex->data * sizeof(unsigned long));

Related to the earlier comment: Simply pass just ex->data here, leaving the
multiplication to regs_get_gpr()?

> +    BUG_ON(!trap_info);
> +
> +    trap_info->sepc = csr_read(CSR_SEPC);
> +    trap_info->scause = csr_read(CSR_SCAUSE);
> +    trap_info->stval = csr_read(CSR_STVAL);

Do you really need to re-read all three registers here? Didn't you read at least
scause already, in order to make it here in the first place?

> --- a/xen/arch/riscv/include/asm/extable.h
> +++ b/xen/arch/riscv/include/asm/extable.h
> @@ -3,17 +3,24 @@
>  #ifndef ASM__RISCV__ASM_EXTABLE_H
>  #define ASM__RISCV__ASM_EXTABLE_H
>  
> +#include <asm/gpr-num.h>
> +
> +#define EX_TYPE_FIXUP 0
> +#define EX_TYPE_TRAP_INFO 1
> +
>  #ifdef __ASSEMBLER__
>  
> -#define ASM_EXTABLE(insn, fixup) \
> -    .pushsection .ex_table, "a"; \
> -    .balign     4;               \
> -    .word       (insn) - .;      \
> -    .word       (fixup) - .;     \
> -    .popsection
> +#define ASM_EXTABLE_RAW(insn, fixup, type, data)    \
> +    .pushsection .ex_table, "a";                    \
> +    .balign     4;                                  \
> +    .long       ((insn) - .);                       \
> +    .long       ((fixup) - .);                      \

Why the change from .word to .long? And why the extra pairs of parens?

> +    .short      (type);                             \
> +    .short      (data);                             \

Alongside .word, these then likely want to be .half.

> @@ -23,20 +30,36 @@
>  
>  struct cpu_user_regs;
>  
> -#define ASM_EXTABLE(insn, fixup)      \
> -    ".pushsection .ex_table, \"a\"\n" \
> -    ".balign    4\n"                  \
> -    ".word      (" #insn " - .)\n"    \
> -    ".word      (" #fixup " - .)\n"   \
> +#define ASM_EXTABLE_RAW(insn, fixup, type, data)    \
> +    ".pushsection .ex_table, \"a\"\n"               \
> +    ".balign    4\n"                                \
> +    ".long      ((" insn ") - .)\n"                 \
> +    ".long      ((" fixup ") - .)\n"                \

Same questions here then.

> --- /dev/null
> +++ b/xen/arch/riscv/include/asm/gpr-num.h
> @@ -0,0 +1,33 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef RISCV_GPR_NUM_H
> +#define RISCV_GPR_NUM_H
> +
> +/* GPR ABI names, in register-number order (x0 .. x31). */
> +#define GPR_ABI_NAMES                   \
> +    zero, ra, sp, gp, tp, t0, t1, t2,   \
> +    s0, s1, a0, a1, a2, a3, a4, a5,     \
> +    a6, a7, s2, s3, s4, s5, s6, s7,     \
> +    s8, s9, s10, s11, t3, t4, t5, t6
> +
> +#ifdef __ASSEMBLER__
> +
> +    .equ    .L_gpr_num, 0
> +    .irp    name, GPR_ABI_NAMES
> +    .equ    .L_gpr_num_\name, .L_gpr_num
> +    .equ    .L_gpr_num, .L_gpr_num + 1
> +    .endr

So this is emitted no matter whether a .S file actually uses any of the constants.
Perhaps okayish, but somewhat wasteful.

Jan

Re: [PATCH v1 12/17] xen/riscv: extend exception tables with type and data fields

[Oleksii Kurochko]

On 8/12/26 4:37 PM, Jan Beulich wrote:
> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>> @@ -60,6 +68,40 @@ static void ex_handler_fixup(const struct exception_table_entry *ex,
>>       regs->sepc = ex_fixup(ex);
>>   }
>>   
>> +static inline unsigned long regs_get_gpr(struct cpu_user_regs *regs,
>> +                                         unsigned int offset)
>> +{
>> +    /*
>> +     * The GPR number -> offset arithmetic below relies on x0..x31 being
>> +     * laid out at the start of struct cpu_user_regs in architectural
>> +     * order.
>> +     */
>> +    BUILD_BUG_ON(offsetof(struct cpu_user_regs, ra) !=
>> +                 sizeof(unsigned long));
>> +    BUILD_BUG_ON(offsetof(struct cpu_user_regs, t6) !=
>> +                 31 * sizeof(unsigned long));
>> +
>> +    if ( unlikely(!offset || (offset > MAX_REG_OFFSET)) )
>> +        return 0;
> 
> And an offset not divisible by sizeof(unsigned long) is okay?

No, it isn't okay. I will apply your comment ...

> 
> Returning 0 as error indicator also feels fragile.

With what I suggested below returning could be just dropped.

> 
>> +    return *(unsigned long *)((unsigned long)regs + offset);
>> +}
>> +
>> +static void ex_handler_trap_info(const struct exception_table_entry *ex,
>> +                                 struct cpu_user_regs *regs)
>> +{
>> +    struct trap_info *trap_info =
>> +        (struct trap_info *)regs_get_gpr(regs, ex->data * sizeof(unsigned long));
> 
> Related to the earlier comment: Simply pass just ex->data here, leaving the
> multiplication to regs_get_gpr()?

... It would be better to move the multiplication inside regs_get_gpr().

Your comment made me think about whether the multiplication is needed at 
all (regardless of where it is done). In other words, ex->data contains 
the register number, so we could just write:

static unsigned long regs_get_gpr(const struct cpu_user_regs *regs,
                                   unsigned int num)
{
     /*
      * The GPR number -> offset arithmetic below relies on x0..x31 being
      * laid out at the start of struct cpu_user_regs in architectural 
order.
      */
     BUILD_BUG_ON(offsetof(struct cpu_user_regs, ra) != sizeof(unsigned 
long));
     BUILD_BUG_ON(offsetof(struct cpu_user_regs, t6) != 31 * 
sizeof(unsigned long));

     ASSERT(num && (num < 32));

     return ((const unsigned long *)regs)[num];
}

Probably, we want to consider this function out of context (for now 
context is that we use it to recieve a pointer to trap_info which can't 
be obviously stored in x0 as it should be always hardwired zero). In 
that case, there is no need to check that num is 0.

So, it probably makes sense to just have:
   ASSERT(num < 32);

ASSERT() is fine here as I don't think that compiler will use incorrect 
number during register allocation.
> 
>> +    BUG_ON(!trap_info);
>> +
>> +    trap_info->sepc = csr_read(CSR_SEPC);
>> +    trap_info->scause = csr_read(CSR_SCAUSE);
>> +    trap_info->stval = csr_read(CSR_STVAL);
> 
> Do you really need to re-read all three registers here? Didn't you read at least
> scause already, in order to make it here in the first place?

Agree, ->scause and ->sepc are already read.


> 
>> --- a/xen/arch/riscv/include/asm/extable.h
>> +++ b/xen/arch/riscv/include/asm/extable.h
>> @@ -3,17 +3,24 @@
>>   #ifndef ASM__RISCV__ASM_EXTABLE_H
>>   #define ASM__RISCV__ASM_EXTABLE_H
>>   
>> +#include <asm/gpr-num.h>
>> +
>> +#define EX_TYPE_FIXUP 0
>> +#define EX_TYPE_TRAP_INFO 1
>> +
>>   #ifdef __ASSEMBLER__
>>   
>> -#define ASM_EXTABLE(insn, fixup) \
>> -    .pushsection .ex_table, "a"; \
>> -    .balign     4;               \
>> -    .word       (insn) - .;      \
>> -    .word       (fixup) - .;     \
>> -    .popsection
>> +#define ASM_EXTABLE_RAW(insn, fixup, type, data)    \
>> +    .pushsection .ex_table, "a";                    \
>> +    .balign     4;                                  \
>> +    .long       ((insn) - .);                       \
>> +    .long       ((fixup) - .);                      \
> 
> Why the change from .word to .long? And why the extra pairs of parens?

I don't see any sense now in changing type and of extra pairs of parens.
This part of changes will be reverted.

> 
>> +    .short      (type);                             \
>> +    .short      (data);                             \
> 
> Alongside .word, these then likely want to be .half.

.half will be better if .word is used.

> 
>> @@ -23,20 +30,36 @@
>>   
>>   struct cpu_user_regs;
>>   
>> -#define ASM_EXTABLE(insn, fixup)      \
>> -    ".pushsection .ex_table, \"a\"\n" \
>> -    ".balign    4\n"                  \
>> -    ".word      (" #insn " - .)\n"    \
>> -    ".word      (" #fixup " - .)\n"   \
>> +#define ASM_EXTABLE_RAW(insn, fixup, type, data)    \
>> +    ".pushsection .ex_table, \"a\"\n"               \
>> +    ".balign    4\n"                                \
>> +    ".long      ((" insn ") - .)\n"                 \
>> +    ".long      ((" fixup ") - .)\n"                \
> 
> Same questions here then.

I will revert these changes too.

> 
>> --- /dev/null
>> +++ b/xen/arch/riscv/include/asm/gpr-num.h
>> @@ -0,0 +1,33 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +#ifndef RISCV_GPR_NUM_H
>> +#define RISCV_GPR_NUM_H
>> +
>> +/* GPR ABI names, in register-number order (x0 .. x31). */
>> +#define GPR_ABI_NAMES                   \
>> +    zero, ra, sp, gp, tp, t0, t1, t2,   \
>> +    s0, s1, a0, a1, a2, a3, a4, a5,     \
>> +    a6, a7, s2, s3, s4, s5, s6, s7,     \
>> +    s8, s9, s10, s11, t3, t4, t5, t6
>> +
>> +#ifdef __ASSEMBLER__
>> +
>> +    .equ    .L_gpr_num, 0
>> +    .irp    name, GPR_ABI_NAMES
>> +    .equ    .L_gpr_num_\name, .L_gpr_num
>> +    .equ    .L_gpr_num, .L_gpr_num + 1
>> +    .endr
> 
> So this is emitted no matter whether a .S file actually uses any of the constants.
> Perhaps okayish, but somewhat wasteful.

I can move #include <asm/gpr-num.h> inside "#else /* __ASSEMBLER__ */" 
in asm/extable.h and it will be enough for now. Or just drop declaration 
of .L_gpr_num for assembler code until it will be needed by it.

~ Oleksii

Re: [PATCH v1 12/17] xen/riscv: extend exception tables with type and data fields

[Oleksii Kurochko]

On 8/17/26 1:33 PM, Oleksii Kurochko wrote:
>>
>>> +    BUG_ON(!trap_info);
>>> +
>>> +    trap_info->sepc = csr_read(CSR_SEPC);
>>> +    trap_info->scause = csr_read(CSR_SCAUSE);
>>> +    trap_info->stval = csr_read(CSR_STVAL);
>>
>> Do you really need to re-read all three registers here? Didn't you 
>> read at least
>> scause already, in order to make it here in the first place?
> 
> Agree, ->scause and ->sepc are already read.

scause should be re-reaad as we don't save it inside cpu_user_regs 
structure.

~ Oleksii

Re: [PATCH v1 12/17] xen/riscv: extend exception tables with type and data fields

[Jan Beulich]

On 17.08.2026 13:39, Oleksii Kurochko wrote:
> On 8/17/26 1:33 PM, Oleksii Kurochko wrote:
>>>
>>>> +    BUG_ON(!trap_info);
>>>> +
>>>> +    trap_info->sepc = csr_read(CSR_SEPC);
>>>> +    trap_info->scause = csr_read(CSR_SCAUSE);
>>>> +    trap_info->stval = csr_read(CSR_STVAL);
>>>
>>> Do you really need to re-read all three registers here? Didn't you 
>>> read at least
>>> scause already, in order to make it here in the first place?
>>
>> Agree, ->scause and ->sepc are already read.
> 
> scause should be re-reaad as we don't save it inside cpu_user_regs 
> structure.

It could be propagated as a function argument. Question really is how
expensive these CSR reads are.

Jan

Re: [PATCH v1 12/17] xen/riscv: extend exception tables with type and data fields

[Oleksii Kurochko]

On 8/18/26 9:58 AM, Jan Beulich wrote:
> On 17.08.2026 13:39, Oleksii Kurochko wrote:
>> On 8/17/26 1:33 PM, Oleksii Kurochko wrote:
>>>>
>>>>> +    BUG_ON(!trap_info);
>>>>> +
>>>>> +    trap_info->sepc = csr_read(CSR_SEPC);
>>>>> +    trap_info->scause = csr_read(CSR_SCAUSE);
>>>>> +    trap_info->stval = csr_read(CSR_STVAL);
>>>>
>>>> Do you really need to re-read all three registers here? Didn't you
>>>> read at least
>>>> scause already, in order to make it here in the first place?
>>>
>>> Agree, ->scause and ->sepc are already read.
>>
>> scause should be re-reaad as we don't save it inside cpu_user_regs
>> structure.
> 
> It could be propagated as a function argument. Question really is how
> expensive these CSR reads are.

Considering that each RISC-V hart normally observes its own CSR 
accesses, including its implicit CSR accesses, as performed in program 
order what affects out of order execution maybe it will be really better 
to propagate scause as a function argument.

~ Oleksii

Re: [PATCH v1 12/17] xen/riscv: extend exception tables with type and data fields

[Jan Beulich]

On 17.08.2026 13:33, Oleksii Kurochko wrote:
> On 8/12/26 4:37 PM, Jan Beulich wrote:
>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>> @@ -60,6 +68,40 @@ static void ex_handler_fixup(const struct exception_table_entry *ex,
>>>       regs->sepc = ex_fixup(ex);
>>>   }
>>>   
>>> +static inline unsigned long regs_get_gpr(struct cpu_user_regs *regs,
>>> +                                         unsigned int offset)
>>> +{
>>> +    /*
>>> +     * The GPR number -> offset arithmetic below relies on x0..x31 being
>>> +     * laid out at the start of struct cpu_user_regs in architectural
>>> +     * order.
>>> +     */
>>> +    BUILD_BUG_ON(offsetof(struct cpu_user_regs, ra) !=
>>> +                 sizeof(unsigned long));
>>> +    BUILD_BUG_ON(offsetof(struct cpu_user_regs, t6) !=
>>> +                 31 * sizeof(unsigned long));
>>> +
>>> +    if ( unlikely(!offset || (offset > MAX_REG_OFFSET)) )
>>> +        return 0;
>>
>> And an offset not divisible by sizeof(unsigned long) is okay?
> 
> No, it isn't okay. I will apply your comment ...
> 
>>
>> Returning 0 as error indicator also feels fragile.
> 
> With what I suggested below returning could be just dropped.
> 
>>
>>> +    return *(unsigned long *)((unsigned long)regs + offset);
>>> +}
>>> +
>>> +static void ex_handler_trap_info(const struct exception_table_entry *ex,
>>> +                                 struct cpu_user_regs *regs)
>>> +{
>>> +    struct trap_info *trap_info =
>>> +        (struct trap_info *)regs_get_gpr(regs, ex->data * sizeof(unsigned long));
>>
>> Related to the earlier comment: Simply pass just ex->data here, leaving the
>> multiplication to regs_get_gpr()?
> 
> ... It would be better to move the multiplication inside regs_get_gpr().
> 
> Your comment made me think about whether the multiplication is needed at 
> all (regardless of where it is done). In other words, ex->data contains 
> the register number, so we could just write:
> 
> static unsigned long regs_get_gpr(const struct cpu_user_regs *regs,
>                                    unsigned int num)
> {
>      /*
>       * The GPR number -> offset arithmetic below relies on x0..x31 being
>       * laid out at the start of struct cpu_user_regs in architectural 
> order.
>       */
>      BUILD_BUG_ON(offsetof(struct cpu_user_regs, ra) != sizeof(unsigned 
> long));
>      BUILD_BUG_ON(offsetof(struct cpu_user_regs, t6) != 31 * 
> sizeof(unsigned long));
> 
>      ASSERT(num && (num < 32));
> 
>      return ((const unsigned long *)regs)[num];
> }
> 
> Probably, we want to consider this function out of context (for now 
> context is that we use it to recieve a pointer to trap_info which can't 
> be obviously stored in x0 as it should be always hardwired zero). In 
> that case, there is no need to check that num is 0.
> 
> So, it probably makes sense to just have:
>    ASSERT(num < 32);
> 
> ASSERT() is fine here as I don't think that compiler will use incorrect 
> number during register allocation.

I agree.

However, the x0 aspect is still odd. Why again is it that struct cpu_user_regs
has a field for it, when the register value is always 0? (And tangentially,
what's the pregs field there, and what is stack_cpu_regs?)

>>> --- /dev/null
>>> +++ b/xen/arch/riscv/include/asm/gpr-num.h
>>> @@ -0,0 +1,33 @@
>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>> +#ifndef RISCV_GPR_NUM_H
>>> +#define RISCV_GPR_NUM_H
>>> +
>>> +/* GPR ABI names, in register-number order (x0 .. x31). */
>>> +#define GPR_ABI_NAMES                   \
>>> +    zero, ra, sp, gp, tp, t0, t1, t2,   \
>>> +    s0, s1, a0, a1, a2, a3, a4, a5,     \
>>> +    a6, a7, s2, s3, s4, s5, s6, s7,     \
>>> +    s8, s9, s10, s11, t3, t4, t5, t6

Having looked at struct cpu_user_regs for the response above: How is this
macro intended to be kept in sync with struct cpu_user_regs? Yes, the ABI
isn't going to change, but (a) still and (b) if later another ABI was
introduced, names here and fields there could still easily diverge.

>>> +#ifdef __ASSEMBLER__
>>> +
>>> +    .equ    .L_gpr_num, 0
>>> +    .irp    name, GPR_ABI_NAMES
>>> +    .equ    .L_gpr_num_\name, .L_gpr_num
>>> +    .equ    .L_gpr_num, .L_gpr_num + 1
>>> +    .endr
>>
>> So this is emitted no matter whether a .S file actually uses any of the constants.
>> Perhaps okayish, but somewhat wasteful.
> 
> I can move #include <asm/gpr-num.h> inside "#else /* __ASSEMBLER__ */" 
> in asm/extable.h and it will be enough for now. Or just drop declaration 
> of .L_gpr_num for assembler code until it will be needed by it.

How would either of these address the remark I made? Not every .S file
including asm/extable.h will need these constants. Imo this new file wants
strictly only including by files which actually need .L_gpr_num_*.

Jan

Re: [PATCH v1 12/17] xen/riscv: extend exception tables with type and data fields

[Oleksii Kurochko]

On 8/18/26 9:56 AM, Jan Beulich wrote:
> On 17.08.2026 13:33, Oleksii Kurochko wrote:
>> On 8/12/26 4:37 PM, Jan Beulich wrote:
>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>> @@ -60,6 +68,40 @@ static void ex_handler_fixup(const struct exception_table_entry *ex,
>>>>        regs->sepc = ex_fixup(ex);
>>>>    }
>>>>    
>>>> +static inline unsigned long regs_get_gpr(struct cpu_user_regs *regs,
>>>> +                                         unsigned int offset)
>>>> +{
>>>> +    /*
>>>> +     * The GPR number -> offset arithmetic below relies on x0..x31 being
>>>> +     * laid out at the start of struct cpu_user_regs in architectural
>>>> +     * order.
>>>> +     */
>>>> +    BUILD_BUG_ON(offsetof(struct cpu_user_regs, ra) !=
>>>> +                 sizeof(unsigned long));
>>>> +    BUILD_BUG_ON(offsetof(struct cpu_user_regs, t6) !=
>>>> +                 31 * sizeof(unsigned long));
>>>> +
>>>> +    if ( unlikely(!offset || (offset > MAX_REG_OFFSET)) )
>>>> +        return 0;
>>>
>>> And an offset not divisible by sizeof(unsigned long) is okay?
>>
>> No, it isn't okay. I will apply your comment ...
>>
>>>
>>> Returning 0 as error indicator also feels fragile.
>>
>> With what I suggested below returning could be just dropped.
>>
>>>
>>>> +    return *(unsigned long *)((unsigned long)regs + offset);
>>>> +}
>>>> +
>>>> +static void ex_handler_trap_info(const struct exception_table_entry *ex,
>>>> +                                 struct cpu_user_regs *regs)
>>>> +{
>>>> +    struct trap_info *trap_info =
>>>> +        (struct trap_info *)regs_get_gpr(regs, ex->data * sizeof(unsigned long));
>>>
>>> Related to the earlier comment: Simply pass just ex->data here, leaving the
>>> multiplication to regs_get_gpr()?
>>
>> ... It would be better to move the multiplication inside regs_get_gpr().
>>
>> Your comment made me think about whether the multiplication is needed at
>> all (regardless of where it is done). In other words, ex->data contains
>> the register number, so we could just write:
>>
>> static unsigned long regs_get_gpr(const struct cpu_user_regs *regs,
>>                                     unsigned int num)
>> {
>>       /*
>>        * The GPR number -> offset arithmetic below relies on x0..x31 being
>>        * laid out at the start of struct cpu_user_regs in architectural
>> order.
>>        */
>>       BUILD_BUG_ON(offsetof(struct cpu_user_regs, ra) != sizeof(unsigned
>> long));
>>       BUILD_BUG_ON(offsetof(struct cpu_user_regs, t6) != 31 *
>> sizeof(unsigned long));
>>
>>       ASSERT(num && (num < 32));
>>
>>       return ((const unsigned long *)regs)[num];
>> }
>>
>> Probably, we want to consider this function out of context (for now
>> context is that we use it to recieve a pointer to trap_info which can't
>> be obviously stored in x0 as it should be always hardwired zero). In
>> that case, there is no need to check that num is 0.
>>
>> So, it probably makes sense to just have:
>>     ASSERT(num < 32);
>>
>> ASSERT() is fine here as I don't think that compiler will use incorrect
>> number during register allocation.
> 
> I agree.
> 
> However, the x0 aspect is still odd. Why again is it that struct cpu_user_regs
> has a field for it, when the register value is always 0? 

zero field isn't there to hold a value, it's there so the first 32 slots 
form an x0..x31 array indexed by GPR number. It is useful for SET_RD() 
implementation, for example.

> (And tangentially,
> what's the pregs field there, and what is stack_cpu_regs?)

It is rudiment, I don't use it anymore.

I will drop it in separate patch.

> 
>>>> --- /dev/null
>>>> +++ b/xen/arch/riscv/include/asm/gpr-num.h
>>>> @@ -0,0 +1,33 @@
>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
>>>> +#ifndef RISCV_GPR_NUM_H
>>>> +#define RISCV_GPR_NUM_H
>>>> +
>>>> +/* GPR ABI names, in register-number order (x0 .. x31). */
>>>> +#define GPR_ABI_NAMES                   \
>>>> +    zero, ra, sp, gp, tp, t0, t1, t2,   \
>>>> +    s0, s1, a0, a1, a2, a3, a4, a5,     \
>>>> +    a6, a7, s2, s3, s4, s5, s6, s7,     \
>>>> +    s8, s9, s10, s11, t3, t4, t5, t6
> 
> Having looked at struct cpu_user_regs for the response above: How is this
> macro intended to be kept in sync with struct cpu_user_regs? Yes, the ABI
> isn't going to change, but (a) still and (b) if later another ABI was
> introduced, names here and fields there could still easily diverge.

Yes, this macro should be in sync with struct cpu_user_regs too.

Then it is needed to turn GPR_ABI_NAMES into a numbered X-macro list and 
generate everything from it:


/* asm/gpr-num.h */
/*
  * GPRs in register-number order (x0 .. x31), by ABI name.  Single 
source of
  * truth: generates the .L_gpr_num_* assembler symbols, and is 
cross-checked
  * against struct cpu_user_regs at build time (see regs_get_gpr()).
  */
#define GPR_LIST(x)                                 \
     x(0,  zero) x(1,  ra)  x(2,  sp)  x(3,  gp)     \
     x(4,  tp)   x(5,  t0)  x(6,  t1)  x(7,  t2)     \
     x(8,  s0)   x(9,  s1)  x(10, a0)  x(11, a1)     \
     x(12, a2)   x(13, a3)  x(14, a4)  x(15, a5)     \
     x(16, a6)   x(17, a7)  x(18, s2)  x(19, s3)     \
     x(20, s4)   x(21, s5)  x(22, s6)  x(23, s7)     \
     x(24, s8)   x(25, s9)  x(26, s10) x(27, s11)    \
     x(28, t3)   x(29, t4)  x(30, t5)  x(31, t6)

#ifdef __ASSEMBLER__

#define GPR_NUM_EQU(num, name)  .equ .L_gpr_num_##name, num;
GPR_LIST(GPR_NUM_EQU)

#else /* __ASSEMBLER__ */

#define GPR_NUM_EQU(num, name)  "   .equ .L_gpr_num_" #name ", " #num "\n"
#define DEFINE_ASM_GPR_NUMS     GPR_LIST(GPR_NUM_EQU)

#endif

and then also:


/* extable.c, replacing the two existing BUILD_BUG_ONs */
#define CHECK_GPR_OFFSET(num, name)                     \
     BUILD_BUG_ON(offsetof(struct cpu_user_regs, name)   \
                  != (num) * sizeof(unsigned long));

static inline unsigned long regs_get_gpr(struct cpu_user_regs *regs,
                                          unsigned int offset)
{
     /* GPR number N must be field N of struct cpu_user_regs. */
     GPR_LIST(CHECK_GPR_OFFSET)
     ...
}

> 
>>>> +#ifdef __ASSEMBLER__
>>>> +
>>>> +    .equ    .L_gpr_num, 0
>>>> +    .irp    name, GPR_ABI_NAMES
>>>> +    .equ    .L_gpr_num_\name, .L_gpr_num
>>>> +    .equ    .L_gpr_num, .L_gpr_num + 1
>>>> +    .endr
>>>
>>> So this is emitted no matter whether a .S file actually uses any of the constants.
>>> Perhaps okayish, but somewhat wasteful.
>>
>> I can move #include <asm/gpr-num.h> inside "#else /* __ASSEMBLER__ */"
>> in asm/extable.h and it will be enough for now. Or just drop declaration
>> of .L_gpr_num for assembler code until it will be needed by it.
> 
> How would either of these address the remark I made? Not every .S file
> including asm/extable.h will need these constants. Imo this new file wants
> strictly only including by files which actually need .L_gpr_num_*.

Oh, now I got your idea. There is no need to icnlude asm/gpr-num.h 
inside <asm/extable.h>. It seems to me then it will be better to follow 
the way which was intrdouced originally just have asm/gpr-num.h included 
at the top of asm/extable.h, this is not a big price for .S file which 
including asm/extrable.h and doesn't really need asm/gpr-num.h.

~ Oleksii

Re: [PATCH v1 12/17] xen/riscv: extend exception tables with type and data fields

[Jan Beulich]

On 18.08.2026 11:14, Oleksii Kurochko wrote:
> On 8/18/26 9:56 AM, Jan Beulich wrote:
>> On 17.08.2026 13:33, Oleksii Kurochko wrote:
>>> On 8/12/26 4:37 PM, Jan Beulich wrote:
>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>> @@ -60,6 +68,40 @@ static void ex_handler_fixup(const struct exception_table_entry *ex,
>>>>>        regs->sepc = ex_fixup(ex);
>>>>>    }
>>>>>    
>>>>> +static inline unsigned long regs_get_gpr(struct cpu_user_regs *regs,
>>>>> +                                         unsigned int offset)
>>>>> +{
>>>>> +    /*
>>>>> +     * The GPR number -> offset arithmetic below relies on x0..x31 being
>>>>> +     * laid out at the start of struct cpu_user_regs in architectural
>>>>> +     * order.
>>>>> +     */
>>>>> +    BUILD_BUG_ON(offsetof(struct cpu_user_regs, ra) !=
>>>>> +                 sizeof(unsigned long));
>>>>> +    BUILD_BUG_ON(offsetof(struct cpu_user_regs, t6) !=
>>>>> +                 31 * sizeof(unsigned long));
>>>>> +
>>>>> +    if ( unlikely(!offset || (offset > MAX_REG_OFFSET)) )
>>>>> +        return 0;
>>>>
>>>> And an offset not divisible by sizeof(unsigned long) is okay?
>>>
>>> No, it isn't okay. I will apply your comment ...
>>>
>>>>
>>>> Returning 0 as error indicator also feels fragile.
>>>
>>> With what I suggested below returning could be just dropped.
>>>
>>>>
>>>>> +    return *(unsigned long *)((unsigned long)regs + offset);
>>>>> +}
>>>>> +
>>>>> +static void ex_handler_trap_info(const struct exception_table_entry *ex,
>>>>> +                                 struct cpu_user_regs *regs)
>>>>> +{
>>>>> +    struct trap_info *trap_info =
>>>>> +        (struct trap_info *)regs_get_gpr(regs, ex->data * sizeof(unsigned long));
>>>>
>>>> Related to the earlier comment: Simply pass just ex->data here, leaving the
>>>> multiplication to regs_get_gpr()?
>>>
>>> ... It would be better to move the multiplication inside regs_get_gpr().
>>>
>>> Your comment made me think about whether the multiplication is needed at
>>> all (regardless of where it is done). In other words, ex->data contains
>>> the register number, so we could just write:
>>>
>>> static unsigned long regs_get_gpr(const struct cpu_user_regs *regs,
>>>                                     unsigned int num)
>>> {
>>>       /*
>>>        * The GPR number -> offset arithmetic below relies on x0..x31 being
>>>        * laid out at the start of struct cpu_user_regs in architectural
>>> order.
>>>        */
>>>       BUILD_BUG_ON(offsetof(struct cpu_user_regs, ra) != sizeof(unsigned
>>> long));
>>>       BUILD_BUG_ON(offsetof(struct cpu_user_regs, t6) != 31 *
>>> sizeof(unsigned long));
>>>
>>>       ASSERT(num && (num < 32));
>>>
>>>       return ((const unsigned long *)regs)[num];
>>> }
>>>
>>> Probably, we want to consider this function out of context (for now
>>> context is that we use it to recieve a pointer to trap_info which can't
>>> be obviously stored in x0 as it should be always hardwired zero). In
>>> that case, there is no need to check that num is 0.
>>>
>>> So, it probably makes sense to just have:
>>>     ASSERT(num < 32);
>>>
>>> ASSERT() is fine here as I don't think that compiler will use incorrect
>>> number during register allocation.
>>
>> I agree.
>>
>> However, the x0 aspect is still odd. Why again is it that struct cpu_user_regs
>> has a field for it, when the register value is always 0? 
> 
> zero field isn't there to hold a value, it's there so the first 32 slots 
> form an x0..x31 array indexed by GPR number. It is useful for SET_RD() 
> implementation, for example.

But SET_RD() will need to avoid touching .zero anyway. Why waste the space,
when something useful can be put there?

Jan

Re: [PATCH v1 12/17] xen/riscv: extend exception tables with type and data fields

[Oleksii Kurochko]

On 8/18/26 11:26 AM, Jan Beulich wrote:
> On 18.08.2026 11:14, Oleksii Kurochko wrote:
>> On 8/18/26 9:56 AM, Jan Beulich wrote:
>>> On 17.08.2026 13:33, Oleksii Kurochko wrote:
>>>> On 8/12/26 4:37 PM, Jan Beulich wrote:
>>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>>> @@ -60,6 +68,40 @@ static void ex_handler_fixup(const struct exception_table_entry *ex,
>>>>>>         regs->sepc = ex_fixup(ex);
>>>>>>     }
>>>>>>     
>>>>>> +static inline unsigned long regs_get_gpr(struct cpu_user_regs *regs,
>>>>>> +                                         unsigned int offset)
>>>>>> +{
>>>>>> +    /*
>>>>>> +     * The GPR number -> offset arithmetic below relies on x0..x31 being
>>>>>> +     * laid out at the start of struct cpu_user_regs in architectural
>>>>>> +     * order.
>>>>>> +     */
>>>>>> +    BUILD_BUG_ON(offsetof(struct cpu_user_regs, ra) !=
>>>>>> +                 sizeof(unsigned long));
>>>>>> +    BUILD_BUG_ON(offsetof(struct cpu_user_regs, t6) !=
>>>>>> +                 31 * sizeof(unsigned long));
>>>>>> +
>>>>>> +    if ( unlikely(!offset || (offset > MAX_REG_OFFSET)) )
>>>>>> +        return 0;
>>>>>
>>>>> And an offset not divisible by sizeof(unsigned long) is okay?
>>>>
>>>> No, it isn't okay. I will apply your comment ...
>>>>
>>>>>
>>>>> Returning 0 as error indicator also feels fragile.
>>>>
>>>> With what I suggested below returning could be just dropped.
>>>>
>>>>>
>>>>>> +    return *(unsigned long *)((unsigned long)regs + offset);
>>>>>> +}
>>>>>> +
>>>>>> +static void ex_handler_trap_info(const struct exception_table_entry *ex,
>>>>>> +                                 struct cpu_user_regs *regs)
>>>>>> +{
>>>>>> +    struct trap_info *trap_info =
>>>>>> +        (struct trap_info *)regs_get_gpr(regs, ex->data * sizeof(unsigned long));
>>>>>
>>>>> Related to the earlier comment: Simply pass just ex->data here, leaving the
>>>>> multiplication to regs_get_gpr()?
>>>>
>>>> ... It would be better to move the multiplication inside regs_get_gpr().
>>>>
>>>> Your comment made me think about whether the multiplication is needed at
>>>> all (regardless of where it is done). In other words, ex->data contains
>>>> the register number, so we could just write:
>>>>
>>>> static unsigned long regs_get_gpr(const struct cpu_user_regs *regs,
>>>>                                      unsigned int num)
>>>> {
>>>>        /*
>>>>         * The GPR number -> offset arithmetic below relies on x0..x31 being
>>>>         * laid out at the start of struct cpu_user_regs in architectural
>>>> order.
>>>>         */
>>>>        BUILD_BUG_ON(offsetof(struct cpu_user_regs, ra) != sizeof(unsigned
>>>> long));
>>>>        BUILD_BUG_ON(offsetof(struct cpu_user_regs, t6) != 31 *
>>>> sizeof(unsigned long));
>>>>
>>>>        ASSERT(num && (num < 32));
>>>>
>>>>        return ((const unsigned long *)regs)[num];
>>>> }
>>>>
>>>> Probably, we want to consider this function out of context (for now
>>>> context is that we use it to recieve a pointer to trap_info which can't
>>>> be obviously stored in x0 as it should be always hardwired zero). In
>>>> that case, there is no need to check that num is 0.
>>>>
>>>> So, it probably makes sense to just have:
>>>>      ASSERT(num < 32);
>>>>
>>>> ASSERT() is fine here as I don't think that compiler will use incorrect
>>>> number during register allocation.
>>>
>>> I agree.
>>>
>>> However, the x0 aspect is still odd. Why again is it that struct cpu_user_regs
>>> has a field for it, when the register value is always 0?
>>
>> zero field isn't there to hold a value, it's there so the first 32 slots
>> form an x0..x31 array indexed by GPR number. It is useful for SET_RD()
>> implementation, for example.
> 
> But SET_RD() will need to avoid touching .zero anyway. Why waste the space,
> when something useful can be put there?

For SET_RD() agree, it doesn't make sense. Bad example. But it could be 
somehow a "protection" to not clobber something useful if SET_RD 
arguments won't handled correctly.

SET_RD() is only half of it. The read accessors matter more: sd x0, 
0(a0) to an MMIO address is the normal way a guest writes 0 to a device 
register, and emulate_store() fetches the operand with GET_RS2(), which 
is a plain index by register number. If something useful lived at offset 
0, that store would hand the device that value instead of zero. So 
whatever we put there would have to survive being read as a source 
operand and being clobbered by SET_RD(), which means nothing can go there.

~ Oleksii

Re: [PATCH v1 12/17] xen/riscv: extend exception tables with type and data fields

[Jan Beulich]

On 18.08.2026 11:40, Oleksii Kurochko wrote:
> On 8/18/26 11:26 AM, Jan Beulich wrote:
>> On 18.08.2026 11:14, Oleksii Kurochko wrote:
>>> On 8/18/26 9:56 AM, Jan Beulich wrote:
>>>> On 17.08.2026 13:33, Oleksii Kurochko wrote:
>>>>> On 8/12/26 4:37 PM, Jan Beulich wrote:
>>>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>>>> @@ -60,6 +68,40 @@ static void ex_handler_fixup(const struct exception_table_entry *ex,
>>>>>>>         regs->sepc = ex_fixup(ex);
>>>>>>>     }
>>>>>>>     
>>>>>>> +static inline unsigned long regs_get_gpr(struct cpu_user_regs *regs,
>>>>>>> +                                         unsigned int offset)
>>>>>>> +{
>>>>>>> +    /*
>>>>>>> +     * The GPR number -> offset arithmetic below relies on x0..x31 being
>>>>>>> +     * laid out at the start of struct cpu_user_regs in architectural
>>>>>>> +     * order.
>>>>>>> +     */
>>>>>>> +    BUILD_BUG_ON(offsetof(struct cpu_user_regs, ra) !=
>>>>>>> +                 sizeof(unsigned long));
>>>>>>> +    BUILD_BUG_ON(offsetof(struct cpu_user_regs, t6) !=
>>>>>>> +                 31 * sizeof(unsigned long));
>>>>>>> +
>>>>>>> +    if ( unlikely(!offset || (offset > MAX_REG_OFFSET)) )
>>>>>>> +        return 0;
>>>>>>
>>>>>> And an offset not divisible by sizeof(unsigned long) is okay?
>>>>>
>>>>> No, it isn't okay. I will apply your comment ...
>>>>>
>>>>>>
>>>>>> Returning 0 as error indicator also feels fragile.
>>>>>
>>>>> With what I suggested below returning could be just dropped.
>>>>>
>>>>>>
>>>>>>> +    return *(unsigned long *)((unsigned long)regs + offset);
>>>>>>> +}
>>>>>>> +
>>>>>>> +static void ex_handler_trap_info(const struct exception_table_entry *ex,
>>>>>>> +                                 struct cpu_user_regs *regs)
>>>>>>> +{
>>>>>>> +    struct trap_info *trap_info =
>>>>>>> +        (struct trap_info *)regs_get_gpr(regs, ex->data * sizeof(unsigned long));
>>>>>>
>>>>>> Related to the earlier comment: Simply pass just ex->data here, leaving the
>>>>>> multiplication to regs_get_gpr()?
>>>>>
>>>>> ... It would be better to move the multiplication inside regs_get_gpr().
>>>>>
>>>>> Your comment made me think about whether the multiplication is needed at
>>>>> all (regardless of where it is done). In other words, ex->data contains
>>>>> the register number, so we could just write:
>>>>>
>>>>> static unsigned long regs_get_gpr(const struct cpu_user_regs *regs,
>>>>>                                      unsigned int num)
>>>>> {
>>>>>        /*
>>>>>         * The GPR number -> offset arithmetic below relies on x0..x31 being
>>>>>         * laid out at the start of struct cpu_user_regs in architectural
>>>>> order.
>>>>>         */
>>>>>        BUILD_BUG_ON(offsetof(struct cpu_user_regs, ra) != sizeof(unsigned
>>>>> long));
>>>>>        BUILD_BUG_ON(offsetof(struct cpu_user_regs, t6) != 31 *
>>>>> sizeof(unsigned long));
>>>>>
>>>>>        ASSERT(num && (num < 32));
>>>>>
>>>>>        return ((const unsigned long *)regs)[num];
>>>>> }
>>>>>
>>>>> Probably, we want to consider this function out of context (for now
>>>>> context is that we use it to recieve a pointer to trap_info which can't
>>>>> be obviously stored in x0 as it should be always hardwired zero). In
>>>>> that case, there is no need to check that num is 0.
>>>>>
>>>>> So, it probably makes sense to just have:
>>>>>      ASSERT(num < 32);
>>>>>
>>>>> ASSERT() is fine here as I don't think that compiler will use incorrect
>>>>> number during register allocation.
>>>>
>>>> I agree.
>>>>
>>>> However, the x0 aspect is still odd. Why again is it that struct cpu_user_regs
>>>> has a field for it, when the register value is always 0?
>>>
>>> zero field isn't there to hold a value, it's there so the first 32 slots
>>> form an x0..x31 array indexed by GPR number. It is useful for SET_RD()
>>> implementation, for example.
>>
>> But SET_RD() will need to avoid touching .zero anyway. Why waste the space,
>> when something useful can be put there?
> 
> For SET_RD() agree, it doesn't make sense. Bad example. But it could be 
> somehow a "protection" to not clobber something useful if SET_RD 
> arguments won't handled correctly.
> 
> SET_RD() is only half of it. The read accessors matter more: sd x0, 
> 0(a0) to an MMIO address is the normal way a guest writes 0 to a device 
> register, and emulate_store() fetches the operand with GET_RS2(), which 
> is a plain index by register number.

Oh, wow, how fragile. Without a bright comment in struct cpu_user_regs, one
should be permitted to move fields around or insert new ones at arbitrary
positions. That comment would then also help clarify why "zero" is there as
a field.

> If something useful lived at offset 
> 0, that store would hand the device that value instead of zero. So 
> whatever we put there would have to survive being read as a source 
> operand and being clobbered by SET_RD(), which means nothing can go there.

But you can't use it as (reliable zero) source when you allow SET_RD() to
clobber it.

Jan

[PATCH v1 13/17] xen/riscv: add unprivileged guest memory read helper

[Oleksii Kurochko]

Introduce riscv_vcpu_unpriv_read() to allow Xen to safely read guest memory
using HLV/HLVX instructions while reliably capturing trap context.

This is required for instruction fetch emulation and MMIO decoding, where
Xen must inspect guest memory that may not be directly accessible and may
fault.

The implementation is based on kvm_riscv_vcpu_unpriv_read() from Linux,
with one deviation: the hlv/hlvx instructions translate the guest address
through the live vsatp/hgatp CSRs, i.e. through the address space of the
currently running vCPU, so the function can only be called safely for
current. Instead of taking a struct vcpu argument, it always operates on
current directly.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 xen/arch/riscv/guestcopy.c                | 91 +++++++++++++++++++++++
 xen/arch/riscv/include/asm/guest_access.h |  6 ++
 2 files changed, 97 insertions(+)

diff --git a/xen/arch/riscv/guestcopy.c b/xen/arch/riscv/guestcopy.c
index 8a89212e0bea..57844bc442f6 100644
--- a/xen/arch/riscv/guestcopy.c
+++ b/xen/arch/riscv/guestcopy.c
@@ -6,6 +6,7 @@
 #include <xen/string.h>
 
 #include <asm/guest_access.h>
+#include <asm/traps.h>
 
 #define COPY_from_guest     0U
 #define COPY_to_guest       BIT(0, U)
@@ -114,3 +115,93 @@ unsigned long copy_to_guest_phys(struct domain *d, paddr_t gpa, void *buf,
     return copy_guest(buf, gpa, len, GPA_INFO(d),
                       COPY_to_guest | COPY_gpa);
 }
+
+/*
+ * Read machine word from Guest memory
+ *
+ * @read_insn: Flag representing whether we are reading instruction
+ * @guest_addr: Guest address to read
+ * @trap: Output pointer to trap details
+ *
+ * The hlv/hlvx instructions translate guest_addr through the live
+ * vsatp/hgatp CSRs, so the read is only meaningful for the address
+ * space of the currently running vCPU.
+ */
+unsigned long riscv_vcpu_unpriv_read(bool read_insn,
+                                     unsigned long guest_addr,
+                                     struct trap_info *trap)
+{
+    unsigned long val, tmp;
+    unsigned long flags, old_hstatus;
+
+    /*
+     * As hstatus is going to be changed we don't want an interrupt to occur
+     * with guest's hstatus register.
+     */
+    local_irq_save(flags);
+
+    /*
+     * The hypervisor virtual-machine load and store instructions are valid
+     * only in M-mode or HS-mode, or in U-mode when hstatus.HU=1. Each
+     * instruction performs an explicit memory access as though V=1; i.e.,
+     * with the address translation and protection, and the endianness,
+     * that apply to memory accesses in either VS-mode or VU-mode.
+     * Field SPVP of hstatus controls the privilege level of the access.
+     * The explicit memory access is done as though in VU-mode when SPVP=0,
+     * and as though in VS-mode when SPVP=1.
+     *
+     * So it is necessary to restore vCPU's hstatus before execution of
+     * hlv* instruction.
+     */
+    old_hstatus = csr_swap(CSR_HSTATUS,
+                           vcpu_guest_cpu_user_regs(current)->hstatus);
+
+    if ( read_insn )
+    {
+        asm volatile ( "\n"
+            "1:\n"
+            "   hlvx.hu %[val], (%[addr])\n"
+            ASM_EXTABLE_TRAP_INFO(1b, 3f, %[ti])
+            "   andi %[tmp], %[val], 3\n"
+            "   addi %[tmp], %[tmp], -3\n"
+            "   bne %[tmp], zero, 3f\n"
+            "   addi %[addr], %[addr], 2\n"
+            "\n"
+            "2:\n"
+            "   hlvx.hu %[tmp], (%[addr])\n"
+            ASM_EXTABLE_TRAP_INFO(2b, 3f, %[ti])
+            "   sll %[tmp], %[tmp], 16\n"
+            "   add %[val], %[val], %[tmp]\n"
+            "3:\n"
+        : [val] "=&r" (val), [tmp] "=&r" (tmp), [addr] "+&r" (guest_addr)
+        : [ti] "r" (trap) : "memory" );
+
+        /*
+         * Although HLVX instructions' explicit memory accesses require execute
+         * permissions, they still raise the same exceptions as other load
+         * instructions, rather than raising fetch exceptions instead.
+         */
+        if ( trap->scause == CAUSE_LOAD_PAGE_FAULT )
+            trap->scause = CAUSE_FETCH_PAGE_FAULT;
+    }
+    else
+    {
+        asm volatile ( "\n"
+            "1:\n"
+#ifdef CONFIG_RISCV_64
+            "hlv.d %[val], (%[addr])\n"
+#else
+            "hlv.w %[val], (%[addr])\n"
+#endif
+            "2:\n"
+            ASM_EXTABLE_TRAP_INFO(1b, 2b, %[ti])
+        : [val] "=&r" (val)
+        : [addr] "r" (guest_addr), [ti] "r" (trap) : "memory" );
+    }
+
+    csr_write(CSR_HSTATUS, old_hstatus);
+
+    local_irq_restore(flags);
+
+    return val;
+}
diff --git a/xen/arch/riscv/include/asm/guest_access.h b/xen/arch/riscv/include/asm/guest_access.h
index 8d679319ded0..153ec6ce26d1 100644
--- a/xen/arch/riscv/include/asm/guest_access.h
+++ b/xen/arch/riscv/include/asm/guest_access.h
@@ -5,6 +5,8 @@
 #include <xen/types.h>
 
 struct domain;
+struct trap_info;
+struct vcpu;
 
 unsigned long raw_copy_to_guest(void *to, const void *from, unsigned len);
 unsigned long raw_copy_from_guest(void *to, const void *from, unsigned len);
@@ -25,6 +27,10 @@ unsigned long raw_clear_guest(void *to, unsigned int len);
 unsigned long copy_to_guest_phys(struct domain *d, paddr_t gpa, void *buf,
                                  unsigned long len);
 
+unsigned long riscv_vcpu_unpriv_read(bool read_insn,
+                                     unsigned long guest_addr,
+                                     struct trap_info *trap);
+
 #endif /* ASM__RISCV__GUEST_ACCESS_H */
 /*
  * Local variables:
-- 
2.54.0

Re: [PATCH v1 13/17] xen/riscv: add unprivileged guest memory read helper

[Jan Beulich]

On 20.07.2026 18:02, Oleksii Kurochko wrote:
> Introduce riscv_vcpu_unpriv_read() to allow Xen to safely read guest memory
> using HLV/HLVX instructions while reliably capturing trap context.

Both for the title and the function name: How does "unprivileged" matter here?
The same functions would be use for reading Dom0's memory, wouldn't they?

> @@ -114,3 +115,93 @@ unsigned long copy_to_guest_phys(struct domain *d, paddr_t gpa, void *buf,
>      return copy_guest(buf, gpa, len, GPA_INFO(d),
>                        COPY_to_guest | COPY_gpa);
>  }
> +
> +/*
> + * Read machine word from Guest memory
> + *
> + * @read_insn: Flag representing whether we are reading instruction
> + * @guest_addr: Guest address to read
> + * @trap: Output pointer to trap details
> + *
> + * The hlv/hlvx instructions translate guest_addr through the live
> + * vsatp/hgatp CSRs, so the read is only meaningful for the address
> + * space of the currently running vCPU.
> + */
> +unsigned long riscv_vcpu_unpriv_read(bool read_insn,
> +                                     unsigned long guest_addr,

Personally for such a function I'd expect the address to be the main (first)
parameter.

> +                                     struct trap_info *trap)
> +{
> +    unsigned long val, tmp;
> +    unsigned long flags, old_hstatus;
> +
> +    /*
> +     * As hstatus is going to be changed we don't want an interrupt to occur
> +     * with guest's hstatus register.
> +     */

I don't think "guest's hstatus register" is something real. hstatus is
entirely the hypervisor's register, controlling the guest.

> +    local_irq_save(flags);
> +
> +    /*
> +     * The hypervisor virtual-machine load and store instructions are valid
> +     * only in M-mode or HS-mode, or in U-mode when hstatus.HU=1. Each
> +     * instruction performs an explicit memory access as though V=1; i.e.,
> +     * with the address translation and protection, and the endianness,
> +     * that apply to memory accesses in either VS-mode or VU-mode.
> +     * Field SPVP of hstatus controls the privilege level of the access.
> +     * The explicit memory access is done as though in VU-mode when SPVP=0,
> +     * and as though in VS-mode when SPVP=1.
> +     *
> +     * So it is necessary to restore vCPU's hstatus before execution of
> +     * hlv* instruction.
> +     */
> +    old_hstatus = csr_swap(CSR_HSTATUS,
> +                           vcpu_guest_cpu_user_regs(current)->hstatus);

As you're limiting use of the function to the current vCPU, why would hstatus
need fiddling with? The fields of interest aren't being altered between exit
from guest and making it here, are they?

Without that IRQs also wouldn't need turning off (what about NMIs, btw, once
supported on Xen?), which would help real-time use cases (latency here can
otherwise be affected by guests, by wait of forcing exceptions to be raised).

> +    if ( read_insn )
> +    {
> +        asm volatile ( "\n"
> +            "1:\n"
> +            "   hlvx.hu %[val], (%[addr])\n"
> +            ASM_EXTABLE_TRAP_INFO(1b, 3f, %[ti])

Imo labels used for extable entries would better live on the same line as
the insn they mark.

> +            "   andi %[tmp], %[val], 3\n"
> +            "   addi %[tmp], %[tmp], -3\n"
> +            "   bne %[tmp], zero, 3f\n"

Use BNEZ?

> +            "   addi %[addr], %[addr], 2\n"
> +            "\n"
> +            "2:\n"
> +            "   hlvx.hu %[tmp], (%[addr])\n"
> +            ASM_EXTABLE_TRAP_INFO(2b, 3f, %[ti])
> +            "   sll %[tmp], %[tmp], 16\n"
> +            "   add %[val], %[val], %[tmp]\n"

May I suggest OR instead of ADD?

> +            "3:\n"

If this is an insn wider than 32 bits, you won't have fetched all of it.
I think you want to at least add a comment here indicating that e.g. it's
the callers responsibility to deal with that. (How they would do that is
entirely unclear to me, as they can't simply invoke this function again
passing guest_addr + 4.)

> +        : [val] "=&r" (val), [tmp] "=&r" (tmp), [addr] "+&r" (guest_addr)
> +        : [ti] "r" (trap) : "memory" );

You want to tell the compiler that *trap is written. Instead I don't see
why a memory clobber would be needed: You access a different address space,
i.e. nothing the compiler can make any assumptions about.

You also need to take precautions for not returning an uninitialized "val".
I think the variable wants initializing (perhaps to ~0) and "+r" wants
using as constraint. (Afaik & isn't necessary to use together with +.)

> +        /*
> +         * Although HLVX instructions' explicit memory accesses require execute
> +         * permissions, they still raise the same exceptions as other load
> +         * instructions, rather than raising fetch exceptions instead.
> +         */
> +        if ( trap->scause == CAUSE_LOAD_PAGE_FAULT )
> +            trap->scause = CAUSE_FETCH_PAGE_FAULT;
> +    }
> +    else
> +    {
> +        asm volatile ( "\n"
> +            "1:\n"
> +#ifdef CONFIG_RISCV_64
> +            "hlv.d %[val], (%[addr])\n"
> +#else
> +            "hlv.w %[val], (%[addr])\n"
> +#endif

Once again please use enough care that RV128 would at least obviously fail to
build, rather than building something which then doesn't work.

> +            "2:\n"
> +            ASM_EXTABLE_TRAP_INFO(1b, 2b, %[ti])
> +        : [val] "=&r" (val)
> +        : [addr] "r" (guest_addr), [ti] "r" (trap) : "memory" );
> +    }
> +
> +    csr_write(CSR_HSTATUS, old_hstatus);
> +
> +    local_irq_restore(flags);
> +
> +    return val;
> +}
For both reads and fetches - are there no alignment constraints at all on the
incoming guest_addr?

Jan

Re: [PATCH v1 13/17] xen/riscv: add unprivileged guest memory read helper

[Oleksii Kurochko]

On 8/12/26 5:30 PM, Jan Beulich wrote:
> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>> Introduce riscv_vcpu_unpriv_read() to allow Xen to safely read guest memory
>> using HLV/HLVX instructions while reliably capturing trap context.
> 
> Both for the title and the function name: How does "unprivileged" matter here?

Unprivileged because HLV/HLVX reads guest memory as if it were accessed 
from a less-privileged (guest) context, rather than by the hypervisor in 
HS mode.

I think I am okay generally to drop "unpriv..." from the function name.

> The same functions would be use for reading Dom0's memory, wouldn't they?

Yes, I don't see any issue to let this function to read DomO's memory 
too. But Dom0 could be counted as "unprivileged" too as it is executed 
in VS-mode which is less-privileged then HS-mode.

> 
>> @@ -114,3 +115,93 @@ unsigned long copy_to_guest_phys(struct domain *d, paddr_t gpa, void *buf,
>>       return copy_guest(buf, gpa, len, GPA_INFO(d),
>>                         COPY_to_guest | COPY_gpa);
>>   }
>> +
>> +/*
>> + * Read machine word from Guest memory
>> + *
>> + * @read_insn: Flag representing whether we are reading instruction
>> + * @guest_addr: Guest address to read
>> + * @trap: Output pointer to trap details
>> + *
>> + * The hlv/hlvx instructions translate guest_addr through the live
>> + * vsatp/hgatp CSRs, so the read is only meaningful for the address
>> + * space of the currently running vCPU.
>> + */
>> +unsigned long riscv_vcpu_unpriv_read(bool read_insn,
>> +                                     unsigned long guest_addr,
> 
> Personally for such a function I'd expect the address to be the main (first)
> parameter.

Agree, it will be better. I will update prototype of the function.

> 
>> +                                     struct trap_info *trap)
>> +{
>> +    unsigned long val, tmp;
>> +    unsigned long flags, old_hstatus;
>> +
>> +    /*
>> +     * As hstatus is going to be changed we don't want an interrupt to occur
>> +     * with guest's hstatus register.
>> +     */
> 
> I don't think "guest's hstatus register" is something real. hstatus is
> entirely the hypervisor's register, controlling the guest.

Agree, the wording is incorrect what I meant it is that we don't want to 
corrupt hstatus which was saved during guest exit to hypervisor.

I will just put the following comment "As hstatus is going to be changed 
we don't want an interrupt to change it".

> 
>> +    local_irq_save(flags);
>> +
>> +    /*
>> +     * The hypervisor virtual-machine load and store instructions are valid
>> +     * only in M-mode or HS-mode, or in U-mode when hstatus.HU=1. Each
>> +     * instruction performs an explicit memory access as though V=1; i.e.,
>> +     * with the address translation and protection, and the endianness,
>> +     * that apply to memory accesses in either VS-mode or VU-mode.
>> +     * Field SPVP of hstatus controls the privilege level of the access.
>> +     * The explicit memory access is done as though in VU-mode when SPVP=0,
>> +     * and as though in VS-mode when SPVP=1.
>> +     *
>> +     * So it is necessary to restore vCPU's hstatus before execution of
>> +     * hlv* instruction.
>> +     */
>> +    old_hstatus = csr_swap(CSR_HSTATUS,
>> +                           vcpu_guest_cpu_user_regs(current)->hstatus);
> 
> As you're limiting use of the function to the current vCPU, why would hstatus
> need fiddling with? The fields of interest aren't being altered between exit
> from guest and making it here, are they?

You're right, it doesn't. handle_trap() only saves hstatus into the trap 
frame on entry and restores it before sret; nothing in between installs 
a hypervisor-specific value. So the guest's hstatus (in particular SPVP, 
the only field HLV cares about here (HU only matters in U-mode) ) is 
still live when we get here. Traps taken from HS-mode update only SPV 
and GVA, neither of which affects HLV.

> 
> Without that IRQs also wouldn't need turning off (what about NMIs, btw, once
> supported on Xen?), which would help real-time use cases (latency here can
> otherwise be affected by guests, by wait of forcing exceptions to be raised).

Correct, and I'll drop local_irq_save() too. In fact it is already a 
no-op at both call sites: do_trap() runs with interrupts disabled by the 
trap entry itself. And even with the hstatus write in place it would not 
have been needed: any nested trap goes through the same entry path, 
which saves and restores hstatus (an NMI path built the same way would 
be safe for the same reason, rather than relying on interrupts being 
masked).

What I will do instead is document and assert the actual precondition: 
the function may only be called on the trap-handling path of the current 
vCPU, before returning to the guest. That is where hstatus, vsatp and 
hgatp are guaranteed to still be that vCPU's. do_trap() reaches 
check_for_pcpu_work(), and hence any reschedule, only after handling is 
done.

The following check I will add instead csr_swap() and local_irq_save():
   ASSERT(vcpu_guest_cpu_user_regs(current)->hstatus & HSTATUS_SPV);

> 
>> +    if ( read_insn )
>> +    {
>> +        asm volatile ( "\n"
>> +            "1:\n"
>> +            "   hlvx.hu %[val], (%[addr])\n"
>> +            ASM_EXTABLE_TRAP_INFO(1b, 3f, %[ti])
> 
> Imo labels used for extable entries would better live on the same line as
> the insn they mark.
> 
>> +            "   andi %[tmp], %[val], 3\n"
>> +            "   addi %[tmp], %[tmp], -3\n"
>> +            "   bne %[tmp], zero, 3f\n"
> 
> Use BNEZ?
> 
>> +            "   addi %[addr], %[addr], 2\n"
>> +            "\n"
>> +            "2:\n"
>> +            "   hlvx.hu %[tmp], (%[addr])\n"
>> +            ASM_EXTABLE_TRAP_INFO(2b, 3f, %[ti])
>> +            "   sll %[tmp], %[tmp], 16\n"
>> +            "   add %[val], %[val], %[tmp]\n"
> 
> May I suggest OR instead of ADD?
> 
>> +            "3:\n"
> 
> If this is an insn wider than 32 bits, you won't have fetched all of it.
> I think you want to at least add a comment here indicating that e.g. it's
> the callers responsibility to deal with that. 

I will add the following comment above the function:

  * At most two halfwords are fetched when @read_insn is true, i.e. 
encodings
  * wider than 32 bits are not supported. Such an encoding cannot be 
completed
  * by calling this function again at @guest_addr + 4: the length check is
  * applied to the first halfword read, which would then be a 
continuation of
  * the instruction rather than its opcode. It is up to the caller to reject
  * anything that is neither a 16- nor a 32-bit encoding.

> (How they would do that is
> entirely unclear to me, as they can't simply invoke this function again
> passing guest_addr + 4.)

Then it will be needed to update the code of riscv_unpriv_read().

For now we could something like:

        /*
          * Only two halfwords are fetched, so an encoding wider than 32 
bits
          * would have been truncated. Report it as illegal with a zero 
stval:
          * a nonzero one would have to hold the actual faulting 
instruction,
          * whereas zero simply means the value isn't provided.
          */
         if ( !INSN_IS_16BIT(insn) && !INSN_IS_32BIT(insn) )
             return truly_illegal_insn(v, 0);

> 
>> +        : [val] "=&r" (val), [tmp] "=&r" (tmp), [addr] "+&r" (guest_addr)
>> +        : [ti] "r" (trap) : "memory" );
> 
> You want to tell the compiler that *trap is written. Instead I don't see
> why a memory clobber would be needed: You access a different address space,
> i.e. nothing the compiler can make any assumptions about.

memory clobber tells the compiler that the assembly code performs memory 
reads or writes to items other than those listed in the input and output 
operands and so I don't tell here that *trap will be changed.

Why this understanding is wrong?

Alternative, I think, could be:
         : [val] "+r" (val), "+m" (*trap)
         : [addr] "r" (guest_addr), [ti] "r" (trap) );
And then memory clobber could be dropped.



> 
> You also need to take precautions for not returning an uninitialized "val".
> I think the variable wants initializing (perhaps to ~0) and "+r" wants
> using as constraint. (Afaik & isn't necessary to use together with +.)

I agree with '+' if we will initialize val with some value.

Regarding, '&' my understanding is that I have to use it always when

> 
>> +        /*
>> +         * Although HLVX instructions' explicit memory accesses require execute
>> +         * permissions, they still raise the same exceptions as other load
>> +         * instructions, rather than raising fetch exceptions instead.
>> +         */
>> +        if ( trap->scause == CAUSE_LOAD_PAGE_FAULT )
>> +            trap->scause = CAUSE_FETCH_PAGE_FAULT;
>> +    }
>> +    else
>> +    {
>> +        asm volatile ( "\n"
>> +            "1:\n"
>> +#ifdef CONFIG_RISCV_64
>> +            "hlv.d %[val], (%[addr])\n"
>> +#else
>> +            "hlv.w %[val], (%[addr])\n"
>> +#endif
> 
> Once again please use enough care that RV128 would at least obviously fail to
> build, rather than building something which then doesn't work.

Sure, I will do the following:

#if defined(CONFIG_RISCV_64)
             "hlv.d %[val], (%[addr])\n"
#elif defined(CONFIG_RISCV_32)
             "hlv.w %[val], (%[addr])\n"
#else
              #error "unsupported RISC-V variant: no hlv for a machine word"
#endif

> 
>> +            "2:\n"
>> +            ASM_EXTABLE_TRAP_INFO(1b, 2b, %[ti])
>> +        : [val] "=&r" (val)
>> +        : [addr] "r" (guest_addr), [ti] "r" (trap) : "memory" );
>> +    }
>> +
>> +    csr_write(CSR_HSTATUS, old_hstatus);
>> +
>> +    local_irq_restore(flags);
>> +
>> +    return val;
>> +}
> For both reads and fetches - are there no alignment constraints at all on the
> incoming guest_addr?
> 

For the fetch path there is an implicit constraint, but the architecture
guarantees it: guest_addr is always the guest's sepc, and IALIGN is 16
bits (32 without the C extension), so it cannot be odd, the guest would
have taken an instruction-address-misaligned exception before we ever 
saw this trap. hlvx.hu is then a naturally aligned halfword access, and
advancing by 2 preserves that.

For the data read there is deliberately no constraint: HLV behaves as 
the guest's own access would, so on a hart which handles misaligned 
accesses it simply works, and on one which doesn't it raises 
load-address-misaligned, which the exception table turns into 
trap->scause for the caller to redirect.

But then it will be need to:

--- a/xen/arch/riscv/traps.c
+++ b/xen/arch/riscv/traps.c
@@ -565,51 +565,73 @@ static void do_unexpected_trap(const struct 
cpu_user_regs *regs)
  void do_trap(struct cpu_user_regs *cpu_regs)
  {
      register_t pc = cpu_regs->sepc;
      unsigned long cause = csr_read(CSR_SCAUSE);

+    /*
+     * A synchronous trap taken while Xen itself was running may come 
from an
+     * access done on a vCPU's behalf, e.g. the hlv/hlvx sequences in
+     * riscv_vcpu_unpriv_read(). Those accesses are covered by 
exception table
+     * entries which record the fault details for the caller and resume
+     * execution past the faulting instruction.
+     *
+     * Interrupts must be excluded here: one taken at an address which 
happens
+     * to be listed in the exception table would otherwise be "fixed 
up" as if
+     * the access itself had faulted, silently skipping it.
+     *
+     * Returning early skips check_for_pcpu_work() below, which is correct:
+     * that only runs for traps taken from the guest.
+     */
+    if ( !(cause & CAUSE_IRQ_FLAG) && !(cpu_regs->hstatus & HSTATUS_SPV) &&
+         fixup_exception(cpu_regs) )
+        return;
+
      switch ( cause )
      {
      case CAUSE_VIRTUAL_SUPERVISOR_ECALL:
          /* CAUSE_VIRTUAL_SUPERVISOR_ECALL should come from VS-mode */
          BUG_ON(!(cpu_regs->hstatus & HSTATUS_SPV));

          vsbi_handle_ecall(cpu_regs);
          break;

      case CAUSE_LOAD_GUEST_PAGE_FAULT:
      case CAUSE_STORE_GUEST_PAGE_FAULT:
+        /*
+         * Anything not recovered by the exception table above must 
have come
+         * from the guest: a G-stage fault taken in Xen context, e.g. by an
+         * hlv/hlvx not covered by an entry, is a bug.
+         */
+        BUG_ON(!(cpu_regs->hstatus & HSTATUS_SPV));
+
          handle_guest_page_fault(cause, cpu_regs);
          break;

      case CAUSE_VIRTUAL_INST_FAULT:
      {
          int ret;

          BUG_ON(!(cpu_regs->hstatus & HSTATUS_SPV));

          ret = handle_virt_instruction_fault(current);
          if ( ret < 0 )
              /* TODO: crash only domain instead of Xen? */
              /* domain_crash(current->domain); */
              panic("couldn't handle CAUSE_VIRTUAL_INST_FAULT: %d\n", ret);
          break;
      }

      case CAUSE_ILLEGAL_INSTRUCTION:
          if ( do_bug_frame(cpu_regs, pc) >= 0 )
          {
              if ( !(is_kernel_text(pc) || is_kernel_inittext(pc)) )
              {
                  printk("Something wrong with PC: %#lx\n", pc);
                  die();
              }

              cpu_regs->sepc += GET_INSN_LENGTH(*(uint16_t *)pc);

              break;
          }

-        if ( fixup_exception(cpu_regs) )
-            break;
-
          fallthrough;
      default:
          if ( cause & CAUSE_IRQ_FLAG )
          {
              /* Handle interrupt */

Does it make sense?

Thanks.

~ Oleksii

Re: [PATCH v1 13/17] xen/riscv: add unprivileged guest memory read helper

[Jan Beulich]

On 17.08.2026 17:36, Oleksii Kurochko wrote:
> On 8/12/26 5:30 PM, Jan Beulich wrote:
>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>> Introduce riscv_vcpu_unpriv_read() to allow Xen to safely read guest memory
>>> using HLV/HLVX instructions while reliably capturing trap context.
>>
>> Both for the title and the function name: How does "unprivileged" matter here?
> 
> Unprivileged because HLV/HLVX reads guest memory as if it were accessed 
> from a less-privileged (guest) context, rather than by the hypervisor in 
> HS mode.
> 
> I think I am okay generally to drop "unpriv..." from the function name.
> 
>> The same functions would be use for reading Dom0's memory, wouldn't they?
> 
> Yes, I don't see any issue to let this function to read DomO's memory 
> too. But Dom0 could be counted as "unprivileged" too as it is executed 
> in VS-mode which is less-privileged then HS-mode.

Well, you need to properly distinguish the different cases of "unprivileged"
when writing titles / descriptions. The way you put it in your reply above
("less-privileged (guest)") is sufficiently unambiguous, but what you have
in the title ("unprivileged guest") clearly is a synonym for "DomU".

In the end, as you confirm, "guest" alone is all that's needed here to know
what is being talked about. (That said: We often distinguish "guest" and
"domain", with the former meaning DomU but the latter meaning all domains,
including Dom0 and service domains. Yet for the purpose here I think "guest"
is good to use, not matter that all kinds of domains are meant.)

>>> +    if ( read_insn )
>>> +    {
>>> +        asm volatile ( "\n"
>>> +            "1:\n"
>>> +            "   hlvx.hu %[val], (%[addr])\n"
>>> +            ASM_EXTABLE_TRAP_INFO(1b, 3f, %[ti])
>>> +            "   andi %[tmp], %[val], 3\n"
>>> +            "   addi %[tmp], %[tmp], -3\n"
>>> +            "   bne %[tmp], zero, 3f\n"
>>> +            "   addi %[addr], %[addr], 2\n"
>>> +            "\n"
>>> +            "2:\n"
>>> +            "   hlvx.hu %[tmp], (%[addr])\n"
>>> +            ASM_EXTABLE_TRAP_INFO(2b, 3f, %[ti])
>>> +            "   sll %[tmp], %[tmp], 16\n"
>>> +            "   add %[val], %[val], %[tmp]\n"
>>> +            "3:\n"
>>> +        : [val] "=&r" (val), [tmp] "=&r" (tmp), [addr] "+&r" (guest_addr)
>>> +        : [ti] "r" (trap) : "memory" );
>>
>> You want to tell the compiler that *trap is written. Instead I don't see
>> why a memory clobber would be needed: You access a different address space,
>> i.e. nothing the compiler can make any assumptions about.
> 
> memory clobber tells the compiler that the assembly code performs memory 
> reads or writes to items other than those listed in the input and output 
> operands and so I don't tell here that *trap will be changed.
> 
> Why this understanding is wrong?

You can (ab)use "memory" for that purpose, but why would you when you can
properly express the operand? All that achieves is the compiler possibly
having to emit less efficient code.

> Alternative, I think, could be:
>          : [val] "+r" (val), "+m" (*trap)
>          : [addr] "r" (guest_addr), [ti] "r" (trap) );
> And then memory clobber could be dropped.
> 
> 
> 
>>
>> You also need to take precautions for not returning an uninitialized "val".
>> I think the variable wants initializing (perhaps to ~0) and "+r" wants
>> using as constraint. (Afaik & isn't necessary to use together with +.)
> 
> I agree with '+' if we will initialize val with some value.
> 
> Regarding, '&' my understanding is that I have to use it always when

When what exactly? If an operand is both input and output, how could the
compiler re-use the (generally) register for any further purpose? '&'
indicates to the compiler that it may not use the register used for an
output to hold some input's value, as that value may be lost by the time
the input is actually consumed.

>>> +        /*
>>> +         * Although HLVX instructions' explicit memory accesses require execute
>>> +         * permissions, they still raise the same exceptions as other load
>>> +         * instructions, rather than raising fetch exceptions instead.
>>> +         */
>>> +        if ( trap->scause == CAUSE_LOAD_PAGE_FAULT )
>>> +            trap->scause = CAUSE_FETCH_PAGE_FAULT;
>>> +    }
>>> +    else
>>> +    {
>>> +        asm volatile ( "\n"
>>> +            "1:\n"
>>> +#ifdef CONFIG_RISCV_64
>>> +            "hlv.d %[val], (%[addr])\n"
>>> +#else
>>> +            "hlv.w %[val], (%[addr])\n"
>>> +#endif
>>
>> Once again please use enough care that RV128 would at least obviously fail to
>> build, rather than building something which then doesn't work.
> 
> Sure, I will do the following:
> 
> #if defined(CONFIG_RISCV_64)
>              "hlv.d %[val], (%[addr])\n"
> #elif defined(CONFIG_RISCV_32)
>              "hlv.w %[val], (%[addr])\n"
> #else
>               #error "unsupported RISC-V variant: no hlv for a machine word"
> #endif

And that, as before, without indenting #, please.

Jan

Re: [PATCH v1 13/17] xen/riscv: add unprivileged guest memory read helper

[Oleksii Kurochko]

On 8/18/26 10:17 AM, Jan Beulich wrote:
> On 17.08.2026 17:36, Oleksii Kurochko wrote:
>> On 8/12/26 5:30 PM, Jan Beulich wrote:
>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>> Introduce riscv_vcpu_unpriv_read() to allow Xen to safely read guest memory
>>>> using HLV/HLVX instructions while reliably capturing trap context.
>>>
>>> Both for the title and the function name: How does "unprivileged" matter here?
>>
>> Unprivileged because HLV/HLVX reads guest memory as if it were accessed
>> from a less-privileged (guest) context, rather than by the hypervisor in
>> HS mode.
>>
>> I think I am okay generally to drop "unpriv..." from the function name.
>>
>>> The same functions would be use for reading Dom0's memory, wouldn't they?
>>
>> Yes, I don't see any issue to let this function to read DomO's memory
>> too. But Dom0 could be counted as "unprivileged" too as it is executed
>> in VS-mode which is less-privileged then HS-mode.
> 
> Well, you need to properly distinguish the different cases of "unprivileged"
> when writing titles / descriptions. The way you put it in your reply above
> ("less-privileged (guest)") is sufficiently unambiguous, but what you have
> in the title ("unprivileged guest") clearly is a synonym for "DomU".
> 
> In the end, as you confirm, "guest" alone is all that's needed here to know
> what is being talked about. (That said: We often distinguish "guest" and
> "domain", with the former meaning DomU but the latter meaning all domains,
> including Dom0 and service domains. Yet for the purpose here I think "guest"
> is good to use, not matter that all kinds of domains are meant.)

I will use then guest.

> 
>>>> +    if ( read_insn )
>>>> +    {
>>>> +        asm volatile ( "\n"
>>>> +            "1:\n"
>>>> +            "   hlvx.hu %[val], (%[addr])\n"
>>>> +            ASM_EXTABLE_TRAP_INFO(1b, 3f, %[ti])
>>>> +            "   andi %[tmp], %[val], 3\n"
>>>> +            "   addi %[tmp], %[tmp], -3\n"
>>>> +            "   bne %[tmp], zero, 3f\n"
>>>> +            "   addi %[addr], %[addr], 2\n"
>>>> +            "\n"
>>>> +            "2:\n"
>>>> +            "   hlvx.hu %[tmp], (%[addr])\n"
>>>> +            ASM_EXTABLE_TRAP_INFO(2b, 3f, %[ti])
>>>> +            "   sll %[tmp], %[tmp], 16\n"
>>>> +            "   add %[val], %[val], %[tmp]\n"
>>>> +            "3:\n"
>>>> +        : [val] "=&r" (val), [tmp] "=&r" (tmp), [addr] "+&r" (guest_addr)
>>>> +        : [ti] "r" (trap) : "memory" );
>>>
>>> You want to tell the compiler that *trap is written. Instead I don't see
>>> why a memory clobber would be needed: You access a different address space,
>>> i.e. nothing the compiler can make any assumptions about.
>>
>> memory clobber tells the compiler that the assembly code performs memory
>> reads or writes to items other than those listed in the input and output
>> operands and so I don't tell here that *trap will be changed.
>>
>> Why this understanding is wrong?
> 
> You can (ab)use "memory" for that purpose, but why would you when you can
> properly express the operand? All that achieves is the compiler possibly
> having to emit less efficient code.

Then I will use the option mentioned ...

> 
>> Alternative, I think, could be:
>>           : [val] "+r" (val), "+m" (*trap)
>>           : [addr] "r" (guest_addr), [ti] "r" (trap) );
>> And then memory clobber could be dropped.

... here.

Probably I have to return '[addr] "r" (guest_addr)' to output and use 
+&r constraint.

>>
>>
>>
>>>
>>> You also need to take precautions for not returning an uninitialized "val".
>>> I think the variable wants initializing (perhaps to ~0) and "+r" wants
>>> using as constraint. (Afaik & isn't necessary to use together with +.)
>>
>> I agree with '+' if we will initialize val with some value.
>>
>> Regarding, '&' my understanding is that I have to use it always when
> 
> When what exactly? If an operand is both input and output, how could the
> compiler re-use the (generally) register for any further purpose? '&'
> indicates to the compiler that it may not use the register used for an
> output to hold some input's value, as that value may be lost by the time
> the input is actually consumed.

But what is written in the gcc doc:

& - Means (in a particular alternative) that this operand is an 
earlyclobber operand, which is written before the instruction is 
finished using the input operands.

What sounds like if an operand (val) in our case is written before the 
instruction which using the input operands (and after the write 
instuction which writes val there are instructions which are using input 
operands) it is needed to have &.

t1:  hlvx.hu %[val], (%[addr])   W:val R:addr + can trap -> read register ti

t2:  andi    %[tmp], %[val], 3   W:tmp   R:val
t3:  addi    %[tmp], %[tmp], -3
t4:  bnez    %[tmp], 3f
t5:  addi    %[addr], %[addr], 2 W:addr  R:addr
t6:  hlvx.hu %[tmp], (%[addr])   W:tmp   R:addr + can trap -> read 
register ti

t7:  slli    %[tmp], %[tmp], 16
t8:  or      %[val], %[val], %[tmp] W:val

So val is written on t1 before t6 where addr and ti still alive.

The similar is for [addr] "+&r" (guest_addr):

addr is written on t5 and input ti is alive till t6. So if allocator 
will allocate the same register for addr and ti then addi %[addr], 
%[addr], 2 will break a pointer and handler will get something wrong.

Am I missing something?

If I am still wrong then in both cases should be just "+r"?

~ Oleksii

Re: [PATCH v1 13/17] xen/riscv: add unprivileged guest memory read helper

[Jan Beulich]

On 18.08.2026 12:27, Oleksii Kurochko wrote:
> On 8/18/26 10:17 AM, Jan Beulich wrote:
>> On 17.08.2026 17:36, Oleksii Kurochko wrote:
>>> On 8/12/26 5:30 PM, Jan Beulich wrote:
>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>> +    if ( read_insn )
>>>>> +    {
>>>>> +        asm volatile ( "\n"
>>>>> +            "1:\n"
>>>>> +            "   hlvx.hu %[val], (%[addr])\n"
>>>>> +            ASM_EXTABLE_TRAP_INFO(1b, 3f, %[ti])
>>>>> +            "   andi %[tmp], %[val], 3\n"
>>>>> +            "   addi %[tmp], %[tmp], -3\n"
>>>>> +            "   bne %[tmp], zero, 3f\n"
>>>>> +            "   addi %[addr], %[addr], 2\n"
>>>>> +            "\n"
>>>>> +            "2:\n"
>>>>> +            "   hlvx.hu %[tmp], (%[addr])\n"
>>>>> +            ASM_EXTABLE_TRAP_INFO(2b, 3f, %[ti])
>>>>> +            "   sll %[tmp], %[tmp], 16\n"
>>>>> +            "   add %[val], %[val], %[tmp]\n"
>>>>> +            "3:\n"
>>>>> +        : [val] "=&r" (val), [tmp] "=&r" (tmp), [addr] "+&r" (guest_addr)
>>>>> +        : [ti] "r" (trap) : "memory" );
>>>>
>>>> You want to tell the compiler that *trap is written. Instead I don't see
>>>> why a memory clobber would be needed: You access a different address space,
>>>> i.e. nothing the compiler can make any assumptions about.
>>>
>>> memory clobber tells the compiler that the assembly code performs memory
>>> reads or writes to items other than those listed in the input and output
>>> operands and so I don't tell here that *trap will be changed.
>>>
>>> Why this understanding is wrong?
>>
>> You can (ab)use "memory" for that purpose, but why would you when you can
>> properly express the operand? All that achieves is the compiler possibly
>> having to emit less efficient code.
> 
> Then I will use the option mentioned ...
> 
>>
>>> Alternative, I think, could be:
>>>           : [val] "+r" (val), "+m" (*trap)
>>>           : [addr] "r" (guest_addr), [ti] "r" (trap) );
>>> And then memory clobber could be dropped.
> 
> ... here.
> 
> Probably I have to return '[addr] "r" (guest_addr)' to output and use 
> +&r constraint.
> 
>>>> You also need to take precautions for not returning an uninitialized "val".
>>>> I think the variable wants initializing (perhaps to ~0) and "+r" wants
>>>> using as constraint. (Afaik & isn't necessary to use together with +.)
>>>
>>> I agree with '+' if we will initialize val with some value.
>>>
>>> Regarding, '&' my understanding is that I have to use it always when
>>
>> When what exactly? If an operand is both input and output, how could the
>> compiler re-use the (generally) register for any further purpose? '&'
>> indicates to the compiler that it may not use the register used for an
>> output to hold some input's value, as that value may be lost by the time
>> the input is actually consumed.
> 
> But what is written in the gcc doc:
> 
> & - Means (in a particular alternative) that this operand is an 
> earlyclobber operand, which is written before the instruction is 
> finished using the input operands.
> 
> What sounds like if an operand (val) in our case is written before the 
> instruction which using the input operands (and after the write 
> instuction which writes val there are instructions which are using input 
> operands) it is needed to have &.

And that's indeed relevant, just not here. My crucial earlier question was:
"If an operand is both input and output, how could the compiler re-use the
(generally) register for any further purpose?" There is a case where the
answer to this is not "it can't". In your case all inputs are distinct; in
e.g. (using x86 assembly, sorry):

int test(int i, int j) {
	asm("nop %0; nop %1" : "+r" (i) : "r" (i));
	asm("cmc; nop %0; nop %1" : "+&r" (j) : "r" (j));

	return i + j;
}

using "+&r" indeed makes a difference.

> t1:  hlvx.hu %[val], (%[addr])   W:val R:addr + can trap -> read register ti
> 
> t2:  andi    %[tmp], %[val], 3   W:tmp   R:val
> t3:  addi    %[tmp], %[tmp], -3
> t4:  bnez    %[tmp], 3f
> t5:  addi    %[addr], %[addr], 2 W:addr  R:addr
> t6:  hlvx.hu %[tmp], (%[addr])   W:tmp   R:addr + can trap -> read 
> register ti
> 
> t7:  slli    %[tmp], %[tmp], 16
> t8:  or      %[val], %[val], %[tmp] W:val
> 
> So val is written on t1 before t6 where addr and ti still alive.
> 
> The similar is for [addr] "+&r" (guest_addr):
> 
> addr is written on t5 and input ti is alive till t6. So if allocator 
> will allocate the same register for addr and ti then addi %[addr], 
> %[addr], 2 will break a pointer and handler will get something wrong.
> 
> Am I missing something?
> 
> If I am still wrong then in both cases should be just "+r"?

As per above, if you want to play absolutely by the rules, use "+&r",
even if that's unnecessary here.

Jan

Re: [PATCH v1 13/17] xen/riscv: add unprivileged guest memory read helper

[Oleksii Kurochko]

On 8/18/26 12:43 PM, Jan Beulich wrote:
> On 18.08.2026 12:27, Oleksii Kurochko wrote:
>> On 8/18/26 10:17 AM, Jan Beulich wrote:
>>> On 17.08.2026 17:36, Oleksii Kurochko wrote:
>>>> On 8/12/26 5:30 PM, Jan Beulich wrote:
>>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>>> +    if ( read_insn )
>>>>>> +    {
>>>>>> +        asm volatile ( "\n"
>>>>>> +            "1:\n"
>>>>>> +            "   hlvx.hu %[val], (%[addr])\n"
>>>>>> +            ASM_EXTABLE_TRAP_INFO(1b, 3f, %[ti])
>>>>>> +            "   andi %[tmp], %[val], 3\n"
>>>>>> +            "   addi %[tmp], %[tmp], -3\n"
>>>>>> +            "   bne %[tmp], zero, 3f\n"
>>>>>> +            "   addi %[addr], %[addr], 2\n"
>>>>>> +            "\n"
>>>>>> +            "2:\n"
>>>>>> +            "   hlvx.hu %[tmp], (%[addr])\n"
>>>>>> +            ASM_EXTABLE_TRAP_INFO(2b, 3f, %[ti])
>>>>>> +            "   sll %[tmp], %[tmp], 16\n"
>>>>>> +            "   add %[val], %[val], %[tmp]\n"
>>>>>> +            "3:\n"
>>>>>> +        : [val] "=&r" (val), [tmp] "=&r" (tmp), [addr] "+&r" (guest_addr)
>>>>>> +        : [ti] "r" (trap) : "memory" );
>>>>>
>>>>> You want to tell the compiler that *trap is written. Instead I don't see
>>>>> why a memory clobber would be needed: You access a different address space,
>>>>> i.e. nothing the compiler can make any assumptions about.
>>>>
>>>> memory clobber tells the compiler that the assembly code performs memory
>>>> reads or writes to items other than those listed in the input and output
>>>> operands and so I don't tell here that *trap will be changed.
>>>>
>>>> Why this understanding is wrong?
>>>
>>> You can (ab)use "memory" for that purpose, but why would you when you can
>>> properly express the operand? All that achieves is the compiler possibly
>>> having to emit less efficient code.
>>
>> Then I will use the option mentioned ...
>>
>>>
>>>> Alternative, I think, could be:
>>>>            : [val] "+r" (val), "+m" (*trap)
>>>>            : [addr] "r" (guest_addr), [ti] "r" (trap) );
>>>> And then memory clobber could be dropped.
>>
>> ... here.
>>
>> Probably I have to return '[addr] "r" (guest_addr)' to output and use
>> +&r constraint.
>>
>>>>> You also need to take precautions for not returning an uninitialized "val".
>>>>> I think the variable wants initializing (perhaps to ~0) and "+r" wants
>>>>> using as constraint. (Afaik & isn't necessary to use together with +.)
>>>>
>>>> I agree with '+' if we will initialize val with some value.
>>>>
>>>> Regarding, '&' my understanding is that I have to use it always when
>>>
>>> When what exactly? If an operand is both input and output, how could the
>>> compiler re-use the (generally) register for any further purpose? '&'
>>> indicates to the compiler that it may not use the register used for an
>>> output to hold some input's value, as that value may be lost by the time
>>> the input is actually consumed.
>>
>> But what is written in the gcc doc:
>>
>> & - Means (in a particular alternative) that this operand is an
>> earlyclobber operand, which is written before the instruction is
>> finished using the input operands.
>>
>> What sounds like if an operand (val) in our case is written before the
>> instruction which using the input operands (and after the write
>> instuction which writes val there are instructions which are using input
>> operands) it is needed to have &.
> 
> And that's indeed relevant, just not here. My crucial earlier question was:
> "If an operand is both input and output, how could the compiler re-use the
> (generally) register for any further purpose?" There is a case where the
> answer to this is not "it can't". In your case all inputs are distinct; in
> e.g. (using x86 assembly, sorry):
> 
> int test(int i, int j) {
> 	asm("nop %0; nop %1" : "+r" (i) : "r" (i));
> 	asm("cmc; nop %0; nop %1" : "+&r" (j) : "r" (j));
> 
> 	return i + j;
> }
> 
> using "+&r" indeed makes a difference.

I think it is clear when operands are equal. but what about the case 
when they are different in  first  case:
...
   asm("nop %0; nop %1" : "+r" (i) : "r" (j));
...

What guarantees that i and j will be in different registers?

We have the similar situation in RISC-V code of riscv_vcpu_unpriv_read():

         : [val] "+r" (val), [tmp] "=&r" (tmp), [addr] "+r" (guest_addr),
           "+m" (*trap)
         : [ti] "r" (trap) );

With having & for val and guest_addr & guarantees that the same 
registers won't be re-used for ti (and so ti won't be corrupted) but 
without it?

~ Oleksii

> 
>> t1:  hlvx.hu %[val], (%[addr])   W:val R:addr + can trap -> read register ti
>>
>> t2:  andi    %[tmp], %[val], 3   W:tmp   R:val
>> t3:  addi    %[tmp], %[tmp], -3
>> t4:  bnez    %[tmp], 3f
>> t5:  addi    %[addr], %[addr], 2 W:addr  R:addr
>> t6:  hlvx.hu %[tmp], (%[addr])   W:tmp   R:addr + can trap -> read
>> register ti
>>
>> t7:  slli    %[tmp], %[tmp], 16
>> t8:  or      %[val], %[val], %[tmp] W:val
>>
>> So val is written on t1 before t6 where addr and ti still alive.
>>
>> The similar is for [addr] "+&r" (guest_addr):
>>
>> addr is written on t5 and input ti is alive till t6. So if allocator
>> will allocate the same register for addr and ti then addi %[addr],
>> %[addr], 2 will break a pointer and handler will get something wrong.
>>
>> Am I missing something?
>>
>> If I am still wrong then in both cases should be just "+r"?
> 
> As per above, if you want to play absolutely by the rules, use "+&r",
> even if that's unnecessary here.
> 
> Jan

Re: [PATCH v1 13/17] xen/riscv: add unprivileged guest memory read helper

[Jan Beulich]

On 18.08.2026 15:38, Oleksii Kurochko wrote:
> 
> 
> On 8/18/26 12:43 PM, Jan Beulich wrote:
>> On 18.08.2026 12:27, Oleksii Kurochko wrote:
>>> On 8/18/26 10:17 AM, Jan Beulich wrote:
>>>> On 17.08.2026 17:36, Oleksii Kurochko wrote:
>>>>> On 8/12/26 5:30 PM, Jan Beulich wrote:
>>>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>>>> +    if ( read_insn )
>>>>>>> +    {
>>>>>>> +        asm volatile ( "\n"
>>>>>>> +            "1:\n"
>>>>>>> +            "   hlvx.hu %[val], (%[addr])\n"
>>>>>>> +            ASM_EXTABLE_TRAP_INFO(1b, 3f, %[ti])
>>>>>>> +            "   andi %[tmp], %[val], 3\n"
>>>>>>> +            "   addi %[tmp], %[tmp], -3\n"
>>>>>>> +            "   bne %[tmp], zero, 3f\n"
>>>>>>> +            "   addi %[addr], %[addr], 2\n"
>>>>>>> +            "\n"
>>>>>>> +            "2:\n"
>>>>>>> +            "   hlvx.hu %[tmp], (%[addr])\n"
>>>>>>> +            ASM_EXTABLE_TRAP_INFO(2b, 3f, %[ti])
>>>>>>> +            "   sll %[tmp], %[tmp], 16\n"
>>>>>>> +            "   add %[val], %[val], %[tmp]\n"
>>>>>>> +            "3:\n"
>>>>>>> +        : [val] "=&r" (val), [tmp] "=&r" (tmp), [addr] "+&r" (guest_addr)
>>>>>>> +        : [ti] "r" (trap) : "memory" );
>>>>>>
>>>>>> You want to tell the compiler that *trap is written. Instead I don't see
>>>>>> why a memory clobber would be needed: You access a different address space,
>>>>>> i.e. nothing the compiler can make any assumptions about.
>>>>>
>>>>> memory clobber tells the compiler that the assembly code performs memory
>>>>> reads or writes to items other than those listed in the input and output
>>>>> operands and so I don't tell here that *trap will be changed.
>>>>>
>>>>> Why this understanding is wrong?
>>>>
>>>> You can (ab)use "memory" for that purpose, but why would you when you can
>>>> properly express the operand? All that achieves is the compiler possibly
>>>> having to emit less efficient code.
>>>
>>> Then I will use the option mentioned ...
>>>
>>>>
>>>>> Alternative, I think, could be:
>>>>>            : [val] "+r" (val), "+m" (*trap)
>>>>>            : [addr] "r" (guest_addr), [ti] "r" (trap) );
>>>>> And then memory clobber could be dropped.
>>>
>>> ... here.
>>>
>>> Probably I have to return '[addr] "r" (guest_addr)' to output and use
>>> +&r constraint.
>>>
>>>>>> You also need to take precautions for not returning an uninitialized "val".
>>>>>> I think the variable wants initializing (perhaps to ~0) and "+r" wants
>>>>>> using as constraint. (Afaik & isn't necessary to use together with +.)
>>>>>
>>>>> I agree with '+' if we will initialize val with some value.
>>>>>
>>>>> Regarding, '&' my understanding is that I have to use it always when
>>>>
>>>> When what exactly? If an operand is both input and output, how could the
>>>> compiler re-use the (generally) register for any further purpose? '&'
>>>> indicates to the compiler that it may not use the register used for an
>>>> output to hold some input's value, as that value may be lost by the time
>>>> the input is actually consumed.
>>>
>>> But what is written in the gcc doc:
>>>
>>> & - Means (in a particular alternative) that this operand is an
>>> earlyclobber operand, which is written before the instruction is
>>> finished using the input operands.
>>>
>>> What sounds like if an operand (val) in our case is written before the
>>> instruction which using the input operands (and after the write
>>> instuction which writes val there are instructions which are using input
>>> operands) it is needed to have &.
>>
>> And that's indeed relevant, just not here. My crucial earlier question was:
>> "If an operand is both input and output, how could the compiler re-use the
>> (generally) register for any further purpose?" There is a case where the
>> answer to this is not "it can't". In your case all inputs are distinct; in
>> e.g. (using x86 assembly, sorry):
>>
>> int test(int i, int j) {
>> 	asm("nop %0; nop %1" : "+r" (i) : "r" (i));
>> 	asm("cmc; nop %0; nop %1" : "+&r" (j) : "r" (j));
>>
>> 	return i + j;
>> }
>>
>> using "+&r" indeed makes a difference.
> 
> I think it is clear when operands are equal. but what about the case 
> when they are different in  first  case:
> ...
>    asm("nop %0; nop %1" : "+r" (i) : "r" (j));
> ...
> 
> What guarantees that i and j will be in different registers?

They can both be in the same register only if the compiler sees respective
call sites, and can determine that the same value is passed for i and j.
This ...

> We have the similar situation in RISC-V code of riscv_vcpu_unpriv_read():
> 
>          : [val] "+r" (val), [tmp] "=&r" (tmp), [addr] "+r" (guest_addr),
>            "+m" (*trap)
>          : [ti] "r" (trap) );
> 
> With having & for val and guest_addr & guarantees that the same 
> registers won't be re-used for ti (and so ti won't be corrupted) but 
> without it?

... pretty clearly is impossible in your case: val and guest_addr can't
possibly have the same origin as trap.

Yet to repeat what I said before, and to not needlessly continue this
discussion, just go and use "+&r", despite that being unnecessary here.

Jan

[PATCH v1 14/17] xen/riscv: add guest page fault handling stub

[Oleksii Kurochko]

Add a stub handler for guest page faults and hook it into the trap path,
providing the basic infrastructure for future MMIO trap handling.

This will be used, for example, to trap accesses to APLIC registers in
order to initialize and emulate the interrupt controller for guests.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 xen/arch/riscv/traps.c | 66 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)

diff --git a/xen/arch/riscv/traps.c b/xen/arch/riscv/traps.c
index d35c013e1399..1c97bd101948 100644
--- a/xen/arch/riscv/traps.c
+++ b/xen/arch/riscv/traps.c
@@ -191,6 +191,67 @@ static void timer_interrupt(void)
     raise_softirq(TIMER_SOFTIRQ);
 }
 
+static always_inline unsigned long get_faulting_gpa(void)
+{
+    /*
+     * According to RISC-V spec:
+     *  18.2.8. Hypervisor Trap Value Register (htval)
+     *   ...
+     *   A guest physical address written to htval is shifted right by 2 bits
+     *   to accommodate addresses wider than the current XLEN.
+     *   ...
+     *   If the least-significant two bits of a faulting guest physical address
+     *   are needed, these bits are ordinarily the same as the
+     *   least-significant two bits of the faulting virtual address in stval.
+     *   For faults due to implicit memory accesses for VS-stage address
+     *   translation, the least-significant two bits are instead zeros. These
+     *   cases can be distinguished using the value provided in register htinst.
+     */
+    return (csr_read(CSR_HTVAL) << 2) | (csr_read(CSR_STVAL) & 0x3);
+}
+
+static int emulate_load(unsigned long fault_addr, unsigned long htinst)
+{
+    return -EOPNOTSUPP;
+}
+
+static int emulate_store(unsigned long fault_addr, unsigned long htinst)
+{
+    return -EOPNOTSUPP;
+}
+
+static void handle_guest_page_fault(unsigned long cause,
+                                    struct cpu_user_regs *regs)
+{
+    unsigned long addr;
+    int rc;
+
+    addr = get_faulting_gpa();
+
+    switch ( cause )
+    {
+    case CAUSE_LOAD_GUEST_PAGE_FAULT:
+        rc = emulate_load(addr, csr_read(CSR_HTINST));
+        break;
+
+    case CAUSE_STORE_GUEST_PAGE_FAULT:
+        rc = emulate_store(addr, csr_read(CSR_HTINST));
+        break;
+
+    default:
+        rc = -EOPNOTSUPP;
+        ASSERT_UNREACHABLE();
+        break;
+    }
+
+    if ( rc )
+        domain_crash(current->domain,
+                     "%s: unable to handle faulted guest %s addr %#lx\n",
+                     __func__,
+                     (cause == CAUSE_LOAD_GUEST_PAGE_FAULT) ? "load" : "store",
+                     addr);
+}
+
 void do_trap(struct cpu_user_regs *cpu_regs)
 {
     register_t pc = cpu_regs->sepc;
@@ -205,6 +266,11 @@ void do_trap(struct cpu_user_regs *cpu_regs)
         vsbi_handle_ecall(cpu_regs);
         break;
 
+    case CAUSE_LOAD_GUEST_PAGE_FAULT:
+    case CAUSE_STORE_GUEST_PAGE_FAULT:
+        handle_guest_page_fault(cause, cpu_regs);
+        break;
+
     case CAUSE_ILLEGAL_INSTRUCTION:
         if ( do_bug_frame(cpu_regs, pc) >= 0 )
         {
-- 
2.54.0

Re: [PATCH v1 14/17] xen/riscv: add guest page fault handling stub

[Jan Beulich]

On 20.07.2026 18:02, Oleksii Kurochko wrote:
> --- a/xen/arch/riscv/traps.c
> +++ b/xen/arch/riscv/traps.c
> @@ -191,6 +191,67 @@ static void timer_interrupt(void)
>      raise_softirq(TIMER_SOFTIRQ);
>  }
>  
> +static always_inline unsigned long get_faulting_gpa(void)

May I suggest to use always_inline only when inlining is _functionally_
required?

> +{
> +    /*
> +     * According to RISC-V spec:
> +     *  18.2.8. Hypervisor Trap Value Register (htval)
> +     *   ...
> +     *   A guest physical address written to htval is shifted right by 2 bits
> +     *   to accommodate addresses wider than the current XLEN.
> +     *   ...
> +     *   If the least-significant two bits of a faulting guest physical address
> +     *   are needed, these bits are ordinarily the same as the
> +     *   least-significant two bits of the faulting virtual address in stval.
> +     *   For faults due to implicit memory accesses for VS-stage address
> +     *   translation, the least-significant two bits are instead zeros. These
> +     *   cases can be distinguished using the value provided in register htinst.
> +     */
> +    return (csr_read(CSR_HTVAL) << 2) | (csr_read(CSR_STVAL) & 0x3);

Well, okay, but instead of not losing the bottom two bits you're now losing
the top two ones.

Also the spec reads as if htval only _may_ hold the original address of the
faulting access. What if htval ends up 0?

Further, nit: There's (once again) no real value in the 0x prefix, I don't
think.

> +static int emulate_load(unsigned long fault_addr, unsigned long htinst)
> +{
> +    return -EOPNOTSUPP;
> +}
> +
> +static int emulate_store(unsigned long fault_addr, unsigned long htinst)
> +{
> +    return -EOPNOTSUPP;
> +}
> +
> +static void handle_guest_page_fault(unsigned long cause,
> +                                    struct cpu_user_regs *regs)
> +{
> +    unsigned long addr;
> +    int rc;
> +
> +    addr = get_faulting_gpa();

Can't this become the initializer of the variable?

Jan

Re: [PATCH v1 14/17] xen/riscv: add guest page fault handling stub

[Oleksii Kurochko]

On 8/12/26 5:48 PM, Jan Beulich wrote:
> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>> --- a/xen/arch/riscv/traps.c
>> +++ b/xen/arch/riscv/traps.c
>> @@ -191,6 +191,67 @@ static void timer_interrupt(void)
>>       raise_softirq(TIMER_SOFTIRQ);
>>   }
>>   
>> +static always_inline unsigned long get_faulting_gpa(void)
> 
> May I suggest to use always_inline only when inlining is _functionally_
> required?

Sure. But it ins't clear to me why it isn't a case here? Is it connected 
to that function is static and too simple so a compiler will do by itself?

> 
>> +{
>> +    /*
>> +     * According to RISC-V spec:
>> +     *  18.2.8. Hypervisor Trap Value Register (htval)
>> +     *   ...
>> +     *   A guest physical address written to htval is shifted right by 2 bits
>> +     *   to accommodate addresses wider than the current XLEN.
>> +     *   ...
>> +     *   If the least-significant two bits of a faulting guest physical address
>> +     *   are needed, these bits are ordinarily the same as the
>> +     *   least-significant two bits of the faulting virtual address in stval.
>> +     *   For faults due to implicit memory accesses for VS-stage address
>> +     *   translation, the least-significant two bits are instead zeros. These
>> +     *   cases can be distinguished using the value provided in register htinst.
>> +     */
>> +    return (csr_read(CSR_HTVAL) << 2) | (csr_read(CSR_STVAL) & 0x3);
> 
> Well, okay, but instead of not losing the bottom two bits you're now losing
> the top two ones.

Oh, right, I will add a cast ((uint64_t)csr_read(CSR_HTVAL) << 2) | ...

It will cover all the cases RV32 which has 34-bit guest address and it 
will be enough for RV64 where GPA is 59bit (the highest possible for Sv59).

> 
> Also the spec reads as if htval only _may_ hold the original address of the
> faulting access. What if htval ends up 0?

good point. then we have to emulate fault instruction and get an address 
from an instruction. I think that for now it will be enough just to 
support platforms which always write GPA to HTVAL.

If I understand correctly if htval is supported by platform then htval 
will be always filled for guest page fault. To verify if HTVAL is 
supported we could do:

'Unless it has reason to assume otherwise (such as a platform standard), 
software that writes a value to htval should read back from htval to 
confirm the stored value.'

And is it true because:
```
A value of zero in mtval signifies either that the feature is not 
supported, or an illegal zero instruction was fetched.
```
(yes, it is about mtval but I asssume that htval has the same behaviour').

Otherwise if it won't work then we can't distinguish if it is zero 
because h/w doesn't update htval or it zero because faulty GPA is zero.

So we could add this check under #ifdef CONFIG_DEBUG here and if HTVAL 
isn't supported then we can't work on this platform.

> 
> Further, nit: There's (once again) no real value in the 0x prefix, I don't
> think.

Sure I will drop then.

> 
>> +static int emulate_load(unsigned long fault_addr, unsigned long htinst)
>> +{
>> +    return -EOPNOTSUPP;
>> +}
>> +
>> +static int emulate_store(unsigned long fault_addr, unsigned long htinst)
>> +{
>> +    return -EOPNOTSUPP;
>> +}
>> +
>> +static void handle_guest_page_fault(unsigned long cause,
>> +                                    struct cpu_user_regs *regs)
>> +{
>> +    unsigned long addr;
>> +    int rc;
>> +
>> +    addr = get_faulting_gpa();
> 
> Can't this become the initializer of the variable?

Sure, it can. I will do that.

Thanks.

~ Oleksii

Re: [PATCH v1 14/17] xen/riscv: add guest page fault handling stub

[Jan Beulich]

On 17.08.2026 18:10, Oleksii Kurochko wrote:
> On 8/12/26 5:48 PM, Jan Beulich wrote:
>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>> --- a/xen/arch/riscv/traps.c
>>> +++ b/xen/arch/riscv/traps.c
>>> @@ -191,6 +191,67 @@ static void timer_interrupt(void)
>>>       raise_softirq(TIMER_SOFTIRQ);
>>>   }
>>>   
>>> +static always_inline unsigned long get_faulting_gpa(void)
>>
>> May I suggest to use always_inline only when inlining is _functionally_
>> required?
> 
> Sure. But it ins't clear to me why it isn't a case here? Is it connected 
> to that function is static and too simple so a compiler will do by itself?

Counter question: What is it that would functionally break if the function
ended up not being inlined? (This is the question you generally need to
answer to justify use of always_inline. Of course there's the additional
case of performance being affected, but I don't view that as applicable
here; I'm open to be proven wrong, though.)

>>> +{
>>> +    /*
>>> +     * According to RISC-V spec:
>>> +     *  18.2.8. Hypervisor Trap Value Register (htval)
>>> +     *   ...
>>> +     *   A guest physical address written to htval is shifted right by 2 bits
>>> +     *   to accommodate addresses wider than the current XLEN.
>>> +     *   ...
>>> +     *   If the least-significant two bits of a faulting guest physical address
>>> +     *   are needed, these bits are ordinarily the same as the
>>> +     *   least-significant two bits of the faulting virtual address in stval.
>>> +     *   For faults due to implicit memory accesses for VS-stage address
>>> +     *   translation, the least-significant two bits are instead zeros. These
>>> +     *   cases can be distinguished using the value provided in register htinst.
>>> +     */
>>> +    return (csr_read(CSR_HTVAL) << 2) | (csr_read(CSR_STVAL) & 0x3);
>>
>> Well, okay, but instead of not losing the bottom two bits you're now losing
>> the top two ones.
> 
> Oh, right, I will add a cast ((uint64_t)csr_read(CSR_HTVAL) << 2) | ...
> 
> It will cover all the cases RV32 which has 34-bit guest address and it 
> will be enough for RV64 where GPA is 59bit (the highest possible for Sv59).

Only if the function return type then also changes.

>> Also the spec reads as if htval only _may_ hold the original address of the
>> faulting access. What if htval ends up 0?
> 
> good point. then we have to emulate fault instruction and get an address 
> from an instruction. I think that for now it will be enough just to 
> support platforms which always write GPA to HTVAL.
> 
> If I understand correctly if htval is supported by platform then htval 
> will be always filled for guest page fault. To verify if HTVAL is 
> supported we could do:
> 
> 'Unless it has reason to assume otherwise (such as a platform standard), 
> software that writes a value to htval should read back from htval to 
> confirm the stored value.'

How does this matter here? It's one thing for htval to be capable of
holding (all?) non-zero values, and another that it would always be
written. If the platform doesn't indicate the behavior, I fear you have
to assume that you may (perhaps even randomly) observe 0.

> And is it true because:
> ```
> A value of zero in mtval signifies either that the feature is not 
> supported, or an illegal zero instruction was fetched.
> ```
> (yes, it is about mtval but I asssume that htval has the same behaviour').

Right, but what you quote is specific to illegal instruction exceptions.

Jan

Re: [PATCH v1 14/17] xen/riscv: add guest page fault handling stub

[Oleksii Kurochko]

On 8/18/26 10:29 AM, Jan Beulich wrote:
> On 17.08.2026 18:10, Oleksii Kurochko wrote:
>> On 8/12/26 5:48 PM, Jan Beulich wrote:
>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>> --- a/xen/arch/riscv/traps.c
>>>> +++ b/xen/arch/riscv/traps.c
>>>> @@ -191,6 +191,67 @@ static void timer_interrupt(void)
>>>>        raise_softirq(TIMER_SOFTIRQ);
>>>>    }
>>>>    
>>>> +static always_inline unsigned long get_faulting_gpa(void)
>>>
>>> May I suggest to use always_inline only when inlining is _functionally_
>>> required?
>>
>> Sure. But it ins't clear to me why it isn't a case here? Is it connected
>> to that function is static and too simple so a compiler will do by itself?
> 
> Counter question: What is it that would functionally break if the function
> ended up not being inlined? (This is the question you generally need to
> answer to justify use of always_inline. Of course there's the additional
> case of performance being affected, but I don't view that as applicable
> here; I'm open to be proven wrong, though.)

Now it is clear how to identrify if function should be always_inline.

I put it only for the purpose to be sure that this function won't be 
called with prologue/epilogue but I agree that compiler will do that by 
itself.

> 
>>>> +{
>>>> +    /*
>>>> +     * According to RISC-V spec:
>>>> +     *  18.2.8. Hypervisor Trap Value Register (htval)
>>>> +     *   ...
>>>> +     *   A guest physical address written to htval is shifted right by 2 bits
>>>> +     *   to accommodate addresses wider than the current XLEN.
>>>> +     *   ...
>>>> +     *   If the least-significant two bits of a faulting guest physical address
>>>> +     *   are needed, these bits are ordinarily the same as the
>>>> +     *   least-significant two bits of the faulting virtual address in stval.
>>>> +     *   For faults due to implicit memory accesses for VS-stage address
>>>> +     *   translation, the least-significant two bits are instead zeros. These
>>>> +     *   cases can be distinguished using the value provided in register htinst.
>>>> +     */
>>>> +    return (csr_read(CSR_HTVAL) << 2) | (csr_read(CSR_STVAL) & 0x3);
>>>
>>> Well, okay, but instead of not losing the bottom two bits you're now losing
>>> the top two ones.
>>
>> Oh, right, I will add a cast ((uint64_t)csr_read(CSR_HTVAL) << 2) | ...
>>
>> It will cover all the cases RV32 which has 34-bit guest address and it
>> will be enough for RV64 where GPA is 59bit (the highest possible for Sv59).
> 
> Only if the function return type then also changes.
> 
>>> Also the spec reads as if htval only _may_ hold the original address of the
>>> faulting access. What if htval ends up 0?
>>
>> good point. then we have to emulate fault instruction and get an address
>> from an instruction. I think that for now it will be enough just to
>> support platforms which always write GPA to HTVAL.
>>
>> If I understand correctly if htval is supported by platform then htval
>> will be always filled for guest page fault. To verify if HTVAL is
>> supported we could do:
>>
>> 'Unless it has reason to assume otherwise (such as a platform standard),
>> software that writes a value to htval should read back from htval to
>> confirm the stored value.'
> 
> How does this matter here? It's one thing for htval to be capable of
> holding (all?) non-zero values, and another that it would always be
> written. If the platform doesn't indicate the behavior, I fear you have
> to assume that you may (perhaps even randomly) observe 0.

So to be very sure we could check for two extensions: Sstval and Shtval.
They will guarantee that under any circumstances it will be filled.

Also, as an option we could check that htinst value isn't zero as 
according to the spec:

For guest-page faults, the trap instruction register is written with a 
special pseudoinstruction value if:
(a) the fault is caused by an implicit memory access for VS-stage 
address translation, and (b) a nonzero
value (the faulting guest physical address) is written to mtval2 or htval.

So if htinst != 0 then htval is filled with GPA and a nonzero guest 
physical address written to mtval2/htval shall correspond to the exact 
virtual address written to mtval/stval.

But if htinst is 0 then we have to do VS-stage software pagewalk to get 
GPA and also we will need to decode instruction to get GVA.

I am thinking if it will be okay for now to cover the case htinst != 0 
and have BUG_ON(!htinst) to not miss that the possible future case when 
VS-stage s/w page walks and parsing of GVA from an instruction are 
needed. I think it is fine as all real boards on which I was able to 
test Xen has htval and stval properly filled and of course QEMU code 
guarantees that htval and stval will be properly filled in the case of QEMU.
Also, KVM is based also only htval and stval and I assume that they 
tested it on real hardware too so it seems like it is okay to go with 
solution that for now we are using htval + stval to get faulty address.

> 
>> And is it true because:
>> ```
>> A value of zero in mtval signifies either that the feature is not
>> supported, or an illegal zero instruction was fetched.
>> ```
>> (yes, it is about mtval but I asssume that htval has the same behaviour').
> 
> Right, but what you quote is specific to illegal instruction exceptions.

Oh, right.

~ Oleksii

Re: [PATCH v1 14/17] xen/riscv: add guest page fault handling stub

[Oleksii Kurochko]

On 8/18/26 6:04 PM, Oleksii Kurochko wrote:
> 
> 
> On 8/18/26 10:29 AM, Jan Beulich wrote:
>> On 17.08.2026 18:10, Oleksii Kurochko wrote:
>>> On 8/12/26 5:48 PM, Jan Beulich wrote:
>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>> --- a/xen/arch/riscv/traps.c
>>>>> +++ b/xen/arch/riscv/traps.c
>>>>> @@ -191,6 +191,67 @@ static void timer_interrupt(void)
>>>>>        raise_softirq(TIMER_SOFTIRQ);
>>>>>    }
>>>>> +static always_inline unsigned long get_faulting_gpa(void)
>>>>
>>>> May I suggest to use always_inline only when inlining is _functionally_
>>>> required?
>>>
>>> Sure. But it ins't clear to me why it isn't a case here? Is it connected
>>> to that function is static and too simple so a compiler will do by 
>>> itself?
>>
>> Counter question: What is it that would functionally break if the 
>> function
>> ended up not being inlined? (This is the question you generally need to
>> answer to justify use of always_inline. Of course there's the additional
>> case of performance being affected, but I don't view that as applicable
>> here; I'm open to be proven wrong, though.)
> 
> Now it is clear how to identrify if function should be always_inline.
> 
> I put it only for the purpose to be sure that this function won't be 
> called with prologue/epilogue but I agree that compiler will do that by 
> itself.
> 
>>
>>>>> +{
>>>>> +    /*
>>>>> +     * According to RISC-V spec:
>>>>> +     *  18.2.8. Hypervisor Trap Value Register (htval)
>>>>> +     *   ...
>>>>> +     *   A guest physical address written to htval is shifted 
>>>>> right by 2 bits
>>>>> +     *   to accommodate addresses wider than the current XLEN.
>>>>> +     *   ...
>>>>> +     *   If the least-significant two bits of a faulting guest 
>>>>> physical address
>>>>> +     *   are needed, these bits are ordinarily the same as the
>>>>> +     *   least-significant two bits of the faulting virtual 
>>>>> address in stval.
>>>>> +     *   For faults due to implicit memory accesses for VS-stage 
>>>>> address
>>>>> +     *   translation, the least-significant two bits are instead 
>>>>> zeros. These
>>>>> +     *   cases can be distinguished using the value provided in 
>>>>> register htinst.
>>>>> +     */
>>>>> +    return (csr_read(CSR_HTVAL) << 2) | (csr_read(CSR_STVAL) & 0x3);
>>>>
>>>> Well, okay, but instead of not losing the bottom two bits you're now 
>>>> losing
>>>> the top two ones.
>>>
>>> Oh, right, I will add a cast ((uint64_t)csr_read(CSR_HTVAL) << 2) | ...
>>>
>>> It will cover all the cases RV32 which has 34-bit guest address and it
>>> will be enough for RV64 where GPA is 59bit (the highest possible for 
>>> Sv59).
>>
>> Only if the function return type then also changes.
>>
>>>> Also the spec reads as if htval only _may_ hold the original address 
>>>> of the
>>>> faulting access. What if htval ends up 0?
>>>
>>> good point. then we have to emulate fault instruction and get an address
>>> from an instruction. I think that for now it will be enough just to
>>> support platforms which always write GPA to HTVAL.
>>>
>>> If I understand correctly if htval is supported by platform then htval
>>> will be always filled for guest page fault. To verify if HTVAL is
>>> supported we could do:
>>>
>>> 'Unless it has reason to assume otherwise (such as a platform standard),
>>> software that writes a value to htval should read back from htval to
>>> confirm the stored value.'
>>
>> How does this matter here? It's one thing for htval to be capable of
>> holding (all?) non-zero values, and another that it would always be
>> written. If the platform doesn't indicate the behavior, I fear you have
>> to assume that you may (perhaps even randomly) observe 0.
> 
> So to be very sure we could check for two extensions: Sstval and Shtval.
> They will guarantee that under any circumstances it will be filled.
> 
> Also, as an option we could check that htinst value isn't zero as 
> according to the spec:
> 
> For guest-page faults, the trap instruction register is written with a 
> special pseudoinstruction value if:
> (a) the fault is caused by an implicit memory access for VS-stage 
> address translation, and (b) a nonzero
> value (the faulting guest physical address) is written to mtval2 or htval.
> 
> So if htinst != 0 then htval is filled with GPA and a nonzero guest 
> physical address written to mtval2/htval shall correspond to the exact 
> virtual address written to mtval/stval.

I've re-read SPEC again and it looks like htinst != 0 doesn't guarantee 
that htval and stval will contain necessary for me here faulty 
instruction. What I wrote above guarantee that if a fault during 
VS-stage translation failed then it htval will contatain GPA of PTE with 
which was an issue.

So I have to blindly believe that HTVAL and STVAL will always contain 
necessary for me data as KVM and other hypervisor does or introduce here 
software guest page table walker if Sstvala and Shtvala aren't provided 
by platform.

~ Oleksii

> 
> But if htinst is 0 then we have to do VS-stage software pagewalk to get 
> GPA and also we will need to decode instruction to get GVA.
> 
> I am thinking if it will be okay for now to cover the case htinst != 0 
> and have BUG_ON(!htinst) to not miss that the possible future case when 
> VS-stage s/w page walks and parsing of GVA from an instruction are 
> needed. I think it is fine as all real boards on which I was able to 
> test Xen has htval and stval properly filled and of course QEMU code 
> guarantees that htval and stval will be properly filled in the case of 
> QEMU.
> Also, KVM is based also only htval and stval and I assume that they 
> tested it on real hardware too so it seems like it is okay to go with 
> solution that for now we are using htval + stval to get faulty address.
> 
>>
>>> And is it true because:
>>> ```
>>> A value of zero in mtval signifies either that the feature is not
>>> supported, or an illegal zero instruction was fetched.
>>> ```
>>> (yes, it is about mtval but I asssume that htval has the same 
>>> behaviour').
>>
>> Right, but what you quote is specific to illegal instruction exceptions.
> 
> Oh, right.
> 
> ~ Oleksii

Re: [PATCH v1 14/17] xen/riscv: add guest page fault handling stub

[Jan Beulich]

On 19.08.2026 10:59, Oleksii Kurochko wrote:
> On 8/18/26 6:04 PM, Oleksii Kurochko wrote:
>> On 8/18/26 10:29 AM, Jan Beulich wrote:
>>> On 17.08.2026 18:10, Oleksii Kurochko wrote:
>>>> On 8/12/26 5:48 PM, Jan Beulich wrote:
>>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>>> +{
>>>>>> +    /*
>>>>>> +     * According to RISC-V spec:
>>>>>> +     *  18.2.8. Hypervisor Trap Value Register (htval)
>>>>>> +     *   ...
>>>>>> +     *   A guest physical address written to htval is shifted 
>>>>>> right by 2 bits
>>>>>> +     *   to accommodate addresses wider than the current XLEN.
>>>>>> +     *   ...
>>>>>> +     *   If the least-significant two bits of a faulting guest 
>>>>>> physical address
>>>>>> +     *   are needed, these bits are ordinarily the same as the
>>>>>> +     *   least-significant two bits of the faulting virtual 
>>>>>> address in stval.
>>>>>> +     *   For faults due to implicit memory accesses for VS-stage 
>>>>>> address
>>>>>> +     *   translation, the least-significant two bits are instead 
>>>>>> zeros. These
>>>>>> +     *   cases can be distinguished using the value provided in 
>>>>>> register htinst.
>>>>>> +     */
>>>>>> +    return (csr_read(CSR_HTVAL) << 2) | (csr_read(CSR_STVAL) & 0x3);
>>>>>
>>>>> Well, okay, but instead of not losing the bottom two bits you're now 
>>>>> losing
>>>>> the top two ones.
>>>>
>>>> Oh, right, I will add a cast ((uint64_t)csr_read(CSR_HTVAL) << 2) | ...
>>>>
>>>> It will cover all the cases RV32 which has 34-bit guest address and it
>>>> will be enough for RV64 where GPA is 59bit (the highest possible for 
>>>> Sv59).
>>>
>>> Only if the function return type then also changes.
>>>
>>>>> Also the spec reads as if htval only _may_ hold the original address 
>>>>> of the
>>>>> faulting access. What if htval ends up 0?
>>>>
>>>> good point. then we have to emulate fault instruction and get an address
>>>> from an instruction. I think that for now it will be enough just to
>>>> support platforms which always write GPA to HTVAL.
>>>>
>>>> If I understand correctly if htval is supported by platform then htval
>>>> will be always filled for guest page fault. To verify if HTVAL is
>>>> supported we could do:
>>>>
>>>> 'Unless it has reason to assume otherwise (such as a platform standard),
>>>> software that writes a value to htval should read back from htval to
>>>> confirm the stored value.'
>>>
>>> How does this matter here? It's one thing for htval to be capable of
>>> holding (all?) non-zero values, and another that it would always be
>>> written. If the platform doesn't indicate the behavior, I fear you have
>>> to assume that you may (perhaps even randomly) observe 0.
>>
>> So to be very sure we could check for two extensions: Sstval and Shtval.
>> They will guarantee that under any circumstances it will be filled.
>>
>> Also, as an option we could check that htinst value isn't zero as 
>> according to the spec:
>>
>> For guest-page faults, the trap instruction register is written with a 
>> special pseudoinstruction value if:
>> (a) the fault is caused by an implicit memory access for VS-stage 
>> address translation, and (b) a nonzero
>> value (the faulting guest physical address) is written to mtval2 or htval.
>>
>> So if htinst != 0 then htval is filled with GPA and a nonzero guest 
>> physical address written to mtval2/htval shall correspond to the exact 
>> virtual address written to mtval/stval.
> 
> I've re-read SPEC again and it looks like htinst != 0 doesn't guarantee 
> that htval and stval will contain necessary for me here faulty 
> instruction. What I wrote above guarantee that if a fault during 
> VS-stage translation failed then it htval will contatain GPA of PTE with 
> which was an issue.
> 
> So I have to blindly believe that HTVAL and STVAL will always contain 
> necessary for me data as KVM and other hypervisor does or introduce here 
> software guest page table walker if Sstvala and Shtvala aren't provided 
> by platform.

Why "blindly believe"? Checking for the necessary extension(s) should be
an option. Adding fallback code for when an extension isn't available can
come later, can't it?

Jan

Re: [PATCH v1 14/17] xen/riscv: add guest page fault handling stub

[Oleksii Kurochko]

On 8/19/26 11:52 AM, Jan Beulich wrote:
> On 19.08.2026 10:59, Oleksii Kurochko wrote:
>> On 8/18/26 6:04 PM, Oleksii Kurochko wrote:
>>> On 8/18/26 10:29 AM, Jan Beulich wrote:
>>>> On 17.08.2026 18:10, Oleksii Kurochko wrote:
>>>>> On 8/12/26 5:48 PM, Jan Beulich wrote:
>>>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>>>> +{
>>>>>>> +    /*
>>>>>>> +     * According to RISC-V spec:
>>>>>>> +     *  18.2.8. Hypervisor Trap Value Register (htval)
>>>>>>> +     *   ...
>>>>>>> +     *   A guest physical address written to htval is shifted
>>>>>>> right by 2 bits
>>>>>>> +     *   to accommodate addresses wider than the current XLEN.
>>>>>>> +     *   ...
>>>>>>> +     *   If the least-significant two bits of a faulting guest
>>>>>>> physical address
>>>>>>> +     *   are needed, these bits are ordinarily the same as the
>>>>>>> +     *   least-significant two bits of the faulting virtual
>>>>>>> address in stval.
>>>>>>> +     *   For faults due to implicit memory accesses for VS-stage
>>>>>>> address
>>>>>>> +     *   translation, the least-significant two bits are instead
>>>>>>> zeros. These
>>>>>>> +     *   cases can be distinguished using the value provided in
>>>>>>> register htinst.
>>>>>>> +     */
>>>>>>> +    return (csr_read(CSR_HTVAL) << 2) | (csr_read(CSR_STVAL) & 0x3);
>>>>>>
>>>>>> Well, okay, but instead of not losing the bottom two bits you're now
>>>>>> losing
>>>>>> the top two ones.
>>>>>
>>>>> Oh, right, I will add a cast ((uint64_t)csr_read(CSR_HTVAL) << 2) | ...
>>>>>
>>>>> It will cover all the cases RV32 which has 34-bit guest address and it
>>>>> will be enough for RV64 where GPA is 59bit (the highest possible for
>>>>> Sv59).
>>>>
>>>> Only if the function return type then also changes.
>>>>
>>>>>> Also the spec reads as if htval only _may_ hold the original address
>>>>>> of the
>>>>>> faulting access. What if htval ends up 0?
>>>>>
>>>>> good point. then we have to emulate fault instruction and get an address
>>>>> from an instruction. I think that for now it will be enough just to
>>>>> support platforms which always write GPA to HTVAL.
>>>>>
>>>>> If I understand correctly if htval is supported by platform then htval
>>>>> will be always filled for guest page fault. To verify if HTVAL is
>>>>> supported we could do:
>>>>>
>>>>> 'Unless it has reason to assume otherwise (such as a platform standard),
>>>>> software that writes a value to htval should read back from htval to
>>>>> confirm the stored value.'
>>>>
>>>> How does this matter here? It's one thing for htval to be capable of
>>>> holding (all?) non-zero values, and another that it would always be
>>>> written. If the platform doesn't indicate the behavior, I fear you have
>>>> to assume that you may (perhaps even randomly) observe 0.
>>>
>>> So to be very sure we could check for two extensions: Sstval and Shtval.
>>> They will guarantee that under any circumstances it will be filled.
>>>
>>> Also, as an option we could check that htinst value isn't zero as
>>> according to the spec:
>>>
>>> For guest-page faults, the trap instruction register is written with a
>>> special pseudoinstruction value if:
>>> (a) the fault is caused by an implicit memory access for VS-stage
>>> address translation, and (b) a nonzero
>>> value (the faulting guest physical address) is written to mtval2 or htval.
>>>
>>> So if htinst != 0 then htval is filled with GPA and a nonzero guest
>>> physical address written to mtval2/htval shall correspond to the exact
>>> virtual address written to mtval/stval.
>>
>> I've re-read SPEC again and it looks like htinst != 0 doesn't guarantee
>> that htval and stval will contain necessary for me here faulty
>> instruction. What I wrote above guarantee that if a fault during
>> VS-stage translation failed then it htval will contatain GPA of PTE with
>> which was an issue.
>>
>> So I have to blindly believe that HTVAL and STVAL will always contain
>> necessary for me data as KVM and other hypervisor does or introduce here
>> software guest page table walker if Sstvala and Shtvala aren't provided
>> by platform.
> 
> Why "blindly believe"? Checking for the necessary extension(s) should be
> an option. Adding fallback code for when an extension isn't available can
> come later, can't it?

It can be an option. It is what I planned to do.

~ Oleksii

[PATCH v1 15/17] xen/riscv: implement trap redirection to a guest

[Oleksii Kurochko]

Some traps taken by Xen on behalf of a guest can't or shouldn't be
handled by the hypervisor and must be forwarded to the guest's own
S-mode exception handler instead: e.g. when riscv_vcpu_unpriv_read()
faults while accessing guest memory, or when emulation hits a condition
only the guest kernel can resolve.

Introduce riscv_vcpu_trap_redirect() for that purpose. It makes the
trap appear to the guest as if it had been taken directly in VS-mode:
the trap information is transferred to the guest's virtual supervisor
CSRs and the vCPU is resumed at its exception vector in supervisor
mode, following the trap entry rules of the RISC-V privileged
specification.

The implementation is based on kvm_riscv_vcpu_trap_redirect() from
Linux, with a few deviations:
 - The function reads and writes physical VS-mode CSRs, so it is only
   meaningful for the currently running vCPU. Instead of taking a
   struct vcpu argument, it always operates on current.
 - The MODE field of vstvec is masked off explicitly when computing the
   exception target PC (exceptions always vector to BASE), rather than
   relying on the hardwired zero bit of sepc to drop it on VM entry.
 - Assertions document the preconditions: the trap must have been taken
   from virtualized mode (hstatus.SPV set), and only synchronous
   exceptions may be redirected - interrupts must be injected via hvip
   instead, so that the hardware performs VS-mode trap entry itself,
   respecting vsstatus.SIE and vectored vstvec dispatch.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 xen/arch/riscv/guestcopy.c                | 54 +++++++++++++++++++++++
 xen/arch/riscv/include/asm/guest_access.h |  2 +
 2 files changed, 56 insertions(+)

diff --git a/xen/arch/riscv/guestcopy.c b/xen/arch/riscv/guestcopy.c
index 57844bc442f6..6dc7a2e3a7ae 100644
--- a/xen/arch/riscv/guestcopy.c
+++ b/xen/arch/riscv/guestcopy.c
@@ -205,3 +205,57 @@ unsigned long riscv_vcpu_unpriv_read(bool read_insn,
 
     return val;
 }
+
+/* Redirect trap to Guest. */
+void riscv_vcpu_trap_redirect(const struct trap_info *trap)
+{
+    struct cpu_user_regs *regs = vcpu_guest_cpu_user_regs(current);
+    unsigned long vsstatus = csr_read(CSR_VSSTATUS);
+
+    /*
+     * Redirecting a trap makes sense only if the trap was taken from
+     * virtualized mode, i.e. sret is going to return to VS-mode.
+     */
+    ASSERT(regs->hstatus & HSTATUS_SPV);
+
+    /*
+     * Only synchronous exceptions can be redirected. Interrupts must be
+     * injected via hvip instead, so that the hardware itself performs
+     * VS-mode trap entry, respecting vsstatus.SIE and the vectored
+     * dispatch (BASE + 4 * cause) if vstvec is configured so.
+     */
+    ASSERT(!(trap->scause & CAUSE_IRQ_FLAG));
+
+    /* Change Guest SSTATUS.SPP bit */
+    vsstatus &= ~SSTATUS_SPP;
+    if ( regs->sstatus & SSTATUS_SPP )
+        vsstatus |= SSTATUS_SPP;
+
+    /* Change Guest SSTATUS.SPIE bit */
+    vsstatus &= ~SSTATUS_SPIE;
+    if ( vsstatus & SSTATUS_SIE )
+        vsstatus |= SSTATUS_SPIE;
+
+    /* Clear Guest SSTATUS.SIE bit */
+    vsstatus &= ~SSTATUS_SIE;
+
+    /* Update Guest SSTATUS */
+    csr_write(CSR_VSSTATUS, vsstatus);
+
+    /* Update Guest SCAUSE, STVAL, and SEPC */
+    csr_write(CSR_VSCAUSE, trap->scause);
+    csr_write(CSR_VSTVAL, trap->stval);
+    csr_write(CSR_VSEPC, trap->sepc);
+
+    /*
+     * Set Guest PC to Guest exception vector.
+     *
+     * vstvec[1:0] is the vector MODE, not part of the address. Exceptions
+     * always target BASE regardless of MODE, so mask it off explicitly
+     * instead of relying on the hardwired zero bit of sepc to drop it.
+     */
+    regs->sepc = csr_read(CSR_VSTVEC) & ~0x3UL;
+
+    /* Set Guest privilege mode to supervisor */
+    regs->sstatus |= SSTATUS_SPP;
+}
diff --git a/xen/arch/riscv/include/asm/guest_access.h b/xen/arch/riscv/include/asm/guest_access.h
index 153ec6ce26d1..d96cbc833d27 100644
--- a/xen/arch/riscv/include/asm/guest_access.h
+++ b/xen/arch/riscv/include/asm/guest_access.h
@@ -31,6 +31,8 @@ unsigned long riscv_vcpu_unpriv_read(bool read_insn,
                                      unsigned long guest_addr,
                                      struct trap_info *trap);
 
+void riscv_vcpu_trap_redirect(const struct trap_info *trap);
+
 #endif /* ASM__RISCV__GUEST_ACCESS_H */
 /*
  * Local variables:
-- 
2.54.0

Re: [PATCH v1 15/17] xen/riscv: implement trap redirection to a guest

[Jan Beulich]

On 20.07.2026 18:02, Oleksii Kurochko wrote:
> Some traps taken by Xen on behalf of a guest can't or shouldn't be
> handled by the hypervisor and must be forwarded to the guest's own
> S-mode exception handler instead: e.g. when riscv_vcpu_unpriv_read()
> faults while accessing guest memory, or when emulation hits a condition
> only the guest kernel can resolve.

Is the plan to use riscv_vcpu_unpriv_read() also for reading hypercall
buffers? In that case trap redirection shouldn't come into play.

> Introduce riscv_vcpu_trap_redirect() for that purpose. It makes the
> trap appear to the guest as if it had been taken directly in VS-mode:
> the trap information is transferred to the guest's virtual supervisor
> CSRs and the vCPU is resumed at its exception vector in supervisor
> mode, following the trap entry rules of the RISC-V privileged
> specification.
> 
> The implementation is based on kvm_riscv_vcpu_trap_redirect() from
> Linux, with a few deviations:
>  - The function reads and writes physical VS-mode CSRs, so it is only
>    meaningful for the currently running vCPU. Instead of taking a
>    struct vcpu argument, it always operates on current.
>  - The MODE field of vstvec is masked off explicitly when computing the
>    exception target PC (exceptions always vector to BASE), rather than
>    relying on the hardwired zero bit of sepc to drop it on VM entry.
>  - Assertions document the preconditions: the trap must have been taken
>    from virtualized mode (hstatus.SPV set), and only synchronous
>    exceptions may be redirected - interrupts must be injected via hvip
>    instead, so that the hardware performs VS-mode trap entry itself,
>    respecting vsstatus.SIE and vectored vstvec dispatch.

For this last bullet point - how is a reviewer supposed to validate the
assertions added when no caller of the new function exists?

> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
>  xen/arch/riscv/guestcopy.c                | 54 +++++++++++++++++++++++
>  xen/arch/riscv/include/asm/guest_access.h |  2 +
>  2 files changed, 56 insertions(+)

I don't understand this placement - trap redirection has nothing
(directly) to do with accessing guest memory.

> --- a/xen/arch/riscv/guestcopy.c
> +++ b/xen/arch/riscv/guestcopy.c
> @@ -205,3 +205,57 @@ unsigned long riscv_vcpu_unpriv_read(bool read_insn,
>  
>      return val;
>  }
> +
> +/* Redirect trap to Guest. */
> +void riscv_vcpu_trap_redirect(const struct trap_info *trap)
> +{
> +    struct cpu_user_regs *regs = vcpu_guest_cpu_user_regs(current);
> +    unsigned long vsstatus = csr_read(CSR_VSSTATUS);
> +
> +    /*
> +     * Redirecting a trap makes sense only if the trap was taken from
> +     * virtualized mode, i.e. sret is going to return to VS-mode.
> +     */
> +    ASSERT(regs->hstatus & HSTATUS_SPV);
> +
> +    /*
> +     * Only synchronous exceptions can be redirected. Interrupts must be
> +     * injected via hvip instead, so that the hardware itself performs
> +     * VS-mode trap entry, respecting vsstatus.SIE and the vectored
> +     * dispatch (BASE + 4 * cause) if vstvec is configured so.
> +     */
> +    ASSERT(!(trap->scause & CAUSE_IRQ_FLAG));
> +
> +    /* Change Guest SSTATUS.SPP bit */
> +    vsstatus &= ~SSTATUS_SPP;
> +    if ( regs->sstatus & SSTATUS_SPP )
> +        vsstatus |= SSTATUS_SPP;
> +
> +    /* Change Guest SSTATUS.SPIE bit */
> +    vsstatus &= ~SSTATUS_SPIE;
> +    if ( vsstatus & SSTATUS_SIE )
> +        vsstatus |= SSTATUS_SPIE;
> +
> +    /* Clear Guest SSTATUS.SIE bit */
> +    vsstatus &= ~SSTATUS_SIE;
> +
> +    /* Update Guest SSTATUS */
> +    csr_write(CSR_VSSTATUS, vsstatus);
> +
> +    /* Update Guest SCAUSE, STVAL, and SEPC */
> +    csr_write(CSR_VSCAUSE, trap->scause);
> +    csr_write(CSR_VSTVAL, trap->stval);
> +    csr_write(CSR_VSEPC, trap->sepc);
> +
> +    /*
> +     * Set Guest PC to Guest exception vector.
> +     *
> +     * vstvec[1:0] is the vector MODE, not part of the address. Exceptions
> +     * always target BASE regardless of MODE, so mask it off explicitly
> +     * instead of relying on the hardwired zero bit of sepc to drop it.
> +     */
> +    regs->sepc = csr_read(CSR_VSTVEC) & ~0x3UL;

Can there be a proper constant please for this mask?

Jan

Re: [PATCH v1 15/17] xen/riscv: implement trap redirection to a guest

[Oleksii Kurochko]

On 8/12/26 6:03 PM, Jan Beulich wrote:
> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>> Some traps taken by Xen on behalf of a guest can't or shouldn't be
>> handled by the hypervisor and must be forwarded to the guest's own
>> S-mode exception handler instead: e.g. when riscv_vcpu_unpriv_read()
>> faults while accessing guest memory, or when emulation hits a condition
>> only the guest kernel can resolve.
> 
> Is the plan to use riscv_vcpu_unpriv_read() also for reading hypercall
> buffers?

Yes, it could also be used to read hypercall buffers, but I don't think 
it's the best option, as hypercall buffers could be larger than 8 bytes 
(which is the size supported by the `hlv` instruction on the RV64 
platform). For that case, I think it would be better to map the Xen page 
corresponding to the GVA of the hypercall buffer and then use the usual 
memcpy(). So, basically, use copy_guest() on RISC-V for that purpose.


> In that case trap redirection shouldn't come into play.

It isn't mandatory to perform a redirection in the case of 
riscv_vcpu_unpriv_read(), so if trap redirection shouldn't happen for 
hypercall buffers, then the caller of riscv_vcpu_unpriv_read() needs to 
handle that properly by checking utrap.cause. Something like:

```
     *insn = riscv_vcpu_unpriv_read(true, regs->sepc, &utrap);
     if ( utrap.scause )
     {
         ...
         utrap.sepc = regs->sepc;
         utrap.stval = utrap.sepc;

         riscv_vcpu_trap_redirect(&utrap);

         return true;
     }
```

So, if this cannot happen in the case of a hypercall buffer, then we 
need to return -EFAULT in the if ( utrap.scause ) case.

I don't think I understand why redirection shouldn't come into play. Do 
you mean that the hypercall buffer will always be available, and that it 
is impossible for the hlv instruction to fail, so there is no point in 
handling redirection at all in this case?

> 
>> Introduce riscv_vcpu_trap_redirect() for that purpose. It makes the
>> trap appear to the guest as if it had been taken directly in VS-mode:
>> the trap information is transferred to the guest's virtual supervisor
>> CSRs and the vCPU is resumed at its exception vector in supervisor
>> mode, following the trap entry rules of the RISC-V privileged
>> specification.
>>
>> The implementation is based on kvm_riscv_vcpu_trap_redirect() from
>> Linux, with a few deviations:
>>   - The function reads and writes physical VS-mode CSRs, so it is only
>>     meaningful for the currently running vCPU. Instead of taking a
>>     struct vcpu argument, it always operates on current.
>>   - The MODE field of vstvec is masked off explicitly when computing the
>>     exception target PC (exceptions always vector to BASE), rather than
>>     relying on the hardwired zero bit of sepc to drop it on VM entry.
>>   - Assertions document the preconditions: the trap must have been taken
>>     from virtualized mode (hstatus.SPV set), and only synchronous
>>     exceptions may be redirected - interrupts must be injected via hvip
>>     instead, so that the hardware performs VS-mode trap entry itself,
>>     respecting vsstatus.SIE and vectored vstvec dispatch.
> 
> For this last bullet point - how is a reviewer supposed to validate the
> assertions added when no caller of the new function exists?

My bad (again). The caller appears in "[PATCH v1 16/17] xen/riscv: add 
guest load emulation for trapped MMIO accesses" (as you already know) so 
I have to re-order patches and put this patch after PATCH v1 16/17.

> 
>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>> ---
>>   xen/arch/riscv/guestcopy.c                | 54 +++++++++++++++++++++++
>>   xen/arch/riscv/include/asm/guest_access.h |  2 +
>>   2 files changed, 56 insertions(+)
> 
> I don't understand this placement - trap redirection has nothing
> (directly) to do with accessing guest memory.

Agree, at some point. I put trap redirection there as the idea was to 
catch trap from hlv{x} instructions and redirect some of them to guest 
so I put it to guestcopy.h.

I will put inside traps.{c,h} instead.

> 
>> --- a/xen/arch/riscv/guestcopy.c
>> +++ b/xen/arch/riscv/guestcopy.c
>> @@ -205,3 +205,57 @@ unsigned long riscv_vcpu_unpriv_read(bool read_insn,
>>   
>>       return val;
>>   }
>> +
>> +/* Redirect trap to Guest. */
>> +void riscv_vcpu_trap_redirect(const struct trap_info *trap)
>> +{
>> +    struct cpu_user_regs *regs = vcpu_guest_cpu_user_regs(current);
>> +    unsigned long vsstatus = csr_read(CSR_VSSTATUS);
>> +
>> +    /*
>> +     * Redirecting a trap makes sense only if the trap was taken from
>> +     * virtualized mode, i.e. sret is going to return to VS-mode.
>> +     */
>> +    ASSERT(regs->hstatus & HSTATUS_SPV);
>> +
>> +    /*
>> +     * Only synchronous exceptions can be redirected. Interrupts must be
>> +     * injected via hvip instead, so that the hardware itself performs
>> +     * VS-mode trap entry, respecting vsstatus.SIE and the vectored
>> +     * dispatch (BASE + 4 * cause) if vstvec is configured so.
>> +     */
>> +    ASSERT(!(trap->scause & CAUSE_IRQ_FLAG));
>> +
>> +    /* Change Guest SSTATUS.SPP bit */
>> +    vsstatus &= ~SSTATUS_SPP;
>> +    if ( regs->sstatus & SSTATUS_SPP )
>> +        vsstatus |= SSTATUS_SPP;
>> +
>> +    /* Change Guest SSTATUS.SPIE bit */
>> +    vsstatus &= ~SSTATUS_SPIE;
>> +    if ( vsstatus & SSTATUS_SIE )
>> +        vsstatus |= SSTATUS_SPIE;
>> +
>> +    /* Clear Guest SSTATUS.SIE bit */
>> +    vsstatus &= ~SSTATUS_SIE;
>> +
>> +    /* Update Guest SSTATUS */
>> +    csr_write(CSR_VSSTATUS, vsstatus);
>> +
>> +    /* Update Guest SCAUSE, STVAL, and SEPC */
>> +    csr_write(CSR_VSCAUSE, trap->scause);
>> +    csr_write(CSR_VSTVAL, trap->stval);
>> +    csr_write(CSR_VSEPC, trap->sepc);
>> +
>> +    /*
>> +     * Set Guest PC to Guest exception vector.
>> +     *
>> +     * vstvec[1:0] is the vector MODE, not part of the address. Exceptions
>> +     * always target BASE regardless of MODE, so mask it off explicitly
>> +     * instead of relying on the hardwired zero bit of sepc to drop it.
>> +     */
>> +    regs->sepc = csr_read(CSR_VSTVEC) & ~0x3UL;
> 
> Can there be a proper constant please for this mask?

Sure, I will introduce one.

Thanks.

~ Oleksii

Re: [PATCH v1 15/17] xen/riscv: implement trap redirection to a guest

[Jan Beulich]

On 18.08.2026 09:47, Oleksii Kurochko wrote:
> On 8/12/26 6:03 PM, Jan Beulich wrote:
>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>> Some traps taken by Xen on behalf of a guest can't or shouldn't be
>>> handled by the hypervisor and must be forwarded to the guest's own
>>> S-mode exception handler instead: e.g. when riscv_vcpu_unpriv_read()
>>> faults while accessing guest memory, or when emulation hits a condition
>>> only the guest kernel can resolve.
>>
>> Is the plan to use riscv_vcpu_unpriv_read() also for reading hypercall
>> buffers?
> 
> Yes, it could also be used to read hypercall buffers, but I don't think 
> it's the best option, as hypercall buffers could be larger than 8 bytes 
> (which is the size supported by the `hlv` instruction on the RV64 
> platform). For that case, I think it would be better to map the Xen page 
> corresponding to the GVA of the hypercall buffer and then use the usual 
> memcpy(). So, basically, use copy_guest() on RISC-V for that purpose.
> 
> 
>> In that case trap redirection shouldn't come into play.
> 
> It isn't mandatory to perform a redirection in the case of 
> riscv_vcpu_unpriv_read(), so if trap redirection shouldn't happen for 
> hypercall buffers, then the caller of riscv_vcpu_unpriv_read() needs to 
> handle that properly by checking utrap.cause. Something like:
> 
> ```
>      *insn = riscv_vcpu_unpriv_read(true, regs->sepc, &utrap);
>      if ( utrap.scause )
>      {
>          ...
>          utrap.sepc = regs->sepc;
>          utrap.stval = utrap.sepc;
> 
>          riscv_vcpu_trap_redirect(&utrap);
> 
>          return true;
>      }
> ```
> 
> So, if this cannot happen in the case of a hypercall buffer, then we 
> need to return -EFAULT in the if ( utrap.scause ) case.
> 
> I don't think I understand why redirection shouldn't come into play. Do 
> you mean that the hypercall buffer will always be available, and that it 
> is impossible for the hlv instruction to fail, so there is no point in 
> handling redirection at all in this case?

Failure to access a hypercall buffer should result in a -EFAULT return
value, not in any kind of exception.

Jan

Re: [PATCH v1 00/17] [RISC-V] virtual interrupt controller (vAPLIC/vIMSIC) support

[Jan Beulich]

On 20.07.2026 18:01, Oleksii Kurochko wrote:
> Hi all,
> 
> This series adds the initial virtual interrupt controller (vINTC) support
> for RISC-V guests in Xen, based on the Advanced Interrupt Architecture
> (AIA): a virtual APLIC (vAPLIC) in MSI mode backed by a virtual IMSIC
> (vIMSIC) using hardware guest interrupt files.
> 
> Rather than emulating APLIC in direct-delivery mode (which requires
> trap-and-emulate for every interrupt and is costly), the series targets
> IMSIC from the start. AIA lets a hart implement several "guest interrupt
> files" (up to GEILEN), so external interrupts can be delivered to a vCPU
> directly by hardware via the VGEIN field of hstatus, without a hypervisor
> round-trip. Xen only has to emulate the APLIC MMIO programming interface
> and route the guest's intent onto the physical MSI topology; interrupt
> delivery itself stays in hardware.
> 
> The work breaks down into a few logical blocks:
> 
> Physical APLIC/AIA groundwork (patches 1-3)
>   - Correctly track IRQ_DISABLED across runtime enable/disable and order
>     the ->status update against the IMSIC CSR write.
>   - Per-pCPU VGEIN (guest interrupt file) allocator: assign/release a
>     hardware guest file to a vCPU and program hstatus.VGEIN.
>   - Add the missing APLIC register offsets/masks needed by both the
>     physical and virtual APLIC code (no functional change).
> 
> Device-agnostic MMIO dispatch + vAPLIC emulation (patches 4-5)
>   - A per-domain MMIO handler table modelled on Arm's framework, so
>     emulated devices self-register their GPA ranges and the fault path
>     stays agnostic via a single try_handle_mmio() entry point.
>   - vAPLIC MMIO read/write emulation. Writes are gated by the domain's
>     authorised-IRQ bitmap so a guest cannot touch interrupts it does not
>     own, and TARGET writes are translated from virtual to physical
>     hart/guest-file indices. Delegation (SOURCECFG.D) is not yet
>     supported.
> 
> vIMSIC guest interrupt files and state (patches 6-9)
>   - Stage-2 map a vCPU's physical guest interrupt file to the fixed
>     per-vCPU GPA page the guest expects at offset 0.
>   - vcpu_aia_init(): assign a VGEIN, map its guest file and record the
>     IMSIC state as a consistent unit.
>   - IMSIC state save/restore (currently tracking which pCPU owns the
>     guest file, needed because the pCPU id is part of the MSI address).
>   - has_msi_support() helper to decide whether IMSIC state has to be
>     saved/restored.
> 
> vINTC state save/restore plumbing (patches 10-11)
>   - vintc_state_{save,restore}() wrappers over new store/restore hooks in
>     struct vintc_ops, and the vAPLIC implementation of those hooks. No
>     callers are wired up yet; the context-switch calls arrive with vCPU
>     context switch support.
> 
> Trap and instruction emulation infrastructure (patches 12-17)
>   - Extend the exception-table format with type/data fields and add
>     EX_TYPE_TRAP_INFO so fixups can capture sepc/scause/stval.
>   - riscv_vcpu_unpriv_read() (HLV/HLVX) to read guest memory/instructions
>     safely, and riscv_vcpu_trap_redirect() to forward a synchronous trap
>     back into the guest's VS-mode handler.
>   - A guest page-fault handler that decodes the trapped load/store
>     instruction (HTINST, or an unprivileged fetch as a fallback) and
>     dispatches the MMIO access through try_handle_mmio().
> 
> Note on versioning: the series is posted as v1 as a freshly split-out
> series, but several patches carry v2/v3 changelogs because they were
> previously circulated as part of a larger another patches [1] from which the
> current depends.
> 
> CI tests: https://gitlab.com/xen-project/people/olkur/xen/-/pipelines/2690874319
> 
> [1] https://lore.kernel.org/xen-devel/cover.1784559209.git.oleksii.kurochko@gmail.com/T/#t
> 
> Oleksii Kurochko (17):
>   xen/riscv: manage IRQ_DISABLED flag in APLIC irq enable/disable
>     callbacks
>   xen/riscv: add basic VGEIN management for AIA guests
>   xen/riscv: add missing APLIC register offsets, masks to asm/aplic.h
>   xen/riscv: introduce device-agnostic MMIO emulation dispatch
>   xen/riscv: implement virtual APLIC MMIO emulation
>   xen/riscv: map IMSIC interrupt file for vCPUs
>   xen/riscv: introduce vCPU AIA initialization
>   xen/riscv: add IMSIC state save/restore
>   xen/riscv: add helper to check APLIC MSI mode
>   xen/riscv: introduce vintc_state_{save,restore}()
>   xen/riscv: add vAPLIC state save/restore hooks
>   xen/riscv: extend exception tables with type and data fields
>   xen/riscv: add unprivileged guest memory read helper
>   xen/riscv: add guest page fault handling stub
>   xen/riscv: implement trap redirection to a guest
>   xen/riscv: add guest load emulation for trapped MMIO accesses
>   xen/riscv: add guest store emulation for trapped MMIO accesses

Where did the last two patches go? My inbox agrees with [1] that only 15 of the
17 patches arrived.

Jan

[1] https://lists.xen.org/archives/html/xen-devel/2026-07/threads.html

Re: [PATCH v1 00/17] [RISC-V] virtual interrupt controller (vAPLIC/vIMSIC) support

[Oleksii Kurochko]

On 7/27/26 5:21 PM, Jan Beulich wrote:
> On 20.07.2026 18:01, Oleksii Kurochko wrote:
>> Hi all,
>>
>> This series adds the initial virtual interrupt controller (vINTC) support
>> for RISC-V guests in Xen, based on the Advanced Interrupt Architecture
>> (AIA): a virtual APLIC (vAPLIC) in MSI mode backed by a virtual IMSIC
>> (vIMSIC) using hardware guest interrupt files.
>>
>> Rather than emulating APLIC in direct-delivery mode (which requires
>> trap-and-emulate for every interrupt and is costly), the series targets
>> IMSIC from the start. AIA lets a hart implement several "guest interrupt
>> files" (up to GEILEN), so external interrupts can be delivered to a vCPU
>> directly by hardware via the VGEIN field of hstatus, without a hypervisor
>> round-trip. Xen only has to emulate the APLIC MMIO programming interface
>> and route the guest's intent onto the physical MSI topology; interrupt
>> delivery itself stays in hardware.
>>
>> The work breaks down into a few logical blocks:
>>
>> Physical APLIC/AIA groundwork (patches 1-3)
>>    - Correctly track IRQ_DISABLED across runtime enable/disable and order
>>      the ->status update against the IMSIC CSR write.
>>    - Per-pCPU VGEIN (guest interrupt file) allocator: assign/release a
>>      hardware guest file to a vCPU and program hstatus.VGEIN.
>>    - Add the missing APLIC register offsets/masks needed by both the
>>      physical and virtual APLIC code (no functional change).
>>
>> Device-agnostic MMIO dispatch + vAPLIC emulation (patches 4-5)
>>    - A per-domain MMIO handler table modelled on Arm's framework, so
>>      emulated devices self-register their GPA ranges and the fault path
>>      stays agnostic via a single try_handle_mmio() entry point.
>>    - vAPLIC MMIO read/write emulation. Writes are gated by the domain's
>>      authorised-IRQ bitmap so a guest cannot touch interrupts it does not
>>      own, and TARGET writes are translated from virtual to physical
>>      hart/guest-file indices. Delegation (SOURCECFG.D) is not yet
>>      supported.
>>
>> vIMSIC guest interrupt files and state (patches 6-9)
>>    - Stage-2 map a vCPU's physical guest interrupt file to the fixed
>>      per-vCPU GPA page the guest expects at offset 0.
>>    - vcpu_aia_init(): assign a VGEIN, map its guest file and record the
>>      IMSIC state as a consistent unit.
>>    - IMSIC state save/restore (currently tracking which pCPU owns the
>>      guest file, needed because the pCPU id is part of the MSI address).
>>    - has_msi_support() helper to decide whether IMSIC state has to be
>>      saved/restored.
>>
>> vINTC state save/restore plumbing (patches 10-11)
>>    - vintc_state_{save,restore}() wrappers over new store/restore hooks in
>>      struct vintc_ops, and the vAPLIC implementation of those hooks. No
>>      callers are wired up yet; the context-switch calls arrive with vCPU
>>      context switch support.
>>
>> Trap and instruction emulation infrastructure (patches 12-17)
>>    - Extend the exception-table format with type/data fields and add
>>      EX_TYPE_TRAP_INFO so fixups can capture sepc/scause/stval.
>>    - riscv_vcpu_unpriv_read() (HLV/HLVX) to read guest memory/instructions
>>      safely, and riscv_vcpu_trap_redirect() to forward a synchronous trap
>>      back into the guest's VS-mode handler.
>>    - A guest page-fault handler that decodes the trapped load/store
>>      instruction (HTINST, or an unprivileged fetch as a fallback) and
>>      dispatches the MMIO access through try_handle_mmio().
>>
>> Note on versioning: the series is posted as v1 as a freshly split-out
>> series, but several patches carry v2/v3 changelogs because they were
>> previously circulated as part of a larger another patches [1] from which the
>> current depends.
>>
>> CI tests: https://gitlab.com/xen-project/people/olkur/xen/-/pipelines/2690874319
>>
>> [1] https://lore.kernel.org/xen-devel/cover.1784559209.git.oleksii.kurochko@gmail.com/T/#t
>>
>> Oleksii Kurochko (17):
>>    xen/riscv: manage IRQ_DISABLED flag in APLIC irq enable/disable
>>      callbacks
>>    xen/riscv: add basic VGEIN management for AIA guests
>>    xen/riscv: add missing APLIC register offsets, masks to asm/aplic.h
>>    xen/riscv: introduce device-agnostic MMIO emulation dispatch
>>    xen/riscv: implement virtual APLIC MMIO emulation
>>    xen/riscv: map IMSIC interrupt file for vCPUs
>>    xen/riscv: introduce vCPU AIA initialization
>>    xen/riscv: add IMSIC state save/restore
>>    xen/riscv: add helper to check APLIC MSI mode
>>    xen/riscv: introduce vintc_state_{save,restore}()
>>    xen/riscv: add vAPLIC state save/restore hooks
>>    xen/riscv: extend exception tables with type and data fields
>>    xen/riscv: add unprivileged guest memory read helper
>>    xen/riscv: add guest page fault handling stub
>>    xen/riscv: implement trap redirection to a guest
>>    xen/riscv: add guest load emulation for trapped MMIO accesses
>>    xen/riscv: add guest store emulation for trapped MMIO accesses
> 
> Where did the last two patches go? My inbox agrees with [1] that only 15 of the
> 17 patches arrived.
> 
They lost for some reason, I sent them separately again. Now it should 
be okay.
> [1] https://lists.xen.org/archives/html/xen-devel/2026-07/threads.html

[PATCH v1 16/17] xen/riscv: add guest load emulation for trapped MMIO accesses

[Oleksii Kurochko]

Introduce emulate_load() to decode and emulate guest load instructions
that fault due to MMIO accesses. This provides the basic infrastructure
required for MMIO emulation on RISC-V.

The instruction decode (decode_trapped_insn() and the mask/match chain
for standard and compressed load encodings) is adapted from Linux's KVM
RISC-V implementation. The completion path differs from KVM's,
since Xen dispatches MMIO synchronously to an in-hypervisor handler via
try_handle_mmio() and has no userspace exit/return step equivalent to
KVM's kvm_io_bus_read() / KVM_EXIT_MMIO / kvm_riscv_vcpu_mmio_return()
split.

A fault taken while re-reading the trapped instruction is handled
depending on the faulting translation stage:
 - A VS-stage fault is the guest's own fault (e.g. it modified its page
   tables from another vCPU) and, as in KVM, is redirected to the
   guest's trap vector, with the cause remapped to
   CAUSE_FETCH_PAGE_FAULT since HLVX reports execute-permission failures
   as load faults.
 - A G-stage fault would mean the P2M mapping of the instruction page
   disappeared after the instruction was fetched. KVM must handle this
   by resuming the guest and retrying, as Linux MM can invalidate
   G-stage mappings at any time. Xen does not remove P2M mappings of a
   running domain at the moment, so this case is asserted unreachable with
   BUG_ON(); it will need to be revisited once such removal is implemented.

When a guest load triggers a page fault, the trapped instruction is
decoded using HTINST or, if unavailable, fetched via unprivileged access.
At the moment only virtual interrupt controller (vINTC) traps are
expected to occur, since it is currently the only backend registered
with the MMIO handler dispatch, so in practice the load is emulated via
the vINTC backend and the guest register state is updated accordingly.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 xen/arch/riscv/include/asm/traps.h |   6 ++
 xen/arch/riscv/traps.c             | 162 ++++++++++++++++++++++++++++-
 2 files changed, 167 insertions(+), 1 deletion(-)

diff --git a/xen/arch/riscv/include/asm/traps.h b/xen/arch/riscv/include/asm/traps.h
index 8d4ab664bca9..937295c5c76c 100644
--- a/xen/arch/riscv/include/asm/traps.h
+++ b/xen/arch/riscv/include/asm/traps.h
@@ -4,6 +4,7 @@
 #define ASM__RISCV__TRAPS_H
 
 #include <asm/processor.h>
+#include <asm/riscv_encoding.h>
 
 #ifndef __ASSEMBLER__
 
@@ -13,6 +14,11 @@ struct trap_info {
     register_t stval;
 };
 
+static inline bool is_load_guest_page_fault(unsigned long scause)
+{
+    return (scause == CAUSE_LOAD_GUEST_PAGE_FAULT);
+}
+
 void do_trap(struct cpu_user_regs *cpu_regs);
 void handle_trap(void);
 void trap_init(void);
diff --git a/xen/arch/riscv/traps.c b/xen/arch/riscv/traps.c
index 1c97bd101948..12690ae37aba 100644
--- a/xen/arch/riscv/traps.c
+++ b/xen/arch/riscv/traps.c
@@ -14,7 +14,9 @@
 
 #include <asm/extable.h>
 #include <asm/cpufeature.h>
+#include <asm/guest_access.h>
 #include <asm/intc.h>
+#include <asm/mmio.h>
 #include <asm/processor.h>
 #include <asm/riscv_encoding.h>
 #include <asm/traps.h>
@@ -191,6 +193,11 @@ static void timer_interrupt(void)
     raise_softirq(TIMER_SOFTIRQ);
 }
 
+static always_inline void advance_pc(struct cpu_user_regs *regs, int step)
+{
+    regs->sepc += step;
+}
+
 static always_inline unsigned long get_faulting_gpa(void)
 {
     /*
@@ -210,9 +217,162 @@ static always_inline unsigned long get_faulting_gpa(void)
     return (csr_read(CSR_HTVAL) << 2) | (csr_read(CSR_STVAL) & 0x3);
 }
 
+/*
+ * Determine the trapped instruction which caused a guest MMIO trap.
+ *
+ * Returns true if the trap was redirected to the guest, in which case
+ * the caller must stop emulation and return success. Otherwise *insn
+ * and *insn_len are filled in and the caller should continue decoding.
+ */
+static bool decode_trapped_insn(unsigned long htinst, unsigned long *insn,
+                                unsigned int *insn_len)
+{
+    if ( htinst & 0x1 )
+    {
+        /*
+         * Bit[0] == 1 implies trapped instruction value is
+         * transformed instruction or custom instruction.
+         */
+        *insn = htinst | INSN_16BIT_MASK;
+        *insn_len = (htinst & BIT(1, UL)) ? INSN_LEN(*insn) : 2;
+    }
+    else
+    {
+        struct cpu_user_regs *regs = vcpu_guest_cpu_user_regs(current);
+        struct trap_info utrap = { 0 };
+
+        /*
+         * Bit[0] == 0 implies trapped instruction value is
+         * zero or special value.
+         */
+        *insn = riscv_vcpu_unpriv_read(true, regs->sepc, &utrap);
+        if ( utrap.scause )
+        {
+            /*
+             * A G-stage fault here would mean the P2M mapping of the page
+             * containing the trapped instruction disappeared after it was
+             * fetched. Nothing removes P2M mappings of a running domain yet,
+             * so this cannot happen.
+             *
+             * TODO: Revisit once P2M mappings can be removed at runtime.
+             */
+            BUG_ON(is_load_guest_page_fault(utrap.scause));
+
+            utrap.sepc = regs->sepc;
+            utrap.stval = utrap.sepc;
+
+            riscv_vcpu_trap_redirect(&utrap);
+
+            return true;
+        }
+
+        *insn_len = INSN_LEN(*insn);
+    }
+
+    return false;
+}
+
+/*
+ * Check alignment and dispatch a decoded MMIO access to a registered
+ * handler. On success (0), info->data holds the read value for loads.
+ */
+static int do_mmio(mmio_info_t *info, unsigned long fault_addr,
+                   unsigned int len)
+{
+    /* Fault address should be aligned to length of MMIO */
+    if ( fault_addr & (len - 1) )
+        return -EIO;
+
+    info->gpa = fault_addr;
+    info->len = len;
+
+    switch ( try_handle_mmio(info) )
+    {
+    case IO_HANDLED:
+        return 0;
+    case IO_ABORT:
+        return -EIO;
+    default:
+        return -EOPNOTSUPP;
+    }
+}
+
 static int emulate_load(unsigned long fault_addr, unsigned long htinst)
 {
-    return -EOPNOTSUPP;
+    struct cpu_user_regs *regs = vcpu_guest_cpu_user_regs(current);
+    mmio_info_t info = { .is_write = false };
+    unsigned long insn;
+    unsigned int shift = 0, len, insn_len;
+    bool is_unsigned = false;
+    int rc;
+
+    if ( decode_trapped_insn(htinst, &insn, &insn_len) )
+        return 0;
+
+    /* Decode length of MMIO and whether it is a sign- or zero-extending load */
+    if ( (insn & INSN_MASK_LB) == INSN_MATCH_LB )
+        len = 1;
+    else if ( (insn & INSN_MASK_LBU) == INSN_MATCH_LBU )
+    {
+        len = 1;
+        is_unsigned = true;
+    }
+    else if ( (insn & INSN_MASK_LH) == INSN_MATCH_LH )
+        len = 2;
+    else if ( (insn & INSN_MASK_LHU) == INSN_MATCH_LHU )
+    {
+        len = 2;
+        is_unsigned = true;
+    }
+    else if ( (insn & INSN_MASK_LW) == INSN_MATCH_LW )
+        len = 4;
+#ifndef CONFIG_RISCV_32
+    else if ( (insn & INSN_MASK_LWU) == INSN_MATCH_LWU )
+    {
+        len = 4;
+        is_unsigned = true;
+    }
+#endif
+    else if ( (insn & INSN_MASK_C_LW) == INSN_MATCH_C_LW )
+    {
+        len = 4;
+        insn = RVC_RS2S(insn) << SH_RD;
+    }
+    else if ( (insn & INSN_MASK_C_LWSP) == INSN_MATCH_C_LWSP &&
+              RV_X(insn, SH_RD, 5) )
+        len = 4;
+#ifndef CONFIG_RISCV_32
+    else if ( (insn & INSN_MASK_LD) == INSN_MATCH_LD )
+        len = 8;
+    else if ( (insn & INSN_MASK_C_LD) == INSN_MATCH_C_LD )
+    {
+        len = 8;
+        insn = RVC_RS2S(insn) << SH_RD;
+    }
+    else if ( (insn & INSN_MASK_C_LDSP) == INSN_MATCH_C_LDSP &&
+              RV_X(insn, SH_RD, 5) )
+        len = 8;
+#endif
+    else
+        return -EOPNOTSUPP;
+
+    if ( !is_unsigned )
+        shift = BITS_PER_BYTE * (sizeof(unsigned long) - len);
+
+#ifdef EMULATE_LOAD_DEBUG
+    gdprintk(XENLOG_DEBUG, "pc=%#02lx, addr=%#02lx, len=%d, shift=%d\n",
+             regs->sepc, fault_addr, len, shift);
+#endif
+
+    rc = do_mmio(&info, fault_addr, len);
+    if ( rc )
+        return rc;
+
+    SET_RD(insn, regs, (long)((unsigned long)info.data << shift) >> shift);
+
+    advance_pc(regs, insn_len);
+
+    return 0;
 }
 
 static int emulate_store(unsigned long fault_addr, unsigned long htinst)
-- 
2.54.0

Re: [PATCH v1 16/17] xen/riscv: add guest load emulation for trapped MMIO accesses

[Jan Beulich]

On 29.07.2026 15:40, Oleksii Kurochko wrote:
> Introduce emulate_load() to decode and emulate guest load instructions
> that fault due to MMIO accesses. This provides the basic infrastructure
> required for MMIO emulation on RISC-V.
> 
> The instruction decode (decode_trapped_insn() and the mask/match chain
> for standard and compressed load encodings) is adapted from Linux's KVM
> RISC-V implementation. The completion path differs from KVM's,
> since Xen dispatches MMIO synchronously to an in-hypervisor handler via
> try_handle_mmio() and has no userspace exit/return step equivalent to
> KVM's kvm_io_bus_read() / KVM_EXIT_MMIO / kvm_riscv_vcpu_mmio_return()
> split.
> 
> A fault taken while re-reading the trapped instruction is handled
> depending on the faulting translation stage:
>  - A VS-stage fault is the guest's own fault (e.g. it modified its page
>    tables from another vCPU) and, as in KVM, is redirected to the
>    guest's trap vector, with the cause remapped to
>    CAUSE_FETCH_PAGE_FAULT since HLVX reports execute-permission failures
>    as load faults.
>  - A G-stage fault would mean the P2M mapping of the instruction page
>    disappeared after the instruction was fetched. KVM must handle this
>    by resuming the guest and retrying, as Linux MM can invalidate
>    G-stage mappings at any time. Xen does not remove P2M mappings of a
>    running domain at the moment, so this case is asserted unreachable with
>    BUG_ON(); it will need to be revisited once such removal is implemented.

I don't see why this cannot be implemented correctly right away. The behavior
should be that of an access to unpopulated space on bare hardware, whatever
that behavior is on RISC-V.

> @@ -13,6 +14,11 @@ struct trap_info {
>      register_t stval;
>  };
>  
> +static inline bool is_load_guest_page_fault(unsigned long scause)
> +{
> +    return (scause == CAUSE_LOAD_GUEST_PAGE_FAULT);
> +}

Is something like this really a useful wrapper to have? It doesn't really
shorten anything, nor does (imo) it aid readability.

> @@ -191,6 +193,11 @@ static void timer_interrupt(void)
>      raise_softirq(TIMER_SOFTIRQ);
>  }
>  
> +static always_inline void advance_pc(struct cpu_user_regs *regs, int step)

See my earlier remark regarding always_inline. Also - why plain int? Are
there (going to be) cases where PC is moved backwards (in which case
"advance" isn't suitable naming)?

> +{
> +    regs->sepc += step;
> +}
> +
>  static always_inline unsigned long get_faulting_gpa(void)
>  {
>      /*
> @@ -210,9 +217,162 @@ static always_inline unsigned long get_faulting_gpa(void)
>      return (csr_read(CSR_HTVAL) << 2) | (csr_read(CSR_STVAL) & 0x3);
>  }
>  
> +/*
> + * Determine the trapped instruction which caused a guest MMIO trap.
> + *
> + * Returns true if the trap was redirected to the guest, in which case
> + * the caller must stop emulation and return success. Otherwise *insn
> + * and *insn_len are filled in and the caller should continue decoding.
> + */
> +static bool decode_trapped_insn(unsigned long htinst, unsigned long *insn,
> +                                unsigned int *insn_len)
> +{
> +    if ( htinst & 0x1 )
> +    {
> +        /*
> +         * Bit[0] == 1 implies trapped instruction value is
> +         * transformed instruction or custom instruction.
> +         */
> +        *insn = htinst | INSN_16BIT_MASK;
> +        *insn_len = (htinst & BIT(1, UL)) ? INSN_LEN(*insn) : 2;

In the if() you don't use BIT(), while here you do. Please be consistent.

Why the use of INSN_LEN(), when due to the earlier assignment it'll always
yield 4 here?

Finally, how would the caller know whether it looks at a transformed insn
or (as fetched below) a "normal" one?

> +    }
> +    else
> +    {
> +        struct cpu_user_regs *regs = vcpu_guest_cpu_user_regs(current);

Pointer-to-const.

> +        struct trap_info utrap = { 0 };

Just {} please.

> +        /*
> +         * Bit[0] == 0 implies trapped instruction value is
> +         * zero or special value.
> +         */

How come you get away without dealing with pseudoinsns? The insn pointed at
by regs->sepc is of no interest for faults caused by implicit memory accesses
originating from VS-stage address translation.

> +        *insn = riscv_vcpu_unpriv_read(true, regs->sepc, &utrap);
> +        if ( utrap.scause )
> +        {
> +            /*
> +             * A G-stage fault here would mean the P2M mapping of the page
> +             * containing the trapped instruction disappeared after it was
> +             * fetched.

Does it? What about, again, faults from VS-stage address translation while
hardware was trying to fetch an insn? That is ...

> Nothing removes P2M mappings of a running domain yet,
> +             * so this cannot happen.

... the necessary P2M mapping may never have been there.

> +             * TODO: Revisit once P2M mappings can be removed at runtime.
> +             */
> +            BUG_ON(is_load_guest_page_fault(utrap.scause));
> +
> +            utrap.sepc = regs->sepc;
> +            utrap.stval = utrap.sepc;

How do you know the fault was at .sepc? A 32-bit insn crossing a page boundary
(implying the C extension is available) may well fault only on its higher half.

> +            riscv_vcpu_trap_redirect(&utrap);
> +
> +            return true;
> +        }
> +
> +        *insn_len = INSN_LEN(*insn);
> +    }
> +
> +    return false;
> +}
> +
> +/*
> + * Check alignment and dispatch a decoded MMIO access to a registered
> + * handler. On success (0), info->data holds the read value for loads.
> + */
> +static int do_mmio(mmio_info_t *info, unsigned long fault_addr,
> +                   unsigned int len)
> +{
> +    /* Fault address should be aligned to length of MMIO */
> +    if ( fault_addr & (len - 1) )
> +        return -EIO;
> +
> +    info->gpa = fault_addr;
> +    info->len = len;
> +
> +    switch ( try_handle_mmio(info) )
> +    {
> +    case IO_HANDLED:
> +        return 0;
> +    case IO_ABORT:
> +        return -EIO;
> +    default:
> +        return -EOPNOTSUPP;
> +    }
> +}

And there's no indication of "retry needed", e.g. when something changed
between find_mmio_handler() and handle_{read,write}()?

Also, nit: Blank lines please between non-fall-through case blocks.

>  static int emulate_load(unsigned long fault_addr, unsigned long htinst)
>  {
> -    return -EOPNOTSUPP;
> +    struct cpu_user_regs *regs = vcpu_guest_cpu_user_regs(current);
> +    mmio_info_t info = { .is_write = false };
> +    unsigned long insn;
> +    unsigned int shift = 0, len, insn_len;
> +    bool is_unsigned = false;
> +    int rc;
> +
> +    if ( decode_trapped_insn(htinst, &insn, &insn_len) )
> +        return 0;
> +
> +    /* Decode length of MMIO and whether it is a sign- or zero-extending load */
> +    if ( (insn & INSN_MASK_LB) == INSN_MATCH_LB )
> +        len = 1;
> +    else if ( (insn & INSN_MASK_LBU) == INSN_MATCH_LBU )
> +    {
> +        len = 1;
> +        is_unsigned = true;
> +    }
> +    else if ( (insn & INSN_MASK_LH) == INSN_MATCH_LH )
> +        len = 2;
> +    else if ( (insn & INSN_MASK_LHU) == INSN_MATCH_LHU )
> +    {
> +        len = 2;
> +        is_unsigned = true;
> +    }
> +    else if ( (insn & INSN_MASK_LW) == INSN_MATCH_LW )
> +        len = 4;

Already up to here this demonstrates a weakness of the INSN_MASK_*
set of #define-s (which I similarly observe in binutils, and I expect it
all has the same questionable origin). All INSN_MASK_L* and INSN_MASK_FL*
(also INSN_MASK_S* and INSN_MASK_FS*) are identical, allowing for a nice
switch() to be used here in principle. That said, with access width
nicely encoded in FUNCT3, it's not even clear whether a switch() would
end up being needed / efficient.

Otoh none of these masks cover the pseudoinsns that htinst may supply.

Further, what about A-extension insns? Some (if not all) of them can
plausibly be used on MMIO, I think.

> +#ifndef CONFIG_RISCV_32
> +    else if ( (insn & INSN_MASK_LWU) == INSN_MATCH_LWU )

First: Better use IS_ENABLED() in favor of #if{,n}def, whenever possible.
And then this depends not only on CONFIG_RISCV_32, but also on guest
bitness.

> +    {
> +        len = 4;
> +        is_unsigned = true;
> +    }
> +#endif
> +    else if ( (insn & INSN_MASK_C_LW) == INSN_MATCH_C_LW )
> +    {
> +        len = 4;
> +        insn = RVC_RS2S(insn) << SH_RD;
> +    }
> +    else if ( (insn & INSN_MASK_C_LWSP) == INSN_MATCH_C_LWSP &&
> +              RV_X(insn, SH_RD, 5) )
> +        len = 4;
> +#ifndef CONFIG_RISCV_32
> +    else if ( (insn & INSN_MASK_LD) == INSN_MATCH_LD )
> +        len = 8;
> +    else if ( (insn & INSN_MASK_C_LD) == INSN_MATCH_C_LD )
> +    {
> +        len = 8;
> +        insn = RVC_RS2S(insn) << SH_RD;
> +    }
> +    else if ( (insn & INSN_MASK_C_LDSP) == INSN_MATCH_C_LDSP &&
> +              RV_X(insn, SH_RD, 5) )
> +        len = 8;
> +#endif
> +    else
> +        return -EOPNOTSUPP;

Because you don't permit F/D/Q for guests (yet), FL* and FS* aren't
covered, I expect? I wonder how easy it is going to be to spot the places
needing adjustment once support is to be added. Same perhaps for Zilsd in
RV32 guests.

Jan

Re: [PATCH v1 16/17] xen/riscv: add guest load emulation for trapped MMIO accesses

[Jan Beulich]

On 13.08.2026 09:15, Jan Beulich wrote:
> On 29.07.2026 15:40, Oleksii Kurochko wrote:
>>  static int emulate_load(unsigned long fault_addr, unsigned long htinst)
>>  {
>> -    return -EOPNOTSUPP;
>> +    struct cpu_user_regs *regs = vcpu_guest_cpu_user_regs(current);
>> +    mmio_info_t info = { .is_write = false };
>> +    unsigned long insn;
>> +    unsigned int shift = 0, len, insn_len;
>> +    bool is_unsigned = false;
>> +    int rc;
>> +
>> +    if ( decode_trapped_insn(htinst, &insn, &insn_len) )
>> +        return 0;
>> +
>> +    /* Decode length of MMIO and whether it is a sign- or zero-extending load */
>> +    if ( (insn & INSN_MASK_LB) == INSN_MATCH_LB )
>> +        len = 1;
>> +    else if ( (insn & INSN_MASK_LBU) == INSN_MATCH_LBU )
>> +    {
>> +        len = 1;
>> +        is_unsigned = true;
>> +    }
>> +    else if ( (insn & INSN_MASK_LH) == INSN_MATCH_LH )
>> +        len = 2;
>> +    else if ( (insn & INSN_MASK_LHU) == INSN_MATCH_LHU )
>> +    {
>> +        len = 2;
>> +        is_unsigned = true;
>> +    }
>> +    else if ( (insn & INSN_MASK_LW) == INSN_MATCH_LW )
>> +        len = 4;
> 
> Already up to here this demonstrates a weakness of the INSN_MASK_*
> set of #define-s (which I similarly observe in binutils, and I expect it
> all has the same questionable origin). All INSN_MASK_L* and INSN_MASK_FL*
> (also INSN_MASK_S* and INSN_MASK_FS*) are identical, allowing for a nice
> switch() to be used here in principle. That said, with access width
> nicely encoded in FUNCT3, it's not even clear whether a switch() would
> end up being needed / efficient.
> 
> Otoh none of these masks cover the pseudoinsns that htinst may supply.
> 
> Further, what about A-extension insns? Some (if not all) of them can
> plausibly be used on MMIO, I think.

Because of the further additions that are going to be needed, may I also
suggest to consider putting emulation code in its own file (emulate.c
perhaps), rather than directly in traps.c?

Jan

Re: [PATCH v1 16/17] xen/riscv: add guest load emulation for trapped MMIO accesses

[Oleksii Kurochko]

On 8/13/26 9:28 AM, Jan Beulich wrote:
> On 13.08.2026 09:15, Jan Beulich wrote:
>> On 29.07.2026 15:40, Oleksii Kurochko wrote:
>>>   static int emulate_load(unsigned long fault_addr, unsigned long htinst)
>>>   {
>>> -    return -EOPNOTSUPP;
>>> +    struct cpu_user_regs *regs = vcpu_guest_cpu_user_regs(current);
>>> +    mmio_info_t info = { .is_write = false };
>>> +    unsigned long insn;
>>> +    unsigned int shift = 0, len, insn_len;
>>> +    bool is_unsigned = false;
>>> +    int rc;
>>> +
>>> +    if ( decode_trapped_insn(htinst, &insn, &insn_len) )
>>> +        return 0;
>>> +
>>> +    /* Decode length of MMIO and whether it is a sign- or zero-extending load */
>>> +    if ( (insn & INSN_MASK_LB) == INSN_MATCH_LB )
>>> +        len = 1;
>>> +    else if ( (insn & INSN_MASK_LBU) == INSN_MATCH_LBU )
>>> +    {
>>> +        len = 1;
>>> +        is_unsigned = true;
>>> +    }
>>> +    else if ( (insn & INSN_MASK_LH) == INSN_MATCH_LH )
>>> +        len = 2;
>>> +    else if ( (insn & INSN_MASK_LHU) == INSN_MATCH_LHU )
>>> +    {
>>> +        len = 2;
>>> +        is_unsigned = true;
>>> +    }
>>> +    else if ( (insn & INSN_MASK_LW) == INSN_MATCH_LW )
>>> +        len = 4;
>>
>> Already up to here this demonstrates a weakness of the INSN_MASK_*
>> set of #define-s (which I similarly observe in binutils, and I expect it
>> all has the same questionable origin). All INSN_MASK_L* and INSN_MASK_FL*
>> (also INSN_MASK_S* and INSN_MASK_FS*) are identical, allowing for a nice
>> switch() to be used here in principle. That said, with access width
>> nicely encoded in FUNCT3, it's not even clear whether a switch() would
>> end up being needed / efficient.
>>
>> Otoh none of these masks cover the pseudoinsns that htinst may supply.
>>
>> Further, what about A-extension insns? Some (if not all) of them can
>> plausibly be used on MMIO, I think.
> 
> Because of the further additions that are going to be needed, may I also
> suggest to consider putting emulation code in its own file (emulate.c
> perhaps), rather than directly in traps.c?

Good point. It really makes sense to move emulation now to emulate.c.

Thanks.

~ Oleksii

Re: [PATCH v1 16/17] xen/riscv: add guest load emulation for trapped MMIO accesses

[Oleksii Kurochko]

On 8/13/26 9:15 AM, Jan Beulich wrote:
> On 29.07.2026 15:40, Oleksii Kurochko wrote:
>> Introduce emulate_load() to decode and emulate guest load instructions
>> that fault due to MMIO accesses. This provides the basic infrastructure
>> required for MMIO emulation on RISC-V.
>>
>> The instruction decode (decode_trapped_insn() and the mask/match chain
>> for standard and compressed load encodings) is adapted from Linux's KVM
>> RISC-V implementation. The completion path differs from KVM's,
>> since Xen dispatches MMIO synchronously to an in-hypervisor handler via
>> try_handle_mmio() and has no userspace exit/return step equivalent to
>> KVM's kvm_io_bus_read() / KVM_EXIT_MMIO / kvm_riscv_vcpu_mmio_return()
>> split.
>>
>> A fault taken while re-reading the trapped instruction is handled
>> depending on the faulting translation stage:
>>   - A VS-stage fault is the guest's own fault (e.g. it modified its page
>>     tables from another vCPU) and, as in KVM, is redirected to the
>>     guest's trap vector, with the cause remapped to
>>     CAUSE_FETCH_PAGE_FAULT since HLVX reports execute-permission failures
>>     as load faults.
>>   - A G-stage fault would mean the P2M mapping of the instruction page
>>     disappeared after the instruction was fetched. KVM must handle this
>>     by resuming the guest and retrying, as Linux MM can invalidate
>>     G-stage mappings at any time. Xen does not remove P2M mappings of a
>>     running domain at the moment, so this case is asserted unreachable with
>>     BUG_ON(); it will need to be revisited once such removal is implemented.
> 
> I don't see why this cannot be implemented correctly right away. The behavior
> should be that of an access to unpopulated space on bare hardware, whatever
> that behavior is on RISC-V.

I wasn't able to find a spec what should be returned in this case but in 
QEMU source code I founded (unassigned_mem_ops → MEMTX_DECODE_ERROR → 
io_failed() → riscv_cpu_do_transaction_failed().):

void riscv_cpu_do_transaction_failed(CPUState *cs, hwaddr physaddr,
                                      vaddr addr, unsigned size,
                                      MMUAccessType access_type,
                                      int mmu_idx, MemTxAttrs attrs,
                                      MemTxResult response, uintptr_t 
retaddr)
{
     RISCVCPU *cpu = RISCV_CPU(cs);
     CPURISCVState *env = &cpu->env;

     if (access_type == MMU_DATA_STORE) {
         cs->exception_index = RISCV_EXCP_STORE_AMO_ACCESS_FAULT;
     } else if (access_type == MMU_DATA_LOAD) {
         cs->exception_index = RISCV_EXCP_LOAD_ACCESS_FAULT;
     } else {
         cs->exception_index = RISCV_EXCP_INST_ACCESS_FAULT;
     }

So RISCV_EXCP_INST_ACCESS_FAULT (in Xen it is CAUSE_FETCH_ACCESS) will 
be fine to return.

So do the following:

             if ( is_load_guest_page_fault(utrap.scause) )
                 utrap.scause = CAUSE_FETCH_ACCESS;

will be fair enough instead of: 
BUG_ON(is_load_guest_page_fault(utrap.scause)).

Probably, we want to rename CAUSE_FETCH_ACCESS to be closer to RISC-V 
spec as for value 1 in spec it is used:
    1 Instruction access fault
Interesting that all other CAUSE_* defines are aligned with the spec...

> 
>> @@ -13,6 +14,11 @@ struct trap_info {
>>       register_t stval;
>>   };
>>   
>> +static inline bool is_load_guest_page_fault(unsigned long scause)
>> +{
>> +    return (scause == CAUSE_LOAD_GUEST_PAGE_FAULT);
>> +}
> 
> Is something like this really a useful wrapper to have? It doesn't really
> shorten anything, nor does (imo) it aid readability.
> 
>> @@ -191,6 +193,11 @@ static void timer_interrupt(void)
>>       raise_softirq(TIMER_SOFTIRQ);
>>   }
>>   
>> +static always_inline void advance_pc(struct cpu_user_regs *regs, int step)
> 
> See my earlier remark regarding always_inline. Also - why plain int? Are
> there (going to be) cases where PC is moved backwards (in which case
> "advance" isn't suitable naming)?

No, it won't. At least, I don't see such use cases now. I'll use 
unsigned int instead.

> 
>> +{
>> +    regs->sepc += step;
>> +}
>> +
>>   static always_inline unsigned long get_faulting_gpa(void)
>>   {
>>       /*
>> @@ -210,9 +217,162 @@ static always_inline unsigned long get_faulting_gpa(void)
>>       return (csr_read(CSR_HTVAL) << 2) | (csr_read(CSR_STVAL) & 0x3);
>>   }
>>   
>> +/*
>> + * Determine the trapped instruction which caused a guest MMIO trap.
>> + *
>> + * Returns true if the trap was redirected to the guest, in which case
>> + * the caller must stop emulation and return success. Otherwise *insn
>> + * and *insn_len are filled in and the caller should continue decoding.
>> + */
>> +static bool decode_trapped_insn(unsigned long htinst, unsigned long *insn,
>> +                                unsigned int *insn_len)
>> +{
>> +    if ( htinst & 0x1 )
>> +    {
>> +        /*
>> +         * Bit[0] == 1 implies trapped instruction value is
>> +         * transformed instruction or custom instruction.
>> +         */
>> +        *insn = htinst | INSN_16BIT_MASK;
>> +        *insn_len = (htinst & BIT(1, UL)) ? INSN_LEN(*insn) : 2;
> 
> In the if() you don't use BIT(), while here you do. Please be consistent.
> 
> Why the use of INSN_LEN(), when due to the earlier assignment it'll always
> yield 4 here?

ld/sd instruction which we are trapping here at the moment here could be 
2 bit and 4 bit depends on C extension so we need to pass correct 
instruction length to advance_pc() after it is emulated.

> 
> Finally, how would the caller know whether it looks at a transformed insn
> or (as fetched below) a "normal" one?

According to the spec ((part from htinst ... ):
On a synchronous exception, if a nonzero value is written, one of the 
following shall be true about the value:

• Bit 0 is 1, and replacing bit 1 with 1 makes the value into a valid 
encoding of a standard instruction.
In this case, the instruction that trapped is the same kind as indicated 
by the register value, and the register value is the transformation of 
the trapping instruction, as defined later. For example, if bits 1:0 are 
binary 11 and the register value is the encoding of a standard LW (load 
word) instruction, then the trapping instruction is LW, and the register 
value is the transformation of the trapping LW instruction.

• Bit 0 is 1, and replacing bit 1 with 1 makes the value into an 
instruction encoding that is explicitly designated for a custom 
instruction (not an unused reserved encoding). This is a custom value. 
The instruction that trapped is a non-standard instruction. The
interpretation of a custom value is not otherwise specified by this 
standard.

• The value is one of the special pseudoinstructions defined later, all 
of which have bits 1:0 equal to 00.

So setting bit 0 to 1 we will guarantee that it is normal "normal" 
instruction.

> 
>> +    }
>> +    else
>> +    {
>> +        struct cpu_user_regs *regs = vcpu_guest_cpu_user_regs(current);
> 
> Pointer-to-const.
> 
>> +        struct trap_info utrap = { 0 };
> 
> Just {} please.
> 
>> +        /*
>> +         * Bit[0] == 0 implies trapped instruction value is
>> +         * zero or special value.
>> +         */
> 
> How come you get away without dealing with pseudoinsns? The insn pointed at
> by regs->sepc is of no interest for faults caused by implicit memory accesses
> originating from VS-stage address translation.

It is really problem but I think it should be resolved much earlier in 
handle_guest_page_fault(). I will add the following:

/*
      * A guest page fault taken on an implicit memory access performed for
      * VS-stage address translation (reading a PTE, or updating its A/D 
bits)
      * reports a pseudoinstruction in htinst rather than a transformed
      * instruction. Such a fault can't be emulated: htval holds the guest
      * physical address of a VS-stage PTE rather than of any access the 
guest
      * itself performed (and its two least significant bits are zero 
instead
      * of matching stval), while the instruction at sepc is unrelated 
to the
      * access which actually faulted.
      *
      * Report an access fault to the guest at the original virtual address,
      * which is what stval already holds and what hardware would raise 
for a
      * page table walk hitting an inaccessible address.
      */
     if ( (htinst == INSN_PSEUDO_VS_LOAD) || (htinst == 
INSN_PSEUDO_VS_STORE) )
     {
         struct cpu_user_regs *regs = vcpu_guest_cpu_user_regs(current);
         struct trap_info utrap = {
             .scause = (htinst == INSN_PSEUDO_VS_LOAD) ? CAUSE_LOAD_ACCESS
                                                       : CAUSE_STORE_ACCESS,
             .sepc = regs->sepc,
             .stval = csr_read(CSR_STVAL),
         };

         riscv_trap_redirect(&utrap);
         return;
     }

and will update the comment:

 >> +        /*
 >> +         * Bit[0] == 0 implies trapped instruction value is
 >> +         * zero or special value. It can't be pseudoinstruction as 
it is guaranteed by check in handle_guest_page_fault().
 >> +         */

> 
>> +        *insn = riscv_vcpu_unpriv_read(true, regs->sepc, &utrap);
>> +        if ( utrap.scause )
>> +        {
>> +            /*
>> +             * A G-stage fault here would mean the P2M mapping of the page
>> +             * containing the trapped instruction disappeared after it was
>> +             * fetched.
> 
> Does it? What about, again, faults from VS-stage address translation while
> hardware was trying to fetch an insn? That is ...
> 
>> Nothing removes P2M mappings of a running domain yet,
>> +             * so this cannot happen.
> 
> ... the necessary P2M mapping may never have been there.

If VS-stage failed then CAUSE_LOAD_PAGE_FAULT will happen so BUG_ON() 
won't occur and it will be passed to guest to handle it.

BUG_ON() here catches CAUSE_LOAD_GUEST_PAGE_FAULT (G-stage translation 
failure).

Also, as I mentioned above I will change BUG_ON() too:

             /*
              * If during getting of trapped instruction a fault happen in
              * G-stage translation then CAUSE_LOAD_GUEST_PAGE_FAULT is
              * generated. Such faults during this operation is 
considered as
              * bus
              */
             if ( is_load_guest_page_fault(utrap.scause) )
                 utrap.scause = CAUSE_FETCH_ACCESS;


> 
>> +             * TODO: Revisit once P2M mappings can be removed at runtime.
>> +             */
>> +            BUG_ON(is_load_guest_page_fault(utrap.scause));
>> +
>> +            utrap.sepc = regs->sepc;
>> +            utrap.stval = utrap.sepc;
> 
> How do you know the fault was at .sepc? A 32-bit insn crossing a page boundary
> (implying the C extension is available) may well fault only on its higher half.

According to the spec, if stval is written with a nonzero value when an 
instruction access-fault or page-fault exception occurs on a system with 
variable-length instructions, then stval will contain the virtual 
address of the portion of the instruction that caused the fault, while 
sepc will point to the beginning of the instruction.

So here, we are trying to emulate what real hardware will do in this 
case. In regs->sepc, we have the start of the instruction that we didn't 
touch. sepc is filled according to the spec in this case.

Regarding utrap.stval, we know that utrap.sepc points to the correct 
part of the faulting address, as we are reading the instruction in 
16-bit chunks:

HLVX_HU(%[val], %[addr])        ; low 16 bits from sepc
andi %[tmp], %[val], 3
addi %[tmp], %[tmp], -3
bne  %[tmp], zero, 2f           ; if not (insn & 3) == 3 -> 16-bit, end
addi %[addr], %[addr], 2        ; <- addr is now sepc+2
HLVX_HU(%[tmp], %[addr])        ; high 16 bits, possibly from another page

So, if a trap happens while reading the high 16 bits (which may be 
located on another page), then utrap.sepc, if the read fails, will point 
to the high part of the instruction, which is what the spec requires.

Does that make sense?

> 
>> +            riscv_vcpu_trap_redirect(&utrap);
>> +
>> +            return true;
>> +        }
>> +
>> +        *insn_len = INSN_LEN(*insn);
>> +    }
>> +
>> +    return false;
>> +}
>> +
>> +/*
>> + * Check alignment and dispatch a decoded MMIO access to a registered
>> + * handler. On success (0), info->data holds the read value for loads.
>> + */
>> +static int do_mmio(mmio_info_t *info, unsigned long fault_addr,
>> +                   unsigned int len)
>> +{
>> +    /* Fault address should be aligned to length of MMIO */
>> +    if ( fault_addr & (len - 1) )
>> +        return -EIO;
>> +
>> +    info->gpa = fault_addr;
>> +    info->len = len;
>> +
>> +    switch ( try_handle_mmio(info) )
>> +    {
>> +    case IO_HANDLED:
>> +        return 0;
>> +    case IO_ABORT:
>> +        return -EIO;
>> +    default:
>> +        return -EOPNOTSUPP;
>> +    }
>> +}
> 
> And there's no indication of "retry needed", e.g. when something changed
> between find_mmio_handler() and handle_{read,write}()?

I don't have any specific scenario where it is needed now so I don't 
know what to say.
And there is no race between find_mmio_handler() and 
handle_{read,write}() as find_mmio_handler() returns copy of the 
structure under read_lock():
/*
  * Return a copy of the matching handler rather than a pointer into
  * vmmio->handlers: a concurrent register_mmio_handler() shifts entries
  * up to keep the array sorted, so an escaped pointer could refer to a
  * different (or torn) entry once the lock is dropped. The copy stays
  * valid as the ops structures are never freed.
  */
static bool find_mmio_handler(struct domain *d, paddr_t gpa,
                               struct mmio_handler *out)
{
     struct vmmio *vmmio = &d->arch.vmmio;
     struct mmio_handler key = { .addr = gpa };
     const struct mmio_handler *handler;

     read_lock(&vmmio->lock);
     handler = bsearch(&key, vmmio->handlers, vmmio->num_entries,
                       sizeof(*handler), cmp_mmio_handler);
     if ( handler )
         *out = *handler;
     read_unlock(&vmmio->lock);

     return handler != NULL;
}

At the moment, use cases are pretty strainghforward, a device from guest 
trying to read/write into MMIO and so the result logically could be or 
it is successfully handled or some issue happened and it is just 
aborted. As a real hardware will do, I think.

> 
> Also, nit: Blank lines please between non-fall-through case blocks.
> 
>>   static int emulate_load(unsigned long fault_addr, unsigned long htinst)
>>   {
>> -    return -EOPNOTSUPP;
>> +    struct cpu_user_regs *regs = vcpu_guest_cpu_user_regs(current);
>> +    mmio_info_t info = { .is_write = false };
>> +    unsigned long insn;
>> +    unsigned int shift = 0, len, insn_len;
>> +    bool is_unsigned = false;
>> +    int rc;
>> +
>> +    if ( decode_trapped_insn(htinst, &insn, &insn_len) )
>> +        return 0;
>> +
>> +    /* Decode length of MMIO and whether it is a sign- or zero-extending load */
>> +    if ( (insn & INSN_MASK_LB) == INSN_MATCH_LB )
>> +        len = 1;
>> +    else if ( (insn & INSN_MASK_LBU) == INSN_MATCH_LBU )
>> +    {
>> +        len = 1;
>> +        is_unsigned = true;
>> +    }
>> +    else if ( (insn & INSN_MASK_LH) == INSN_MATCH_LH )
>> +        len = 2;
>> +    else if ( (insn & INSN_MASK_LHU) == INSN_MATCH_LHU )
>> +    {
>> +        len = 2;
>> +        is_unsigned = true;
>> +    }
>> +    else if ( (insn & INSN_MASK_LW) == INSN_MATCH_LW )
>> +        len = 4;
> 
> Already up to here this demonstrates a weakness of the INSN_MASK_*
> set of #define-s (which I similarly observe in binutils, and I expect it
> all has the same questionable origin). All INSN_MASK_L* and INSN_MASK_FL*
> (also INSN_MASK_S* and INSN_MASK_FS*) are identical, allowing for a nice
> switch() to be used here in principle. That said, with access width
> nicely encoded in FUNCT3, it's not even clear whether a switch() would
> end up being needed / efficient.

.......

> 
> Otoh none of these masks cover the pseudoinsns that htinst may supply.

As I answered above we should handle that before this function will call 
so here we won't deal with htinst at all. Of course, if what I wrote 
above is correct. I will double check before applying that.

> 
> Further, what about A-extension insns? Some (if not all) of them can
> plausibly be used on MMIO, I think.

I’m not really sure that the A-extension is actively used for MMIO. At 
least, Linux doesn’t do that for now, which is why we don’t handle 
A-extension instructions here.

I think this is related to the fact that MMIO is usually (if not 
always?) naturally aligned, and naturally aligned loads and stores are 
guaranteed by RISC-V to execute atomically.

Anyway, since we don’t have a case for this for now, I think we could go 
with the current emulation. If this turns out not to be true in the 
future, A-extension support can be added separately.

> 
>> +#ifndef CONFIG_RISCV_32
>> +    else if ( (insn & INSN_MASK_LWU) == INSN_MATCH_LWU )
> 
> First: Better use IS_ENABLED() in favor of #if{,n}def, whenever possible.
> And then this depends not only on CONFIG_RISCV_32, but also on guest
> bitness.

Both points taken. The #ifndef will become a condition in the if-chain; 
the INSN_MATCH_/INSN_MASK_ definitions are unconditional in 
riscv_encoding.h, so that builds either way.

On guest bitness you're right, and it's worse than the 32-bit-only 
encodings being reserved in RV32: the compressed RV64 encodings collide 
with the RV32 single-precision float ones — C.LD and C.FLW are both 
0x6000 under mask 0xe003, likewise C.SD/C.FSW, C.LDSP/C.FLWSP and 
C.SDSP/C.FSWSP. A 32-bit guest doing a c.flw to an emulated MMIO region 
would be decoded as c.ld, i.e.an 8-byte access with the result written 
to an integer register.

I'll fold both into a helper returning the guest's effective XLEN
(hstatus.VSXL, or vsstatus.UXL when the trap was taken from VU-mode, and
unconditionally 32 for a RV32 build) and gate the RV64-only cases on it. 
As a side note, vcpu hstatus setup currently leaves VSXL alone and thus 
relies on the WARL behaviour of the field; I think Xen should set it 
explicitly.

/*
  * The effective XLEN of the guest at the point of the trap: 
hstatus.VSXL for a
  * trap taken from VS-mode, vsstatus.UXL for one taken from VU-mode.
  *
  * It is needed to decode a trapped instruction: the encodings which 
exist only
  * for XLEN=64 must not be recognized for a 32-bit guest. Besides those 
simply
  * being reserved there, the compressed ones are ambiguous: C.LD and C.FLW
  * share the encoding 0x6000 (mask 0xe003), and likewise C.SD/C.FSW,
  * C.LDSP/C.FLWSP and C.SDSP/C.FSWSP.
  *
  * IS_ENABLED() can't be used here as HSTATUS_VSXL is defined for
  * __riscv_xlen == 64 only, the field not existing on RV32 in the first 
place.
  */
static unsigned int guest_xlen(const struct cpu_user_regs *regs)
{
#ifdef CONFIG_RISCV_32
     return 32;
#else
     unsigned long xl = (regs->sstatus & SSTATUS_SPP)
                        ? MASK_EXTR(regs->hstatus, HSTATUS_VSXL)
                        : MASK_EXTR(csr_read(CSR_VSSTATUS), SSTATUS64_UXL);

     /* 1 encodes XLEN=32, 2 encodes XLEN=64. */
     return (xl == HSTATUS_VSXL_32) ? 32 : 64;
#endif
}

and then use it in emulate_store/load():

     unsigned int xlen = guest_xlen(regs);
     ...
     else if ( (xlen == 64) && ((insn & INSN_MASK_LWU) == INSN_MATCH_LWU) )
     {
         len = 4;
         is_unsigned = true;
     }
     ...
     else if ( (xlen == 64) && ((insn & INSN_MASK_C_LD) == 
INSN_MATCH_C_LD) )


> 
>> +    {
>> +        len = 4;
>> +        is_unsigned = true;
>> +    }
>> +#endif
>> +    else if ( (insn & INSN_MASK_C_LW) == INSN_MATCH_C_LW )
>> +    {
>> +        len = 4;
>> +        insn = RVC_RS2S(insn) << SH_RD;
>> +    }
>> +    else if ( (insn & INSN_MASK_C_LWSP) == INSN_MATCH_C_LWSP &&
>> +              RV_X(insn, SH_RD, 5) )
>> +        len = 4;
>> +#ifndef CONFIG_RISCV_32
>> +    else if ( (insn & INSN_MASK_LD) == INSN_MATCH_LD )
>> +        len = 8;
>> +    else if ( (insn & INSN_MASK_C_LD) == INSN_MATCH_C_LD )
>> +    {
>> +        len = 8;
>> +        insn = RVC_RS2S(insn) << SH_RD;
>> +    }
>> +    else if ( (insn & INSN_MASK_C_LDSP) == INSN_MATCH_C_LDSP &&
>> +              RV_X(insn, SH_RD, 5) )
>> +        len = 8;
>> +#endif
>> +    else
>> +        return -EOPNOTSUPP;
> 
> Because you don't permit F/D/Q for guests (yet), FL* and FS* aren't
> covered, I expect? 

At the moment, I wrote this function with handling of MMIO instruction 
in mind, which are at the moment ld and sd.

Even if to permit F/D/Q then do we really need to trap that 
instructions? Hypervisor could allow access to FPU to guest and then it 
will be just a question of context switch to properly save and restore FPU.

> I wonder how easy it is going to be to spot the places
> needing adjustment once support is to be added. Same perhaps for Zilsd in
> RV32 guests.

There is a message in handle_guest_page_fault():

     if ( rc )
         domain_crash(current->domain,
                      "%s: unable to handle faulted guest %s addr %#lx\n",
                      __func__,
                      (cause == CAUSE_LOAD_GUEST_PAGE_FAULT) ? "load" : 
"store",
                      addr);

Probably, it isn't enough and we could print here instruction (in hex) 
before return -EOPNOTSUPP.

Thanks.

~ Oleksii

[PATCH v1 17/17] xen/riscv: add guest store emulation for trapped MMIO accesses

[Oleksii Kurochko]

Extend the guest page fault handler with store emulation to support MMIO
write accesses.

The instruction decode mirrors emulate_load() and, like it, is adapted
from Linux's KVM RISC-V implementation. As with the load path, the
completion is synchronous through try_handle_mmio() rather than KVM's
userspace exit/return split, since Xen's MMIO handlers run in the
hypervisor. Faults taken while re-reading the trapped instruction are
handled by decode_trapped_insn(), shared with the load path.

When a guest store instruction faults, the trapped instruction is decoded
using HTINST or, if unavailable, fetched via unprivileged access. At the
moment only virtual interrupt controller (vINTC) traps are expected to
occur, since it is currently the only backend registered with the MMIO
handler dispatch, so in practice the store is emulated via the vINTC
backend.

Together with load emulation, this completes the basic MMIO handling path
needed for virtual interrupt controller support on RISC-V.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 xen/arch/riscv/traps.c | 55 +++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 54 insertions(+), 1 deletion(-)

diff --git a/xen/arch/riscv/traps.c b/xen/arch/riscv/traps.c
index 12690ae37aba..18156627d7cd 100644
--- a/xen/arch/riscv/traps.c
+++ b/xen/arch/riscv/traps.c
@@ -377,7 +377,60 @@ static int emulate_load(unsigned long fault_addr, unsigned long htinst)
 
 static int emulate_store(unsigned long fault_addr, unsigned long htinst)
 {
-    return -EOPNOTSUPP;
+    struct cpu_user_regs *regs = vcpu_guest_cpu_user_regs(current);
+    mmio_info_t info = { .is_write = true };
+    register_t data;
+    unsigned long insn;
+    unsigned int len, insn_len;
+    int rc;
+
+    if ( decode_trapped_insn(htinst, &insn, &insn_len) )
+        return 0;
+
+    data = GET_RS2(insn, regs);
+
+    if ( (insn & INSN_MASK_SB) == INSN_MATCH_SB )
+        len = 1;
+    else if ( (insn & INSN_MASK_SH) == INSN_MATCH_SH )
+        len = 2;
+    else if ( (insn & INSN_MASK_SW) == INSN_MATCH_SW )
+        len = 4;
+    else if ( (insn & INSN_MASK_C_SW) == INSN_MATCH_C_SW )
+    {
+        len = 4;
+        data = GET_RS2S(insn, regs);
+    }
+    else if ( (insn & INSN_MASK_C_SWSP) == INSN_MATCH_C_SWSP )
+    {
+        len = 4;
+        data = GET_RS2C(insn, regs);
+    }
+#ifndef CONFIG_RISCV_32
+    else if ( (insn & INSN_MASK_SD) == INSN_MATCH_SD )
+        len = 8;
+    else if ( (insn & INSN_MASK_C_SD) == INSN_MATCH_C_SD )
+    {
+        len = 8;
+        data = GET_RS2S(insn, regs);
+    }
+    else if ( (insn & INSN_MASK_C_SDSP) == INSN_MATCH_C_SDSP )
+    {
+        len = 8;
+        data = GET_RS2C(insn, regs);
+    }
+#endif
+    else
+        return -EOPNOTSUPP;
+
+    info.data = data;
+
+    rc = do_mmio(&info, fault_addr, len);
+    if ( rc )
+        return rc;
+
+    advance_pc(regs, insn_len);
+
+    return 0;
 }
 
 static void handle_guest_page_fault(unsigned long cause,
-- 
2.54.0