[PATCH v4 00/12] Introduce eSPI support

[PATCH v4 00/12] Introduce eSPI support

[Leonid Komarianskyi]

Hello everyone!

This is the fourth version of the patch series to introduce support, which
contains fixes and improvements based on the feedback received in V3.
Mostly, these are minor changes in individual patches, except for the following:
xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers

Now includes additional checks for domains without eSPI support to ensure they
behave as they currently do when reading or writing eSPI registers - RAZ or WI
as it also described in the GICv3.1 documentation, when GICD_TYPER.ESPI==0.

Summarized description:
This patch series adds support for the extended shared peripheral
interrupt (eSPI) range (INTIDs 4096-5119 [2](ranges of INTIDs)) for Xen
and guest domains. The implementation uses a generic approach to handle
eSPIs, similar to regular SPIs, while maintaining compatibility with the
existing SPI range. Functionality remains unchanged for setups that do
not require eSPIs.

The series includes:
1) General refactoring of common IRQ operations with GIC registers to
improve code readability, simplify further maintenance and prepare the
key functions for eSPI implementation.
2) Introducing a new Kconfig option (default n) to enable or disable
eSPI support. Disabling this option prevents unnecessary resource
allocation for setups that do not require eSPIs.
3) Adding additional resources to store required information and operate
with up to 1024 interrupts from eSPI range.
4) Adjusting assertions and checks to pass verification for INTIDs in
the eSPI range.
5) Configuration of eSPI-specific registers during GIC initialization
for systems with GICv3.1+ hardware.
6) Enables eSPI MMIO emulation for vGIC, allowing guest domains to
access and operate within the eSPI's INTIDs.
7) Updating documentation and CHANGELOG to reflect the changes made for eSPI
support.

Also, to simplify reviewing, please find below link to unsquashed patches, that
are on top of every patch, that is changed in the series, compared to V3:
https://github.com/LKomaryanskiy/xen/commits/espi-support-master-upstream-v4-unsquashed/


Changes in V4:
- added a patch for documentation
- individual changes in patches

Link on V3:
- https://lists.xenproject.org/archives/html/xen-devel/2025-08/msg01628.html

Changes in V3:
- added a patch to update CHANGELOG.md
- individual changes in patches

Link on V2:
- https://lists.xenproject.org/archives/html/xen-devel/2025-08/msg00372.html

Changes in V2:
- added 2 more patches to implement helper
  functions for gic/vgic:
  xen/arm: gic: implement helper functions for INTID checks
  xen/arm: vgic: implement helper functions for virq checks
- removed 2 patches:
  xen/arm/irq: allow assignment/releasing of eSPI interrupts
  xen/arm: gic/irq: permit routing of eSPI interrupts to Xen and domains
  since their functionality can be moved to appropriate patches after
  introducing patches with helper functions
- individual changes in patches

Link on V1:
- https://lists.xenproject.org/archives/html/xen-devel/2025-07/msg01809.html


Leonid Komarianskyi (12):
  xen/arm: gicv3: refactor obtaining GIC addresses for common operations
  xen/arm: gic: implement helper functions for INTID checks
  xen/arm: vgic: implement helper functions for virq checks
  xen/arm/irq: add handling for IRQs in the eSPI range
  xen/arm: gicv3: implement handling of GICv3.1 eSPI
  xen/arm/irq: allow eSPI processing in the gic_interrupt function
  xen/arm: gicv3: modify ICH_LR_PHYSICAL_MASK to allow eSPI processing
  xen/arm: vgic: add resource management for extended SPIs
  xen/arm: domain_build/dom0less-build: adjust domains config to support
    eSPIs
  xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers
  doc/man: update description for nr_spis with eSPI
  CHANGELOG.md: add mention of GICv3.1 eSPI support

 CHANGELOG.md                           |   1 +
 docs/man/xl.cfg.5.pod.in               |  13 +-
 xen/arch/arm/Kconfig                   |   8 +
 xen/arch/arm/dom0less-build.c          |   2 +-
 xen/arch/arm/domain_build.c            |   2 +-
 xen/arch/arm/gic-v3.c                  | 194 +++++++++++---
 xen/arch/arm/gic.c                     |   7 +-
 xen/arch/arm/include/asm/gic.h         |  31 +++
 xen/arch/arm/include/asm/gic_v3_defs.h |  40 ++-
 xen/arch/arm/include/asm/irq.h         |  27 ++
 xen/arch/arm/include/asm/vgic.h        |  48 ++++
 xen/arch/arm/irq.c                     |  58 ++++-
 xen/arch/arm/vgic-v3.c                 | 333 +++++++++++++++++++++++--
 xen/arch/arm/vgic.c                    | 223 ++++++++++++++++-
 xen/arch/arm/vgic/vgic.c               |   5 +
 15 files changed, 924 insertions(+), 68 deletions(-)

-- 
2.34.1

[PATCH v4 01/12] xen/arm: gicv3: refactor obtaining GIC addresses for common operations

[Leonid Komarianskyi]

Currently, many common functions perform the same operations to calculate
GIC register addresses. This patch consolidates the similar code into
a separate helper function to improve maintainability and reduce duplication.
This refactoring also simplifies the implementation of eSPI support in future
changes.

Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>

---
Changes in V4:
- no changes

Changes in V3:
- changed panic() in get_addr_by_offset() to printing warning and
  ASSERT_UNREACHABLE()
- added verification of return pointer from get_addr_by_offset() in the
  callers
- moved invocation of get_addr_by_offset() from spinlock guards, since
  it is not necessarry
- added RB from Volodymyr Babchuk

Changes in V2:
- no changes
---
 xen/arch/arm/gic-v3.c          | 114 +++++++++++++++++++++++----------
 xen/arch/arm/include/asm/irq.h |   1 +
 2 files changed, 81 insertions(+), 34 deletions(-)

diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index cd3e1acf79..a959fefebe 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -445,17 +445,67 @@ static void gicv3_dump_state(const struct vcpu *v)
     }
 }
 
+static void __iomem *get_addr_by_offset(struct irq_desc *irqd, u32 offset)
+{
+    switch ( irqd->irq )
+    {
+    case 0 ... (NR_GIC_LOCAL_IRQS - 1):
+        switch ( offset )
+        {
+        case GICD_ISENABLER:
+        case GICD_ICENABLER:
+        case GICD_ISPENDR:
+        case GICD_ICPENDR:
+        case GICD_ISACTIVER:
+        case GICD_ICACTIVER:
+            return (GICD_RDIST_SGI_BASE + offset);
+        case GICD_ICFGR:
+            return (GICD_RDIST_SGI_BASE + GICR_ICFGR1);
+        case GICD_IPRIORITYR:
+            return (GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + irqd->irq);
+        default:
+            break;
+        }
+    case NR_GIC_LOCAL_IRQS ... SPI_MAX_INTID:
+        switch ( offset )
+        {
+        case GICD_ISENABLER:
+        case GICD_ICENABLER:
+        case GICD_ISPENDR:
+        case GICD_ICPENDR:
+        case GICD_ISACTIVER:
+        case GICD_ICACTIVER:
+            return (GICD + offset + (irqd->irq / 32) * 4);
+        case GICD_ICFGR:
+            return (GICD + GICD_ICFGR + (irqd->irq / 16) * 4);
+        case GICD_IROUTER:
+            return (GICD + GICD_IROUTER + irqd->irq * 8);
+        case GICD_IPRIORITYR:
+            return (GICD + GICD_IPRIORITYR + irqd->irq);
+        default:
+            break;
+        }
+    default:
+        break;
+    }
+
+    /* Something went wrong, we shouldn't be able to reach here */
+    printk(XENLOG_WARNING "GICv3: WARNING: Invalid offset 0x%x for IRQ#%d",
+           offset, irqd->irq);
+    ASSERT_UNREACHABLE();
+
+    return NULL;
+}
+
 static void gicv3_poke_irq(struct irq_desc *irqd, u32 offset, bool wait_for_rwp)
 {
     u32 mask = 1U << (irqd->irq % 32);
-    void __iomem *base;
+    void __iomem *addr = get_addr_by_offset(irqd, offset);
 
-    if ( irqd->irq < NR_GIC_LOCAL_IRQS )
-        base = GICD_RDIST_SGI_BASE;
-    else
-        base = GICD;
+    if ( addr == NULL )
+        return;
 
-    writel_relaxed(mask, base + offset + (irqd->irq / 32) * 4);
+    writel_relaxed(mask, addr);
 
     if ( wait_for_rwp )
         gicv3_wait_for_rwp(irqd->irq);
@@ -463,15 +513,12 @@ static void gicv3_poke_irq(struct irq_desc *irqd, u32 offset, bool wait_for_rwp)
 
 static bool gicv3_peek_irq(struct irq_desc *irqd, u32 offset)
 {
-    void __iomem *base;
-    unsigned int irq = irqd->irq;
+    void __iomem *addr = get_addr_by_offset(irqd, offset);
 
-    if ( irq >= NR_GIC_LOCAL_IRQS)
-        base = GICD + (irq / 32) * 4;
-    else
-        base = GICD_RDIST_SGI_BASE;
+    if ( addr == NULL )
+        return false;
 
-    return !!(readl(base + offset) & (1U << (irq % 32)));
+    return !!(readl(addr) & (1U << (irqd->irq % 32)));
 }
 
 static void gicv3_unmask_irq(struct irq_desc *irqd)
@@ -558,30 +605,28 @@ static inline uint64_t gicv3_mpidr_to_affinity(int cpu)
 static void gicv3_set_irq_type(struct irq_desc *desc, unsigned int type)
 {
     uint32_t cfg, actual, edgebit;
-    void __iomem *base;
-    unsigned int irq = desc->irq;
+    void __iomem *addr;
 
     /* SGI's are always edge-triggered not need to call GICD_ICFGR0 */
-    ASSERT(irq >= NR_GIC_SGI);
+    ASSERT(desc->irq >= NR_GIC_SGI);
 
-    spin_lock(&gicv3.lock);
+    addr = get_addr_by_offset(desc, GICD_ICFGR);
+    if ( addr == NULL )
+        return;
 
-    if ( irq >= NR_GIC_LOCAL_IRQS)
-        base = GICD + GICD_ICFGR + (irq / 16) * 4;
-    else
-        base = GICD_RDIST_SGI_BASE + GICR_ICFGR1;
+    spin_lock(&gicv3.lock);
 
-    cfg = readl_relaxed(base);
+    cfg = readl_relaxed(addr);
 
-    edgebit = 2u << (2 * (irq % 16));
+    edgebit = 2u << (2 * (desc->irq % 16));
     if ( type & IRQ_TYPE_LEVEL_MASK )
         cfg &= ~edgebit;
     else if ( type & IRQ_TYPE_EDGE_BOTH )
         cfg |= edgebit;
 
-    writel_relaxed(cfg, base);
+    writel_relaxed(cfg, addr);
 
-    actual = readl_relaxed(base);
+    actual = readl_relaxed(addr);
     if ( ( cfg & edgebit ) ^ ( actual & edgebit ) )
     {
         printk(XENLOG_WARNING "GICv3: WARNING: "
@@ -600,16 +645,13 @@ static void gicv3_set_irq_type(struct irq_desc *desc, unsigned int type)
 static void gicv3_set_irq_priority(struct irq_desc *desc,
                                    unsigned int priority)
 {
-    unsigned int irq = desc->irq;
+    void __iomem *addr = get_addr_by_offset(desc, GICD_IPRIORITYR);
 
-    spin_lock(&gicv3.lock);
-
-    /* Set priority */
-    if ( irq < NR_GIC_LOCAL_IRQS )
-        writeb_relaxed(priority, GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + irq);
-    else
-        writeb_relaxed(priority, GICD + GICD_IPRIORITYR + irq);
+    if ( addr == NULL )
+        return;
 
+    spin_lock(&gicv3.lock);
+    writeb_relaxed(priority, addr);
     spin_unlock(&gicv3.lock);
 }
 
@@ -1273,6 +1315,10 @@ static void gicv3_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
 {
     unsigned int cpu;
     uint64_t affinity;
+    void __iomem *addr = get_addr_by_offset(desc, GICD_IROUTER);
+
+    if ( addr == NULL )
+        return;
 
     ASSERT(!cpumask_empty(mask));
 
@@ -1284,7 +1330,7 @@ static void gicv3_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
     affinity &= ~GICD_IROUTER_SPI_MODE_ANY;
 
     if ( desc->irq >= NR_GIC_LOCAL_IRQS )
-        writeq_relaxed_non_atomic(affinity, (GICD + GICD_IROUTER + desc->irq * 8));
+        writeq_relaxed_non_atomic(affinity, addr);
 
     spin_unlock(&gicv3.lock);
 }
diff --git a/xen/arch/arm/include/asm/irq.h b/xen/arch/arm/include/asm/irq.h
index fce7e42a33..5bc6475eb4 100644
--- a/xen/arch/arm/include/asm/irq.h
+++ b/xen/arch/arm/include/asm/irq.h
@@ -29,6 +29,7 @@ struct arch_irq_desc {
  */
 #define NR_IRQS		1024
 
+#define SPI_MAX_INTID   1019
 #define LPI_OFFSET      8192
 
 /* LPIs are always numbered starting at 8192, so 0 is a good invalid case. */
-- 
2.34.1

[PATCH v4 02/12] xen/arm: gic: implement helper functions for INTID checks

[Leonid Komarianskyi]

Introduced two new helper functions: gic_is_valid_line and
gic_is_spi. The first function helps determine whether an IRQ
number is less than the number of lines supported by hardware. The
second function additionally checks if the IRQ number falls within the
SPI range. Also, updated the appropriate checks to use these new helper
functions.

The current checks for the real GIC are very similar to those for the
vGIC but serve a different purpose. For GIC-related code, the interrupt
numbers should be validated based on whether the hardware can operate
with such interrupts. On the other hand, for the vGIC, the indexes must
also be verified to ensure they are available for a specific domain. The
first reason for introducing these helper functions is to avoid
potential confusion with vGIC-related checks. The second reason is to
consolidate similar code into separate functions, which can be more
easily extended by additional conditions, e.g., when implementing
extended SPI interrupts.

The changes, which replace open-coded checks with the use of the new
helper functions, do not introduce any functional changes, as the helper
functions follow the current IRQ index verification logic.

Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>

---
Changes in V4:
- removed redundant parentheses
- added reviewed-by from Volodymyr Babchuk

Changes in V3:
- renamed gic_is_valid_irq to gic_is_valid_line and gic_is_shared_irq to
  gic_is_spi
- updated commit message

Changes in V2:
- introduced this patch

Changes for V4:

Changes in V4:
- removed redundant parentheses
- added reviewed-by from Volodymyr Babchuk
---
 xen/arch/arm/gic.c             | 2 +-
 xen/arch/arm/include/asm/gic.h | 9 +++++++++
 xen/arch/arm/irq.c             | 2 +-
 3 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index e80fe0ca24..9220eef6ea 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -111,7 +111,7 @@ static void gic_set_irq_priority(struct irq_desc *desc, unsigned int priority)
 void gic_route_irq_to_xen(struct irq_desc *desc, unsigned int priority)
 {
     ASSERT(priority <= 0xff);     /* Only 8 bits of priority */
-    ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
+    ASSERT(gic_is_valid_line(desc->irq));/* Can't route interrupts that don't exist */
     ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
     ASSERT(spin_is_locked(&desc->lock));
 
diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/asm/gic.h
index 541f0eeb80..3fcee42675 100644
--- a/xen/arch/arm/include/asm/gic.h
+++ b/xen/arch/arm/include/asm/gic.h
@@ -306,6 +306,15 @@ extern void gic_dump_vgic_info(struct vcpu *v);
 
 /* Number of interrupt lines */
 extern unsigned int gic_number_lines(void);
+static inline bool gic_is_valid_line(unsigned int irq)
+{
+    return irq < gic_number_lines();
+}
+
+static inline bool gic_is_spi(unsigned int irq)
+{
+    return irq >= NR_LOCAL_IRQS && gic_is_valid_line(irq);
+}
 
 /* IRQ translation function for the device tree */
 int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 03fbb90c6c..7dd5a2a453 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -415,7 +415,7 @@ err:
 bool is_assignable_irq(unsigned int irq)
 {
     /* For now, we can only route SPIs to the guest */
-    return (irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines());
+    return gic_is_spi(irq);
 }
 
 /*
-- 
2.34.1

[PATCH v4 03/12] xen/arm: vgic: implement helper functions for virq checks

[Leonid Komarianskyi]

Introduced two new helper functions for vGIC: vgic_is_valid_line and
vgic_is_spi. The functions are similar to the newly introduced
gic_is_valid_line and gic_is_spi, but they verify whether a vIRQ
is available for a specific domain, while GIC-specific functions
validate INTIDs for the real GIC hardware. For example, the GIC may
support all 992 SPI lines, but the domain may use only some part of them
(e.g., 640), depending on the highest IRQ number defined in the domain
configuration. Therefore, for vGIC-related code and checks, the
appropriate functions should be used. Also, updated the appropriate
checks to use these new helper functions.

The purpose of introducing new helper functions for vGIC is essentially
the same as for GIC: to avoid potential confusion with GIC-related
checks and to consolidate similar code into separate functions, which
can be more easily extended by additional conditions, e.g., when
implementing extended SPI interrupts.

Only the validation change in vgic_inject_irq may affect existing
functionality, as it currently checks whether the vIRQ is less than or
equal to vgic_num_irqs. Since IRQ indexes start from 0 (where 32 is the
first SPI), the check should behave consistently with similar logic in
other places and should check if the vIRQ number is less than
vgic_num_irqs. The remaining changes, which replace open-coded checks
with the use of these new helper functions, do not introduce any
functional changes, as the helper functions follow the current vIRQ
index verification logic.

Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>

---
Changes in V4:
- removed redundant parentheses

Changes in V3:
- renamed vgic_is_valid_irq to vgic_is_valid_line and vgic_is_shared_irq
  to vgic_is_spi
- added vgic_is_valid_line implementation for new-vgic, because
  vgic_is_valid_line is called from generic code. It is necessary to fix
  the build for new-vgic.
- updated commit message

Changes in V2:
- introduced this patch
---
 xen/arch/arm/gic.c              |  3 +--
 xen/arch/arm/include/asm/vgic.h |  7 +++++++
 xen/arch/arm/irq.c              |  4 ++--
 xen/arch/arm/vgic.c             | 10 ++++++++--
 xen/arch/arm/vgic/vgic.c        |  5 +++++
 5 files changed, 23 insertions(+), 6 deletions(-)

diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 9220eef6ea..b88237ccda 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -133,8 +133,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
 
     ASSERT(spin_is_locked(&desc->lock));
     /* Caller has already checked that the IRQ is an SPI */
-    ASSERT(virq >= 32);
-    ASSERT(virq < vgic_num_irqs(d));
+    ASSERT(vgic_is_spi(d, virq));
     ASSERT(!is_lpi(virq));
 
     ret = vgic_connect_hw_irq(d, NULL, virq, desc, true);
diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
index 35c0c6a8b0..3e7cbbb196 100644
--- a/xen/arch/arm/include/asm/vgic.h
+++ b/xen/arch/arm/include/asm/vgic.h
@@ -335,6 +335,13 @@ extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
 /* Default number of vGIC SPIs. 32 are substracted to cover local IRQs. */
 #define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
 
+extern bool vgic_is_valid_line(struct domain *d, unsigned int virq);
+
+static inline bool vgic_is_spi(struct domain *d, unsigned int virq)
+{
+    return virq >= NR_LOCAL_IRQS && vgic_is_valid_line(d, virq);
+}
+
 /*
  * Allocate a guest VIRQ
  *  - spi == 0 => allocate a PPI. It will be the same on every vCPU
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 7dd5a2a453..b8eccfc924 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -442,7 +442,7 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
     unsigned long flags;
     int retval = 0;
 
-    if ( virq >= vgic_num_irqs(d) )
+    if ( !vgic_is_valid_line(d, virq) )
     {
         printk(XENLOG_G_ERR
                "the vIRQ number %u is too high for domain %u (max = %u)\n",
@@ -560,7 +560,7 @@ int release_guest_irq(struct domain *d, unsigned int virq)
     int ret;
 
     /* Only SPIs are supported */
-    if ( virq < NR_LOCAL_IRQS || virq >= vgic_num_irqs(d) )
+    if ( !vgic_is_spi(d, virq) )
         return -EINVAL;
 
     desc = vgic_get_hw_irq_desc(d, NULL, virq);
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index c563ba93af..2bbf4d99aa 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -24,6 +24,12 @@
 #include <asm/gic.h>
 #include <asm/vgic.h>
 
+
+bool vgic_is_valid_line(struct domain *d, unsigned int virq)
+{
+    return virq < vgic_num_irqs(d);
+}
+
 static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
                                                   unsigned int rank)
 {
@@ -582,7 +588,7 @@ void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
     if ( !v )
     {
         /* The IRQ needs to be an SPI if no vCPU is specified. */
-        ASSERT(virq >= 32 && virq <= vgic_num_irqs(d));
+        ASSERT(vgic_is_spi(d, virq));
 
         v = vgic_get_target_vcpu(d->vcpu[0], virq);
     };
@@ -659,7 +665,7 @@ bool vgic_emulate(struct cpu_user_regs *regs, union hsr hsr)
 
 bool vgic_reserve_virq(struct domain *d, unsigned int virq)
 {
-    if ( virq >= vgic_num_irqs(d) )
+    if ( !vgic_is_valid_line(d, virq) )
         return false;
 
     return !test_and_set_bit(virq, d->arch.vgic.allocated_irqs);
diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
index 6cabd0496d..b2c0e1873a 100644
--- a/xen/arch/arm/vgic/vgic.c
+++ b/xen/arch/arm/vgic/vgic.c
@@ -718,6 +718,11 @@ bool vgic_reserve_virq(struct domain *d, unsigned int virq)
     return !test_and_set_bit(virq, d->arch.vgic.allocated_irqs);
 }
 
+bool vgic_is_valid_line(struct domain *d, unsigned int virq)
+{
+    return virq < vgic_num_irqs(d);
+}
+
 int vgic_allocate_virq(struct domain *d, bool spi)
 {
     int first, end;
-- 
2.34.1

[PATCH v4 04/12] xen/arm/irq: add handling for IRQs in the eSPI range

[Leonid Komarianskyi]

Currently, Xen does not support eSPI interrupts, leading
to a data abort when such interrupts are defined in the DTS.

This patch introduces a separate array to initialize up to
1024 interrupt descriptors in the eSPI range and adds the
necessary defines and helper function. These changes lay the
groundwork for future implementation of full eSPI interrupt
support. As this GICv3.1 feature is not required by all vendors,
all changes are guarded by ifdefs, depending on the corresponding
Kconfig option.

Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>

---
Changes in V4:
- removed redundant line with 'default n' in Kconfig, as it is disabled
  by default, without explicit specification
- added reviewed-by from Volodymyr Babchuk

Changes in V3:
- introduced a new define NR_ESPI_IRQS to avoid confusion, like in the
  case of using NR_IRQS for espi_desc array
- implemented helper functions espi_to_desc and init_espi_data to make
  it possible to add stubs with the same name, and as a result, reduce
  the number of #ifdefs
- disable CONFIG_GICV3_ESPI default value to n

Changes in V2:
- use (ESPI_MAX_INTID + 1) instead of (ESPI_BASE_INTID + NR_IRQS)
- remove unnecessary comment for nr_irqs initialization
---
 xen/arch/arm/Kconfig           |  8 ++++++
 xen/arch/arm/include/asm/irq.h | 26 +++++++++++++++++
 xen/arch/arm/irq.c             | 52 +++++++++++++++++++++++++++++++++-
 3 files changed, 85 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 17df147b25..43b05533b1 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -135,6 +135,14 @@ config GICV3
 	  Driver for the ARM Generic Interrupt Controller v3.
 	  If unsure, use the default setting.
 
+config GICV3_ESPI
+	bool "Extended SPI range support"
+	depends on GICV3 && !NEW_VGIC
+	help
+	  Allow Xen and domains to use interrupt numbers from the extended SPI
+	  range, from 4096 to 5119. This feature is introduced in GICv3.1
+	  architecture.
+
 config HAS_ITS
         bool "GICv3 ITS MSI controller support (UNSUPPORTED)" if UNSUPPORTED
         depends on GICV3 && !NEW_VGIC && !ARM_32
diff --git a/xen/arch/arm/include/asm/irq.h b/xen/arch/arm/include/asm/irq.h
index 5bc6475eb4..221dbf23a2 100644
--- a/xen/arch/arm/include/asm/irq.h
+++ b/xen/arch/arm/include/asm/irq.h
@@ -32,6 +32,15 @@ struct arch_irq_desc {
 #define SPI_MAX_INTID   1019
 #define LPI_OFFSET      8192
 
+#ifdef CONFIG_GICV3_ESPI
+#define ESPI_BASE_INTID 4096
+#define ESPI_MAX_INTID  5119
+#define NR_ESPI_IRQS    1024
+
+#define ESPI_INTID2IDX(intid) ((intid) - ESPI_BASE_INTID)
+#define ESPI_IDX2INTID(idx)   ((idx) + ESPI_BASE_INTID)
+#endif
+
 /* LPIs are always numbered starting at 8192, so 0 is a good invalid case. */
 #define INVALID_LPI     0
 
@@ -39,7 +48,15 @@ struct arch_irq_desc {
 #define INVALID_IRQ     1023
 
 extern const unsigned int nr_irqs;
+#ifdef CONFIG_GICV3_ESPI
+/*
+ * This will also cover the eSPI range, as some critical devices
+ * for booting Xen (e.g., serial) may use this type of interrupts.
+ */
+#define nr_static_irqs (ESPI_MAX_INTID + 1)
+#else
 #define nr_static_irqs NR_IRQS
+#endif
 
 struct irq_desc;
 struct irqaction;
@@ -55,6 +72,15 @@ static inline bool is_lpi(unsigned int irq)
     return irq >= LPI_OFFSET;
 }
 
+static inline bool is_espi(unsigned int irq)
+{
+#ifdef CONFIG_GICV3_ESPI
+    return (irq >= ESPI_BASE_INTID && irq <= ESPI_MAX_INTID);
+#else
+    return false;
+#endif
+}
+
 #define domain_pirq_to_irq(d, pirq) (pirq)
 
 bool is_assignable_irq(unsigned int irq);
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index b8eccfc924..adb5e49ea3 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -19,7 +19,11 @@
 #include <asm/gic.h>
 #include <asm/vgic.h>
 
+#ifdef CONFIG_GICV3_ESPI
+const unsigned int nr_irqs = ESPI_MAX_INTID + 1;
+#else
 const unsigned int nr_irqs = NR_IRQS;
+#endif
 
 static unsigned int local_irqs_type[NR_LOCAL_IRQS];
 static DEFINE_SPINLOCK(local_irqs_type_lock);
@@ -46,6 +50,49 @@ void irq_end_none(struct irq_desc *irq)
 }
 
 static irq_desc_t irq_desc[NR_IRQS - NR_LOCAL_IRQS];
+#ifdef CONFIG_GICV3_ESPI
+static irq_desc_t espi_desc[NR_ESPI_IRQS];
+
+static struct irq_desc *espi_to_desc(unsigned int irq)
+{
+    return &espi_desc[ESPI_INTID2IDX(irq)];
+}
+
+static int __init init_espi_data(void)
+{
+    int irq;
+
+    for ( irq = ESPI_BASE_INTID; irq <= ESPI_MAX_INTID; irq++ )
+    {
+        struct irq_desc *desc = irq_to_desc(irq);
+        int rc = init_one_irq_desc(desc);
+
+        if ( rc )
+            return rc;
+
+        desc->irq = irq;
+        desc->action  = NULL;
+    }
+
+    return 0;
+}
+#else
+/*
+ * This function is stub and will not be called if CONFIG_GICV3_ESPI=n,
+ * because in this case, is_espi will always return false.
+ */
+static struct irq_desc *espi_to_desc(unsigned int irq)
+{
+    ASSERT_UNREACHABLE();
+    return NULL;
+}
+
+static int __init init_espi_data(void)
+{
+    return 0;
+}
+#endif
+
 static DEFINE_PER_CPU(irq_desc_t[NR_LOCAL_IRQS], local_irq_desc);
 
 struct irq_desc *__irq_to_desc(unsigned int irq)
@@ -53,6 +100,9 @@ struct irq_desc *__irq_to_desc(unsigned int irq)
     if ( irq < NR_LOCAL_IRQS )
         return &this_cpu(local_irq_desc)[irq];
 
+    if ( is_espi(irq) )
+        return espi_to_desc(irq);
+
     return &irq_desc[irq-NR_LOCAL_IRQS];
 }
 
@@ -79,7 +129,7 @@ static int __init init_irq_data(void)
         desc->action  = NULL;
     }
 
-    return 0;
+    return init_espi_data();
 }
 
 static int init_local_irq_data(unsigned int cpu)
-- 
2.34.1

[PATCH v4 05/12] xen/arm: gicv3: implement handling of GICv3.1 eSPI

[Leonid Komarianskyi]

Introduced appropriate register definitions, helper macros,
and initialization of required GICv3.1 distributor registers
to support eSPI. This type of interrupt is handled in the
same way as regular SPI interrupts, with the following
differences:

1) eSPIs can have up to 1024 interrupts, starting from the
beginning of the range, whereas regular SPIs use INTIDs from
32 to 1019, totaling 988 interrupts;
2) eSPIs start at INTID 4096, necessitating additional interrupt
index conversion during register operations.

In case if appropriate config is disabled, or GIC HW doesn't
support eSPI, the existing functionality will remain the same.

Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>

---
Changes in V4:
- added offsets for GICD_IGRPMODRnE and GICD_NSACRnE that are required
  for vGIC emulation
- added a log banner with eSPI information, similar to the one for
  regular SPI
- added newline after ifdef and before gic_is_valid_line
- added reviewed-by from Volodymyr Babchuk

Changes in V3:
- add __init attribute to gicv3_dist_espi_common_init
- change open-codded eSPI register initialization to the appropriate
  gen-mask macro
- fixed formatting for lines with more than 80 symbols
- introduced gicv3_dist_espi_init_aff to be able to use stubs in case of
  CONFIG_GICV3_ESPI disabled
- renamed parameter in the GICD_TYPER_ESPI_RANGE macro to espi_range
  (name was taken from GIC specification) to avoid confusion
- changed type for i variable to unsigned int since it cannot be
  negative

Changes in V2:
- move gic_number_espis function from
  [PATCH 08/10] xen/arm: vgic: add resource management for extended SPIs
  to use it in the newly introduced gic_is_valid_espi
- add gic_is_valid_espi which checks if IRQ number is in supported
  by HW eSPI range
- update gic_is_valid_irq conditions to allow operations with eSPIs

Changes for V4:

Changes in V4:
- added offsets for GICD_IGRPMODRnE and GICD_NSACRnE that are required
  for vGIC emulation
- added newline after ifdef and before gic_is_valid_line
- added reviewed-by from Volodymyr Babchuk
- added a log banner with eSPI information, similar to the one for
  regular SPI
---
 xen/arch/arm/gic-v3.c                  | 82 ++++++++++++++++++++++++++
 xen/arch/arm/include/asm/gic.h         | 22 +++++++
 xen/arch/arm/include/asm/gic_v3_defs.h | 38 ++++++++++++
 3 files changed, 142 insertions(+)

diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index a959fefebe..b939a1f490 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -485,6 +485,36 @@ static void __iomem *get_addr_by_offset(struct irq_desc *irqd, u32 offset)
         default:
             break;
         }
+#ifdef CONFIG_GICV3_ESPI
+    case ESPI_BASE_INTID ... ESPI_MAX_INTID:
+    {
+        u32 irq_index = ESPI_INTID2IDX(irqd->irq);
+
+        switch ( offset )
+        {
+        case GICD_ISENABLER:
+            return (GICD + GICD_ISENABLERnE + (irq_index / 32) * 4);
+        case GICD_ICENABLER:
+            return (GICD + GICD_ICENABLERnE + (irq_index / 32) * 4);
+        case GICD_ISPENDR:
+            return (GICD + GICD_ISPENDRnE + (irq_index / 32) * 4);
+        case GICD_ICPENDR:
+            return (GICD + GICD_ICPENDRnE + (irq_index / 32) * 4);
+        case GICD_ISACTIVER:
+            return (GICD + GICD_ISACTIVERnE + (irq_index / 32) * 4);
+        case GICD_ICACTIVER:
+            return (GICD + GICD_ICACTIVERnE + (irq_index / 32) * 4);
+        case GICD_ICFGR:
+            return (GICD + GICD_ICFGRnE + (irq_index / 16) * 4);
+        case GICD_IROUTER:
+            return (GICD + GICD_IROUTERnE + irq_index * 8);
+        case GICD_IPRIORITYR:
+            return (GICD + GICD_IPRIORITYRnE + irq_index);
+        default:
+            break;
+        }
+    }
+#endif
     default:
         break;
     }
@@ -655,6 +685,54 @@ static void gicv3_set_irq_priority(struct irq_desc *desc,
     spin_unlock(&gicv3.lock);
 }
 
+#ifdef CONFIG_GICV3_ESPI
+unsigned int gic_number_espis(void)
+{
+    return gic_hw_ops->info->nr_espi;
+}
+
+static void __init gicv3_dist_espi_common_init(uint32_t type)
+{
+    unsigned int espi_nr, i;
+
+    espi_nr = min(1024U, GICD_TYPER_ESPIS_NUM(type));
+    gicv3_info.nr_espi = espi_nr;
+    /* The GIC HW doesn't support eSPI, so we can leave from here */
+    if ( gicv3_info.nr_espi == 0 )
+        return;
+
+    printk("GICv3: %d eSPI lines\n", gicv3_info.nr_espi);
+
+    for ( i = 0; i < espi_nr; i += 16 )
+        writel_relaxed(0, GICD + GICD_ICFGRnE + (i / 16) * 4);
+
+    for ( i = 0; i < espi_nr; i += 4 )
+        writel_relaxed(GIC_PRI_IRQ_ALL,
+                       GICD + GICD_IPRIORITYRnE + (i / 4) * 4);
+
+    for ( i = 0; i < espi_nr; i += 32 )
+    {
+        writel_relaxed(GENMASK(31, 0), GICD + GICD_ICENABLERnE + (i / 32) * 4);
+        writel_relaxed(GENMASK(31, 0), GICD + GICD_ICACTIVERnE + (i / 32) * 4);
+    }
+
+    for ( i = 0; i < espi_nr; i += 32 )
+        writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPRnE + (i / 32) * 4);
+}
+
+static void __init gicv3_dist_espi_init_aff(uint64_t affinity)
+{
+    unsigned int i;
+
+    for ( i = 0; i < gicv3_info.nr_espi; i++ )
+        writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTERnE + i * 8);
+}
+#else
+static void __init gicv3_dist_espi_common_init(uint32_t type) { }
+
+static void __init gicv3_dist_espi_init_aff(uint64_t affinity) { }
+#endif
+
 static void __init gicv3_dist_init(void)
 {
     uint32_t type;
@@ -700,6 +778,8 @@ static void __init gicv3_dist_init(void)
     for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i += 32 )
         writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPR + (i / 32) * 4);
 
+    gicv3_dist_espi_common_init(type);
+
     gicv3_dist_wait_for_rwp();
 
     /* Turn on the distributor */
@@ -713,6 +793,8 @@ static void __init gicv3_dist_init(void)
 
     for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i++ )
         writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTER + i * 8);
+
+    gicv3_dist_espi_init_aff(affinity);
 }
 
 static int gicv3_enable_redist(void)
diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/asm/gic.h
index 3fcee42675..1e747dcd99 100644
--- a/xen/arch/arm/include/asm/gic.h
+++ b/xen/arch/arm/include/asm/gic.h
@@ -306,8 +306,26 @@ extern void gic_dump_vgic_info(struct vcpu *v);
 
 /* Number of interrupt lines */
 extern unsigned int gic_number_lines(void);
+#ifdef CONFIG_GICV3_ESPI
+extern unsigned int gic_number_espis(void);
+
+static inline bool gic_is_valid_espi(unsigned int irq)
+{
+    return (irq >= ESPI_BASE_INTID &&
+            irq < ESPI_IDX2INTID(gic_number_espis()));
+}
+#else
+static inline bool gic_is_valid_espi(unsigned int irq)
+{
+    return false;
+}
+#endif
+
 static inline bool gic_is_valid_line(unsigned int irq)
 {
+    if ( gic_is_valid_espi(irq) )
+        return true;
+
     return irq < gic_number_lines();
 }
 
@@ -325,6 +343,10 @@ struct gic_info {
     enum gic_version hw_version;
     /* Number of GIC lines supported */
     unsigned int nr_lines;
+#ifdef CONFIG_GICV3_ESPI
+    /* Number of GIC eSPI supported */
+    unsigned int nr_espi;
+#endif
     /* Number of LR registers */
     uint8_t nr_lrs;
     /* Maintenance irq number */
diff --git a/xen/arch/arm/include/asm/gic_v3_defs.h b/xen/arch/arm/include/asm/gic_v3_defs.h
index 2af093e774..c10db9bd05 100644
--- a/xen/arch/arm/include/asm/gic_v3_defs.h
+++ b/xen/arch/arm/include/asm/gic_v3_defs.h
@@ -37,6 +37,44 @@
 #define GICD_IROUTER1019             (0x7FD8)
 #define GICD_PIDR2                   (0xFFE8)
 
+#ifdef CONFIG_GICV3_ESPI
+/* Additional registers for GICv3.1 */
+#define GICD_IGROUPRnE               (0x1000)
+#define GICD_IGROUPRnEN              (0x107C)
+#define GICD_ISENABLERnE             (0x1200)
+#define GICD_ISENABLERnEN            (0x127C)
+#define GICD_ICENABLERnE             (0x1400)
+#define GICD_ICENABLERnEN            (0x147C)
+#define GICD_ISPENDRnE               (0x1600)
+#define GICD_ISPENDRnEN              (0x167C)
+#define GICD_ICPENDRnE               (0x1800)
+#define GICD_ICPENDRnEN              (0x187C)
+#define GICD_ISACTIVERnE             (0x1A00)
+#define GICD_ISACTIVERnEN            (0x1A7C)
+#define GICD_ICACTIVERnE             (0x1C00)
+#define GICD_ICACTIVERnEN            (0x1C7C)
+#define GICD_IPRIORITYRnE            (0x2000)
+#define GICD_IPRIORITYRnEN           (0x23FC)
+#define GICD_ICFGRnE                 (0x3000)
+#define GICD_ICFGRnEN                (0x30FC)
+#define GICD_IGRPMODRnE              (0x3400)
+#define GICD_IGRPMODRnEN             (0x347C)
+#define GICD_NSACRnE                 (0x3600)
+#define GICD_NSACRnEN                (0x36FC)
+#define GICD_IROUTERnE               (0x8000)
+#define GICD_IROUTERnEN              (0x9FFC)
+
+#define GICD_TYPER_ESPI_SHIFT        8
+#define GICD_TYPER_ESPI_RANGE_SHIFT  27
+#define GICD_TYPER_ESPI_RANGE_MASK   (0x1F)
+#define GICD_TYPER_ESPI              (1U << GICD_TYPER_ESPI_SHIFT)
+#define GICD_TYPER_ESPI_RANGE(espi_range) ((((espi_range) & \
+        GICD_TYPER_ESPI_RANGE_MASK) + 1) * 32)
+#define GICD_TYPER_ESPIS_NUM(typer)    \
+        (((typer) & GICD_TYPER_ESPI) ? \
+        GICD_TYPER_ESPI_RANGE((typer) >> GICD_TYPER_ESPI_RANGE_SHIFT) : 0)
+#endif
+
 /* Common between GICD_PIDR2 and GICR_PIDR2 */
 #define GIC_PIDR2_ARCH_MASK         (0xf0)
 #define GIC_PIDR2_ARCH_GICv3        (0x30)
-- 
2.34.1

[PATCH v4 06/12] xen/arm/irq: allow eSPI processing in the gic_interrupt function

[Leonid Komarianskyi]

The gic_interrupt function is the main handler for processing IRQs.
Currently, due to restrictive checks, it does not process interrupt
numbers greater than 1019. This patch updates the condition to allow
the handling of interrupts from the eSPI range.

Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>

---
Changes in V4:
- fixed commit message
- added reviewed-by from Volodymyr Babchuk

Changes in V3:
- no changes

Changes in V2:
- no changes
---
 xen/arch/arm/gic.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index b88237ccda..634b77c987 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -341,7 +341,7 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
         /* Reading IRQ will ACK it */
         irq = gic_hw_ops->read_irq();
 
-        if ( likely(irq >= GIC_SGI_STATIC_MAX && irq < 1020) )
+        if ( likely(irq >= GIC_SGI_STATIC_MAX && irq < 1020) || is_espi(irq) )
         {
             isb();
             do_IRQ(regs, irq, is_fiq);
-- 
2.34.1

[PATCH v4 07/12] xen/arm: gicv3: modify ICH_LR_PHYSICAL_MASK to allow eSPI processing

[Leonid Komarianskyi]

To properly deactivate guest interrupts and allow them to be retriggered
after the initial trigger, the LR needs to be updated. The current
implementation ignores interrupts outside the range specified by the mask
0x3FF, which only covers IRQ numbers up to 1023. To enable processing of
eSPI interrupts, this patch updates the mask to 0x13FF.

Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>

---
Changes in V4:
- added reviewed-by from Volodymyr Babchuk

Changes in V3:
- no changes

Changes in V2:
- remove unnecessary CONFIG_GICV3_ESPI ifdef guard
---
 xen/arch/arm/include/asm/gic_v3_defs.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/arm/include/asm/gic_v3_defs.h b/xen/arch/arm/include/asm/gic_v3_defs.h
index c10db9bd05..eedb7af1fe 100644
--- a/xen/arch/arm/include/asm/gic_v3_defs.h
+++ b/xen/arch/arm/include/asm/gic_v3_defs.h
@@ -211,7 +211,7 @@
 #define ICH_LR_VIRTUAL_SHIFT         0
 #define ICH_LR_CPUID_MASK            0x7
 #define ICH_LR_CPUID_SHIFT           10
-#define ICH_LR_PHYSICAL_MASK         0x3ff
+#define ICH_LR_PHYSICAL_MASK         0x13ff
 #define ICH_LR_PHYSICAL_SHIFT        32
 #define ICH_LR_STATE_MASK            0x3
 #define ICH_LR_STATE_SHIFT           62
-- 
2.34.1

[PATCH v4 08/12] xen/arm: vgic: add resource management for extended SPIs

[Leonid Komarianskyi]

This change introduces resource management in the VGIC to handle
extended SPIs introduced in GICv3.1. The pending_irqs and
allocated_irqs arrays are resized to support the required
number of eSPIs, based on what is supported by the hardware and
requested by the guest. A new field, ext_shared_irqs, is added
to the VGIC structure to store information about eSPIs, similar
to how shared_irqs is used for regular SPIs.

Since the eSPI range starts at INTID 4096 and INTIDs between 1025
and 4095 are reserved, helper macros are introduced to simplify the
transformation of indices and to enable easier access to eSPI-specific
resources. These changes prepare the VGIC for processing eSPIs as
required by future functionality.

The initialization and deinitialization paths for vgic have been updated
to allocate and free these resources appropriately. Additionally,
updated handling of INTIDs greater than 1024, passed from the toolstack
during domain creation, and verification logic ensures only valid SPI or
eSPI INTIDs are used.

The existing SPI behavior remains unaffected when guests do not request
eSPIs, GIC hardware does not support them, or the CONFIG_GICV3_ESPI
option is disabled.

Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>

---
Changes in V4:
- added has_espi field to simplify determining whether a domain is able
  to operate with eSPI
- fixed formatting issues and misspellings

Changes in V3:
- fixed formatting for lines with more than 80 symbols
- introduced helper functions to be able to use stubs in case of
  CONFIG_GICV3_ESPI disabled, and as a result, reduce the number of
  #ifdefs
- fixed checks for nr_spis in domain_vgic_init
- updated comment about nr_spis adjustments with dom0less mention
- moved comment with additional explanations before checks
- used unsigned int for indexes since they cannot be negative
- removed unnecessary parentheses
- move vgic_ext_rank_offset to the below ifdef guard, to reduce the
  number of ifdefs

Changes in V2:
- change is_espi_rank to is_valid_espi_rank to verify whether the array
  element ext_shared_irqs exists. The previous version, is_espi_rank,
  only checked if the rank index was less than the maximum possible eSPI
  rank index, but this could potentially result in accessing a
  non-existing array element. To address this, is_valid_espi_rank was
  introduced, which ensures that the required eSPI rank exists
- move gic_number_espis to
  xen/arm: gicv3: implement handling of GICv3.1 eSPI
- update vgic_is_valid_irq checks to allow operating with eSPIs
- remove redundant newline in vgic_allocate_virq
---
 xen/arch/arm/include/asm/vgic.h |  20 +++
 xen/arch/arm/vgic.c             | 213 +++++++++++++++++++++++++++++++-
 2 files changed, 230 insertions(+), 3 deletions(-)

diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
index 3e7cbbb196..fb4cea73eb 100644
--- a/xen/arch/arm/include/asm/vgic.h
+++ b/xen/arch/arm/include/asm/vgic.h
@@ -146,6 +146,12 @@ struct vgic_dist {
     int nr_spis; /* Number of SPIs */
     unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
     struct vgic_irq_rank *shared_irqs;
+#ifdef CONFIG_GICV3_ESPI
+    struct vgic_irq_rank *ext_shared_irqs;
+    int nr_espis; /* Number of extended SPIs */
+    /* To simplify determining whether a domain is able to operate with eSPI */
+    bool has_espi;
+#endif
     /*
      * SPIs are domain global, SGIs and PPIs are per-VCPU and stored in
      * struct arch_vcpu.
@@ -243,6 +249,14 @@ struct vgic_ops {
 /* Number of ranks of interrupt registers for a domain */
 #define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_spis+31)/32)
 
+#ifdef CONFIG_GICV3_ESPI
+#define DOMAIN_NR_EXT_RANKS(d) (((d)->arch.vgic.nr_espis+31)/32)
+#define EXT_RANK_MIN (ESPI_BASE_INTID/32)
+#define EXT_RANK_MAX ((ESPI_MAX_INTID+31)/32)
+#define EXT_RANK_NUM2IDX(num) ((num)-EXT_RANK_MIN)
+#define EXT_RANK_IDX2NUM(idx) ((idx)+EXT_RANK_MIN)
+#endif
+
 #define vgic_lock(v)   spin_lock_irq(&(v)->domain->arch.vgic.lock)
 #define vgic_unlock(v) spin_unlock_irq(&(v)->domain->arch.vgic.lock)
 
@@ -302,6 +316,12 @@ extern struct vgic_irq_rank *vgic_rank_offset(struct vcpu *v,
                                               unsigned int b,
                                               unsigned int n,
                                               unsigned int s);
+#ifdef CONFIG_GICV3_ESPI
+extern struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v,
+                                                  unsigned int b,
+                                                  unsigned int n,
+                                                  unsigned int s);
+#endif
 extern struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq);
 extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned int n);
 extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, unsigned int n);
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 2bbf4d99aa..f4b80cb05f 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -27,9 +27,82 @@
 
 bool vgic_is_valid_line(struct domain *d, unsigned int virq)
 {
+#ifdef CONFIG_GICV3_ESPI
+    if ( virq >= ESPI_BASE_INTID &&
+         virq < ESPI_IDX2INTID(d->arch.vgic.nr_espis) )
+        return true;
+#endif
+
     return virq < vgic_num_irqs(d);
 }
 
+#ifdef CONFIG_GICV3_ESPI
+/*
+ * Since eSPI indexes start from 4096 and numbers from 1024 to
+ * 4095 are forbidden, we need to check both lower and upper
+ * limits for ranks.
+ */
+static inline bool is_valid_espi_rank(struct domain *d, unsigned int rank)
+{
+    return rank >= EXT_RANK_MIN &&
+           EXT_RANK_NUM2IDX(rank) < DOMAIN_NR_EXT_RANKS(d);
+}
+
+static inline struct vgic_irq_rank *vgic_get_espi_rank(struct vcpu *v,
+                                                       unsigned int rank)
+{
+    return &v->domain->arch.vgic.ext_shared_irqs[EXT_RANK_NUM2IDX(rank)];
+}
+
+static inline bool vgic_reserve_espi_virq(struct domain *d, unsigned int virq)
+{
+    return !test_and_set_bit(ESPI_INTID2IDX(virq) + vgic_num_irqs(d),
+                             d->arch.vgic.allocated_irqs);
+}
+
+static void arch_move_espis(struct vcpu *v)
+{
+    const cpumask_t *cpu_mask = cpumask_of(v->processor);
+    struct domain *d = v->domain;
+    struct pending_irq *p;
+    struct vcpu *v_target;
+    unsigned int i;
+
+    for ( i = ESPI_BASE_INTID;
+          i < EXT_RANK_IDX2NUM(d->arch.vgic.nr_espis); i++ )
+    {
+        v_target = vgic_get_target_vcpu(v, i);
+        p = irq_to_pending(v_target, i);
+
+        if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) )
+            irq_set_affinity(p->desc, cpu_mask);
+    }
+}
+#else
+static inline bool is_valid_espi_rank(struct domain *d, unsigned int rank)
+{
+    return false;
+}
+
+/*
+ * This function is stub and will not be called if CONFIG_GICV3_ESPI=n,
+ * because in this case, is_valid_espi_rank will always return false.
+ */
+static inline struct vgic_irq_rank *vgic_get_espi_rank(struct vcpu *v,
+                                                       unsigned int rank)
+{
+    ASSERT_UNREACHABLE();
+    return NULL;
+}
+
+static inline bool vgic_reserve_espi_virq(struct domain *d, unsigned int virq)
+{
+    return false;
+}
+
+static void arch_move_espis(struct vcpu *v) { }
+#endif
+
 static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
                                                   unsigned int rank)
 {
@@ -37,6 +110,8 @@ static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
         return v->arch.vgic.private_irqs;
     else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
         return &v->domain->arch.vgic.shared_irqs[rank - 1];
+    else if ( is_valid_espi_rank(v->domain, rank) )
+        return vgic_get_espi_rank(v, rank);
     else
         return NULL;
 }
@@ -117,6 +192,74 @@ int domain_vgic_register(struct domain *d, unsigned int *mmio_count)
     return 0;
 }
 
+#ifdef CONFIG_GICV3_ESPI
+/*
+ * The function behavior is the same as for regular SPIs (vgic_rank_offset),
+ * but it operates with extended SPI ranks.
+ */
+struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v, unsigned int b,
+                                           unsigned int n, unsigned int s)
+{
+    unsigned int rank = REG_RANK_NR(b, (n >> s));
+
+    return vgic_get_rank(v, rank + EXT_RANK_MIN);
+}
+
+static unsigned int vgic_num_spi_lines(struct domain *d)
+{
+    return d->arch.vgic.nr_spis + d->arch.vgic.nr_espis;
+}
+
+static int init_vgic_espi(struct domain *d)
+{
+    unsigned int i, idx;
+
+    if ( d->arch.vgic.nr_espis == 0 )
+        return 0;
+
+    d->arch.vgic.ext_shared_irqs =
+        xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_EXT_RANKS(d));
+    if ( d->arch.vgic.ext_shared_irqs == NULL )
+        return -ENOMEM;
+
+    for ( i = d->arch.vgic.nr_spis, idx = 0;
+          i < vgic_num_spi_lines(d); i++, idx++ )
+        vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i],
+                              ESPI_IDX2INTID(idx));
+
+    for ( i = 0; i < DOMAIN_NR_EXT_RANKS(d); i++ )
+        vgic_rank_init(&d->arch.vgic.ext_shared_irqs[i], i, 0);
+
+    return 0;
+}
+
+struct pending_irq *espi_to_pending(struct domain *d, unsigned int irq)
+{
+    irq = ESPI_INTID2IDX(irq) + d->arch.vgic.nr_spis;
+    return &d->arch.vgic.pending_irqs[irq];
+}
+#else
+static unsigned int init_vgic_espi(struct domain *d)
+{
+    return 0;
+}
+
+static unsigned int vgic_num_spi_lines(struct domain *d)
+{
+    return d->arch.vgic.nr_spis;
+}
+
+struct pending_irq *espi_to_pending(struct domain *d, unsigned int irq)
+{
+    return NULL;
+}
+#endif
+
+static unsigned int vgic_num_alloc_irqs(struct domain *d)
+{
+    return vgic_num_spi_lines(d) + NR_LOCAL_IRQS;
+}
+
 int domain_vgic_init(struct domain *d, unsigned int nr_spis)
 {
     int i;
@@ -131,6 +274,38 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
      */
     nr_spis = ROUNDUP(nr_spis, 32);
 
+#ifdef CONFIG_GICV3_ESPI
+    /*
+     * During domain creation, the dom0less DomUs code or toolstack specifies
+     * the maximum INTID, which is defined in the domain config subtracted by
+     * 32 to cover the local IRQs (please see the comment to VGIC_DEF_NR_SPIS).
+     * To compute the actual number of eSPI that will be usable for,
+     * add back 32.
+     */
+    if ( nr_spis + 32 > ESPI_IDX2INTID(NR_ESPI_IRQS) )
+        return -EINVAL;
+
+    if ( nr_spis + 32 >= ESPI_BASE_INTID )
+    {
+        d->arch.vgic.nr_espis = min(nr_spis - ESPI_BASE_INTID + 32, 1024U);
+        /* Verify if GIC HW can handle provided INTID */
+        if ( d->arch.vgic.nr_espis > gic_number_espis() )
+            return -EINVAL;
+        /*
+         * Set the maximum available number for regular
+         * SPI to pass the next check
+         */
+        nr_spis = VGIC_DEF_NR_SPIS;
+        d->arch.vgic.has_espi = true;
+    }
+    else
+    {
+        /* Domain will use the regular SPI range */
+        d->arch.vgic.nr_espis = 0;
+        d->arch.vgic.has_espi = false;
+    }
+#endif
+
     /* Limit the number of virtual SPIs supported to (1020 - 32) = 988  */
     if ( nr_spis > (1020 - NR_LOCAL_IRQS) )
         return -EINVAL;
@@ -145,7 +320,7 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
         return -ENOMEM;
 
     d->arch.vgic.pending_irqs =
-        xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis);
+        xzalloc_array(struct pending_irq, vgic_num_spi_lines(d));
     if ( d->arch.vgic.pending_irqs == NULL )
         return -ENOMEM;
 
@@ -156,12 +331,16 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
     for ( i = 0; i < DOMAIN_NR_RANKS(d); i++ )
         vgic_rank_init(&d->arch.vgic.shared_irqs[i], i + 1, 0);
 
+    ret = init_vgic_espi(d);
+    if ( ret )
+        return ret;
+
     ret = d->arch.vgic.handler->domain_init(d);
     if ( ret )
         return ret;
 
     d->arch.vgic.allocated_irqs =
-        xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d)));
+        xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_alloc_irqs(d)));
     if ( !d->arch.vgic.allocated_irqs )
         return -ENOMEM;
 
@@ -195,9 +374,27 @@ void domain_vgic_free(struct domain *d)
         }
     }
 
+#ifdef CONFIG_GICV3_ESPI
+    for ( i = 0; i < d->arch.vgic.nr_espis; i++ )
+    {
+        struct pending_irq *p = spi_to_pending(d, ESPI_IDX2INTID(i));
+
+        if ( p->desc )
+        {
+            ret = release_guest_irq(d, p->irq);
+            if ( ret )
+                dprintk(XENLOG_G_WARNING, "d%u: Failed to release virq %u ret = %d\n",
+                        d->domain_id, p->irq, ret);
+        }
+    }
+#endif
+
     if ( d->arch.vgic.handler )
         d->arch.vgic.handler->domain_free(d);
     xfree(d->arch.vgic.shared_irqs);
+#ifdef CONFIG_GICV3_ESPI
+    xfree(d->arch.vgic.ext_shared_irqs);
+#endif
     xfree(d->arch.vgic.pending_irqs);
     xfree(d->arch.vgic.allocated_irqs);
 }
@@ -331,6 +528,8 @@ void arch_move_irqs(struct vcpu *v)
         if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) )
             irq_set_affinity(p->desc, cpu_mask);
     }
+
+    arch_move_espis(v);
 }
 
 void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned int n)
@@ -538,6 +737,8 @@ struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
         n = &v->arch.vgic.pending_irqs[irq];
     else if ( is_lpi(irq) )
         n = v->domain->arch.vgic.handler->lpi_to_pending(v->domain, irq);
+    else if ( is_espi(irq) )
+        n = espi_to_pending(v->domain, irq);
     else
         n = &v->domain->arch.vgic.pending_irqs[irq - 32];
     return n;
@@ -547,6 +748,9 @@ struct pending_irq *spi_to_pending(struct domain *d, unsigned int irq)
 {
     ASSERT(irq >= NR_LOCAL_IRQS);
 
+    if ( is_espi(irq) )
+        return espi_to_pending(d, irq);
+
     return &d->arch.vgic.pending_irqs[irq - 32];
 }
 
@@ -668,6 +872,9 @@ bool vgic_reserve_virq(struct domain *d, unsigned int virq)
     if ( !vgic_is_valid_line(d, virq) )
         return false;
 
+    if ( is_espi(virq) )
+        return vgic_reserve_espi_virq(d, virq);
+
     return !test_and_set_bit(virq, d->arch.vgic.allocated_irqs);
 }
 
@@ -685,7 +892,7 @@ int vgic_allocate_virq(struct domain *d, bool spi)
     else
     {
         first = 32;
-        end = vgic_num_irqs(d);
+        end = vgic_num_alloc_irqs(d);
     }
 
     /*
-- 
2.34.1

[PATCH v4 09/12] xen/arm: domain_build/dom0less-build: adjust domains config to support eSPIs

[Leonid Komarianskyi]

The Dom0 and DomUs logic for the dom0less configuration in create_dom0() and
arch_create_domUs() has been updated to account for extended SPIs when
supported by the hardware and enabled with CONFIG_GICV3_ESPI. These changes
ensure the proper calculation of the maximum number of SPIs and eSPIs available
to Dom0 and DomUs in dom0less setups.

When eSPIs are supported by the hardware and CONFIG_GICV3_ESPI is enabled, the
maximum number of eSPI interrupts is calculated using the ESPI_BASE_INTID
offset (4096) and is limited to 1024, with 32 IRQs subtracted. To ensure
compatibility with non-Dom0 domains, this adjustment is applied by the
toolstack during domain creation, while for Dom0 or DomUs in Dom0, it is
handled directly during VGIC initialization. If eSPIs are not supported, the
calculation defaults to using the standard SPI range, with a maximum value of
992 interrupt lines, as it works currently.

Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>

---
Changes in V4:
- consolidated the eSPI and SPI logic into a new inline function,
  vgic_def_nr_spis. Without eSPI support (either due to config being
  disabled or hardware not supporting it), it will return the regular SPI
  range, as it works currently. There are no functional changes compared
  with the previous patch version
- removed VGIC_DEF_MAX_SPI macro, to reduce the number of ifdefs

Changes in V3:
- renamed macro VGIC_DEF_NR_ESPIS to more appropriate VGIC_DEF_MAX_SPI
- added eSPI initialization for dom0less setups
- fixed comment with mentions about dom0less builds
- fixed formatting for lines with more than 80 symbols
- updated commit message

Changes in V2:
- no changes
---
 xen/arch/arm/dom0less-build.c   |  2 +-
 xen/arch/arm/domain_build.c     |  2 +-
 xen/arch/arm/include/asm/vgic.h | 21 +++++++++++++++++++++
 3 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
index 69b9ea22ce..02d5559102 100644
--- a/xen/arch/arm/dom0less-build.c
+++ b/xen/arch/arm/dom0less-build.c
@@ -285,7 +285,7 @@ void __init arch_create_domUs(struct dt_device_node *node,
     {
         int vpl011_virq = GUEST_VPL011_SPI;
 
-        d_cfg->arch.nr_spis = VGIC_DEF_NR_SPIS;
+        d_cfg->arch.nr_spis = vgic_def_nr_spis();
 
         /*
          * The VPL011 virq is GUEST_VPL011_SPI, unless direct-map is
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index d91a71acfd..39eea0be00 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2054,7 +2054,7 @@ void __init create_dom0(void)
 
     /* The vGIC for DOM0 is exactly emulating the hardware GIC */
     dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
-    dom0_cfg.arch.nr_spis = VGIC_DEF_NR_SPIS;
+    dom0_cfg.arch.nr_spis = vgic_def_nr_spis();
     dom0_cfg.arch.tee_type = tee_get_type();
     dom0_cfg.max_vcpus = dom0_max_vcpus();
 
diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
index fb4cea73eb..11f9d216eb 100644
--- a/xen/arch/arm/include/asm/vgic.h
+++ b/xen/arch/arm/include/asm/vgic.h
@@ -355,6 +355,27 @@ extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
 /* Default number of vGIC SPIs. 32 are substracted to cover local IRQs. */
 #define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
 
+static inline unsigned int vgic_def_nr_spis(void)
+{
+#ifdef CONFIG_GICV3_ESPI
+    /*
+     * Check if the hardware supports extended SPIs (even if the appropriate
+     * config is set). If not, the common SPI range will be used. Otherwise
+     * returns the maximum eSPI INTID, supported by HW GIC, subtracted by 32.
+     * For non-Dom0 domains, the toolstack or arch_create_domUs function
+     * applies the same adjustment to cover local IRQs (please, see comment
+     * for macro that is used for regular SPIs - VGIC_DEF_NR_SPIS). We will
+     * add back this value during VGIC initialization. This ensures consistent
+     * handling for Dom0 and other domains. For the regular SPI range interrupts
+     * in this case, the maximum value of VGIC_DEF_NR_SPIS will be used.
+     */
+    if ( gic_number_espis() > 0 )
+        return ESPI_BASE_INTID + min(gic_number_espis(), 1024U) - 32;
+#endif
+
+    return VGIC_DEF_NR_SPIS;
+}
+
 extern bool vgic_is_valid_line(struct domain *d, unsigned int virq);
 
 static inline bool vgic_is_spi(struct domain *d, unsigned int virq)
-- 
2.34.1

[PATCH v4 10/12] xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers

[Leonid Komarianskyi]

Implemented support for GICv3.1 extended SPI registers for vGICv3,
allowing the emulation of eSPI-specific behavior for guest domains.
The implementation includes read and write emulation for eSPI-related
registers (e.g., GICD_ISENABLERnE, GICD_IROUTERnE, and others),
following a similar approach to the handling of regular SPIs.

The eSPI registers, previously located in reserved address ranges,
are now adjusted to support MMIO read and write operations correctly
when CONFIG_GICV3_ESPI is enabled.

The availability of eSPIs and the number of emulated extended SPIs
for guest domains is reported by setting the appropriate bits in the
GICD_TYPER register, based on the number of eSPIs requested by the
domain and supported by the hardware. In cases where the configuration
option is disabled, the hardware does not support eSPIs, or the domain
does not request such interrupts, the functionality remains unchanged.

Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>

---
Changes in V4:
- added missing RAZ and write ignore eSPI-specific registers ranges:
  GICD_NSACRnE and GICD_IGRPMODRnE
- changed previously reserved range to cover GICD_NSACRnE and
  GICD_IGRPMODRnE
- introduced GICD_RESERVED_RANGE<n>_START/END defines to remove
  hardcoded values and reduce the number of ifdefs
- fixed reserved ranges with eSPI option enabled: added missing range
  0x0F30-0x0F7C
- updated the logic for domains that do not support eSPI, but Xen is
  compiled with the eSPI option. Now, prior to other MMIO checks, we
  verify whether eSPI is available for the domain or not. If not, it
  behaves as it does currently - RAZ and WI
- fixed print for GICD_ICACTIVERnE
- fixed new lines formatting for switch-case

Changes in V3:
- changed vgic_store_irouter parameters - instead of offset virq is
  used, to remove the additional bool espi parameter and simplify
  checks. Also, adjusted parameters for regular SPI. Since the offset
  parameter was used only for calculating virq number and then reused for
  finding rank offset, it will not affect functionality.
- fixed formatting for goto lables - added newlines after condition
- fixed logs for GICD_ISACTIVERnE and GICD_ICACTIVERnE handlers
- removed #ifdefs in 2 places where they were adjacent and could be merged

Changes in V2:
- add missing rank index conversion for pending and inflight irqs
---
 xen/arch/arm/vgic-v3.c | 333 +++++++++++++++++++++++++++++++++++++++--
 1 file changed, 319 insertions(+), 14 deletions(-)

diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 4369c55177..ca19f24634 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -42,6 +42,36 @@
  */
 #define VGICD_CTLR_DEFAULT  (GICD_CTLR_ARE_NS)
 
+/*
+ * Common start and end of reserved ranges for
+ * both eSPI and non-eSPI builds
+ */
+#define GICD_RESERVED_RANGE1_START (0x0F30)
+#define GICD_RESERVED_RANGE2_END   (0xBFFC)
+
+#ifdef CONFIG_GICV3_ESPI
+
+#define GICD_RESERVED_RANGE1_END   (0x0F7C)
+
+#define GICD_RESERVED_RANGE2_START (0xA004)
+
+/*
+ * In case eSPI is enabled, there is an additional
+ * reserved range after the eSPI-specific registers
+ */
+#define GICD_RESERVED_RANGE3_START (0x3700)
+#define GICD_RESERVED_RANGE3_END   (0x60FC)
+#else
+
+/*
+ * In case eSPI is disabled, the range with eSPI-specific
+ * registers is marked as reserved
+ */
+#define GICD_RESERVED_RANGE1_END   (0x60FC)
+
+#define GICD_RESERVED_RANGE2_START (0x7FE0)
+#endif
+
 static struct {
     bool enabled;
     /* Distributor interface address */
@@ -111,13 +141,10 @@ static uint64_t vgic_fetch_irouter(struct vgic_irq_rank *rank,
  * Note the offset will be aligned to the appropriate boundary.
  */
 static void vgic_store_irouter(struct domain *d, struct vgic_irq_rank *rank,
-                               unsigned int offset, uint64_t irouter)
+                               unsigned int virq, uint64_t irouter)
 {
     struct vcpu *new_vcpu, *old_vcpu;
-    unsigned int virq;
-
-    /* There is 1 vIRQ per IROUTER */
-    virq = offset / NR_BYTES_PER_IROUTER;
+    unsigned int offset;
 
     /*
      * The IROUTER0-31, used for SGIs/PPIs, are reserved and should
@@ -685,6 +712,10 @@ static int __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
     {
     case VRANGE32(GICD_IGROUPR, GICD_IGROUPRN):
     case VRANGE32(GICD_IGRPMODR, GICD_IGRPMODRN):
+#ifdef CONFIG_GICV3_ESPI
+    case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
+    case VRANGE32(GICD_IGRPMODRnE, GICD_IGRPMODRnEN):
+#endif
         /* We do not implement security extensions for guests, read zero */
         if ( dabt.size != DABT_WORD ) goto bad_width;
         goto read_as_zero;
@@ -710,11 +741,19 @@ static int __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
     /* Read the pending status of an IRQ via GICD/GICR is not supported */
     case VRANGE32(GICD_ISPENDR, GICD_ISPENDRN):
     case VRANGE32(GICD_ICPENDR, GICD_ICPENDRN):
+#ifdef CONFIG_GICV3_ESPI
+    case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
+    case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
+#endif
         goto read_as_zero;
 
     /* Read the active status of an IRQ via GICD/GICR is not supported */
     case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
     case VRANGE32(GICD_ICACTIVER, GICD_ICACTIVERN):
+#ifdef CONFIG_GICV3_ESPI
+    case VRANGE32(GICD_ISACTIVERnE, GICD_ISACTIVERnEN):
+    case VRANGE32(GICD_ICACTIVERnE, GICD_ICACTIVERnEN):
+#endif
         goto read_as_zero;
 
     case VRANGE32(GICD_IPRIORITYR, GICD_IPRIORITYRN):
@@ -752,6 +791,69 @@ static int __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
         return 1;
     }
 
+#ifdef CONFIG_GICV3_ESPI
+    case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
+        if ( dabt.size != DABT_WORD )
+            goto bad_width;
+        rank = vgic_ext_rank_offset(v, 1, reg - GICD_ISENABLERnE, DABT_WORD);
+        if ( rank == NULL )
+            goto read_as_zero;
+        vgic_lock_rank(v, rank, flags);
+        *r = vreg_reg32_extract(rank->ienable, info);
+        vgic_unlock_rank(v, rank, flags);
+        return 1;
+
+    case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
+        if ( dabt.size != DABT_WORD )
+            goto bad_width;
+        rank = vgic_ext_rank_offset(v, 1, reg - GICD_ICENABLERnE, DABT_WORD);
+        if ( rank == NULL )
+            goto read_as_zero;
+        vgic_lock_rank(v, rank, flags);
+        *r = vreg_reg32_extract(rank->ienable, info);
+        vgic_unlock_rank(v, rank, flags);
+        return 1;
+
+    case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
+    {
+        uint32_t ipriorityr;
+        uint8_t rank_index;
+
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD )
+            goto bad_width;
+        rank = vgic_ext_rank_offset(v, 8, reg - GICD_IPRIORITYRnE, DABT_WORD);
+        if ( rank == NULL )
+            goto read_as_zero;
+        rank_index = REG_RANK_INDEX(8, reg - GICD_IPRIORITYRnE, DABT_WORD);
+
+        vgic_lock_rank(v, rank, flags);
+        ipriorityr = ACCESS_ONCE(rank->ipriorityr[rank_index]);
+        vgic_unlock_rank(v, rank, flags);
+
+        *r = vreg_reg32_extract(ipriorityr, info);
+
+        return 1;
+    }
+
+    case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
+    {
+        uint32_t icfgr;
+
+        if ( dabt.size != DABT_WORD )
+            goto bad_width;
+        rank = vgic_ext_rank_offset(v, 2, reg - GICD_ICFGRnE, DABT_WORD);
+        if ( rank == NULL )
+            goto read_as_zero;
+        vgic_lock_rank(v, rank, flags);
+        icfgr = rank->icfg[REG_RANK_INDEX(2, reg - GICD_ICFGRnE, DABT_WORD)];
+        vgic_unlock_rank(v, rank, flags);
+
+        *r = vreg_reg32_extract(icfgr, info);
+
+        return 1;
+    }
+#endif
+
     default:
         printk(XENLOG_G_ERR
                "%pv: %s: unhandled read r%d offset %#08x\n",
@@ -782,6 +884,10 @@ static int __vgic_v3_distr_common_mmio_write(const char *name, struct vcpu *v,
     {
     case VRANGE32(GICD_IGROUPR, GICD_IGROUPRN):
     case VRANGE32(GICD_IGRPMODR, GICD_IGRPMODRN):
+#ifdef CONFIG_GICV3_ESPI
+    case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
+    case VRANGE32(GICD_IGRPMODRnE, GICD_IGRPMODRnEN):
+#endif
         /* We do not implement security extensions for guests, write ignore */
         goto write_ignore_32;
 
@@ -871,6 +977,102 @@ static int __vgic_v3_distr_common_mmio_write(const char *name, struct vcpu *v,
         vgic_unlock_rank(v, rank, flags);
         return 1;
 
+#ifdef CONFIG_GICV3_ESPI
+    case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
+        if ( dabt.size != DABT_WORD )
+            goto bad_width;
+        rank = vgic_ext_rank_offset(v, 1, reg - GICD_ISENABLERnE, DABT_WORD);
+        if ( rank == NULL )
+            goto write_ignore;
+        vgic_lock_rank(v, rank, flags);
+        tr = rank->ienable;
+        vreg_reg32_setbits(&rank->ienable, r, info);
+        vgic_enable_irqs(v, (rank->ienable) & (~tr), EXT_RANK_IDX2NUM(rank->index));
+        vgic_unlock_rank(v, rank, flags);
+        return 1;
+
+    case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
+        if ( dabt.size != DABT_WORD )
+            goto bad_width;
+        rank = vgic_ext_rank_offset(v, 1, reg - GICD_ICENABLERnE, DABT_WORD);
+        if ( rank == NULL )
+            goto write_ignore;
+        vgic_lock_rank(v, rank, flags);
+        tr = rank->ienable;
+        vreg_reg32_clearbits(&rank->ienable, r, info);
+        vgic_disable_irqs(v, (~rank->ienable) & tr, EXT_RANK_IDX2NUM(rank->index));
+        vgic_unlock_rank(v, rank, flags);
+        return 1;
+
+    case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
+        if ( dabt.size != DABT_WORD )
+            goto bad_width;
+        rank = vgic_ext_rank_offset(v, 1, reg - GICD_ISPENDRnE, DABT_WORD);
+        if ( rank == NULL )
+            goto write_ignore;
+
+        vgic_set_irqs_pending(v, r, EXT_RANK_IDX2NUM(rank->index));
+
+        return 1;
+
+    case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
+        if ( dabt.size != DABT_WORD )
+            goto bad_width;
+        rank = vgic_ext_rank_offset(v, 1, reg - GICD_ICPENDRnE, DABT_WORD);
+        if ( rank == NULL )
+            goto write_ignore;
+
+        vgic_check_inflight_irqs_pending(v, EXT_RANK_IDX2NUM(rank->index), r);
+
+        goto write_ignore;
+
+    case VRANGE32(GICD_ISACTIVERnE, GICD_ISACTIVERnEN):
+        if ( dabt.size != DABT_WORD )
+            goto bad_width;
+        printk(XENLOG_G_ERR
+               "%pv: %s: unhandled word write %#"PRIregister" to ISACTIVER%dE\n",
+               v, name, r, reg - GICD_ISACTIVERnE);
+        return 0;
+
+    case VRANGE32(GICD_ICACTIVERnE, GICD_ICACTIVERnEN):
+        printk(XENLOG_G_ERR
+               "%pv: %s: unhandled word write %#"PRIregister" to ICACTIVER%dE\n",
+               v, name, r, reg - GICD_ICACTIVERnE);
+        goto write_ignore_32;
+
+    case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
+    {
+        uint32_t *ipriorityr, priority;
+
+        if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD )
+            goto bad_width;
+        rank = vgic_ext_rank_offset(v, 8, reg - GICD_IPRIORITYRnE, DABT_WORD);
+        if ( rank == NULL )
+            goto write_ignore;
+        vgic_lock_rank(v, rank, flags);
+        ipriorityr = &rank->ipriorityr[REG_RANK_INDEX(8, reg - GICD_IPRIORITYRnE,
+                                                      DABT_WORD)];
+        priority = ACCESS_ONCE(*ipriorityr);
+        vreg_reg32_update(&priority, r, info);
+        ACCESS_ONCE(*ipriorityr) = priority;
+        vgic_unlock_rank(v, rank, flags);
+        return 1;
+    }
+
+    case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
+        if ( dabt.size != DABT_WORD )
+            goto bad_width;
+        rank = vgic_ext_rank_offset(v, 2, reg - GICD_ICFGRnE, DABT_WORD);
+        if ( rank == NULL )
+            goto write_ignore;
+        vgic_lock_rank(v, rank, flags);
+        vreg_reg32_update(&rank->icfg[REG_RANK_INDEX(2, reg - GICD_ICFGRnE,
+                                                     DABT_WORD)],
+                          r, info);
+        vgic_unlock_rank(v, rank, flags);
+        return 1;
+#endif
+
     default:
         printk(XENLOG_G_ERR
                "%pv: %s: unhandled write r%d=%"PRIregister" offset %#08x\n",
@@ -1129,6 +1331,16 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info,
             typer |= GICD_TYPE_LPIS;
 
         typer |= (v->domain->arch.vgic.intid_bits - 1) << GICD_TYPE_ID_BITS_SHIFT;
+#ifdef CONFIG_GICV3_ESPI
+        if ( v->domain->arch.vgic.has_espi )
+        {
+            /* Set eSPI support bit for the domain */
+            typer |= GICD_TYPER_ESPI;
+            /* Set ESPI range bits */
+            typer |= (DIV_ROUND_UP(v->domain->arch.vgic.nr_espis, 32) - 1)
+                       << GICD_TYPER_ESPI_RANGE_SHIFT;
+        }
+#endif
 
         *r = vreg_reg32_extract(typer, info);
 
@@ -1184,6 +1396,21 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info,
     case VRANGE32(0x005C, 0x007C):
         goto read_reserved;
 
+#ifdef CONFIG_GICV3_ESPI
+    case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
+    case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
+    case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
+    case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
+    case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
+    case VRANGE32(GICD_ISACTIVERnE, GICD_ISACTIVERnEN):
+    case VRANGE32(GICD_ICACTIVERnE, GICD_ICACTIVERnEN):
+    case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
+    case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
+    case VRANGE32(GICD_IGRPMODRnE, GICD_IGRPMODRnEN):
+        if ( !v->domain->arch.vgic.has_espi )
+            goto read_reserved;
+        return __vgic_v3_distr_common_mmio_read("vGICD", v, info, gicd_reg, r);
+#endif
     case VRANGE32(GICD_IGROUPR, GICD_IGROUPRN):
     case VRANGE32(GICD_ISENABLER, GICD_ISENABLERN):
     case VRANGE32(GICD_ICENABLER, GICD_ICENABLERN):
@@ -1200,6 +1427,11 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info,
          */
         return __vgic_v3_distr_common_mmio_read("vGICD", v, info, gicd_reg, r);
 
+#ifdef CONFIG_GICV3_ESPI
+    case VRANGE32(GICD_NSACRnE, GICD_NSACRnEN):
+        if ( !v->domain->arch.vgic.has_espi )
+            goto read_reserved;
+#endif
     case VRANGE32(GICD_NSACR, GICD_NSACRN):
         /* We do not implement security extensions for guests, read zero */
         goto read_as_zero_32;
@@ -1216,7 +1448,7 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info,
         /* Replaced with GICR_ISPENDR0. So ignore write */
         goto read_as_zero_32;
 
-    case VRANGE32(0x0F30, 0x60FC):
+    case VRANGE32(GICD_RESERVED_RANGE1_START, GICD_RESERVED_RANGE1_END):
         goto read_reserved;
 
     case VRANGE64(GICD_IROUTER32, GICD_IROUTER1019):
@@ -1235,8 +1467,32 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info,
 
         return 1;
     }
+#ifdef CONFIG_GICV3_ESPI
+    case VRANGE64(GICD_IROUTERnE, GICD_IROUTERnEN):
+    {
+        uint64_t irouter;
 
-    case VRANGE32(0x7FE0, 0xBFFC):
+        if ( !v->domain->arch.vgic.has_espi )
+            goto read_reserved;
+
+        if ( !vgic_reg64_check_access(dabt) )
+            goto bad_width;
+        rank = vgic_ext_rank_offset(v, 64, gicd_reg - GICD_IROUTERnE,
+                                DABT_DOUBLE_WORD);
+        if ( rank == NULL )
+            goto read_as_zero;
+        vgic_lock_rank(v, rank, flags);
+        irouter = vgic_fetch_irouter(rank, gicd_reg - GICD_IROUTERnE);
+        vgic_unlock_rank(v, rank, flags);
+
+        *r = vreg_reg64_extract(irouter, info);
+
+        return 1;
+    }
+
+    case VRANGE32(GICD_RESERVED_RANGE3_START, GICD_RESERVED_RANGE3_END):
+#endif
+    case VRANGE32(GICD_RESERVED_RANGE2_START, GICD_RESERVED_RANGE2_END):
         goto read_reserved;
 
     case VRANGE32(0xC000, 0xFFCC):
@@ -1372,6 +1628,23 @@ static int vgic_v3_distr_mmio_write(struct vcpu *v, mmio_info_t *info,
     case VRANGE32(0x005C, 0x007C):
         goto write_reserved;
 
+#ifdef CONFIG_GICV3_ESPI
+    case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
+    case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
+    case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
+    case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
+    case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
+    case VRANGE32(GICD_ISACTIVERnE, GICD_ISACTIVERnEN):
+    case VRANGE32(GICD_ICACTIVERnE, GICD_ICACTIVERnEN):
+    case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
+    case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
+    case VRANGE32(GICD_IGRPMODRnE, GICD_IGRPMODRnEN):
+        if ( !v->domain->arch.vgic.has_espi )
+            goto write_reserved;
+        return __vgic_v3_distr_common_mmio_write("vGICD", v, info,
+                                                 gicd_reg, r);
+#endif
+
     case VRANGE32(GICD_IGROUPR, GICD_IGROUPRN):
     case VRANGE32(GICD_ISENABLER, GICD_ISENABLERN):
     case VRANGE32(GICD_ICENABLER, GICD_ICENABLERN):
@@ -1386,7 +1659,11 @@ static int vgic_v3_distr_mmio_write(struct vcpu *v, mmio_info_t *info,
          * Manage in common */
         return __vgic_v3_distr_common_mmio_write("vGICD", v, info,
                                                  gicd_reg, r);
-
+#ifdef CONFIG_GICV3_ESPI
+    case VRANGE32(GICD_NSACRnE, GICD_NSACRnEN):
+        if ( !v->domain->arch.vgic.has_espi )
+            goto write_reserved;
+#endif
     case VRANGE32(GICD_NSACR, GICD_NSACRN):
         /* We do not implement security extensions for guests, write ignore */
         goto write_ignore_32;
@@ -1405,26 +1682,54 @@ static int vgic_v3_distr_mmio_write(struct vcpu *v, mmio_info_t *info,
         if ( dabt.size != DABT_WORD ) goto bad_width;
         return 0;
 
-    case VRANGE32(0x0F30, 0x60FC):
+    case VRANGE32(GICD_RESERVED_RANGE1_START, GICD_RESERVED_RANGE1_END):
         goto write_reserved;
 
     case VRANGE64(GICD_IROUTER32, GICD_IROUTER1019):
     {
         uint64_t irouter;
+        unsigned int offset, virq;
 
         if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
-        rank = vgic_rank_offset(v, 64, gicd_reg - GICD_IROUTER,
-                                DABT_DOUBLE_WORD);
+        offset = gicd_reg - GICD_IROUTER;
+        rank = vgic_rank_offset(v, 64, offset, DABT_DOUBLE_WORD);
         if ( rank == NULL ) goto write_ignore;
         vgic_lock_rank(v, rank, flags);
-        irouter = vgic_fetch_irouter(rank, gicd_reg - GICD_IROUTER);
+        irouter = vgic_fetch_irouter(rank, offset);
         vreg_reg64_update(&irouter, r, info);
-        vgic_store_irouter(v->domain, rank, gicd_reg - GICD_IROUTER, irouter);
+        virq = offset / NR_BYTES_PER_IROUTER;
+        vgic_store_irouter(v->domain, rank, virq, irouter);
         vgic_unlock_rank(v, rank, flags);
         return 1;
     }
 
-    case VRANGE32(0x7FE0, 0xBFFC):
+#ifdef CONFIG_GICV3_ESPI
+    case VRANGE64(GICD_IROUTERnE, GICD_IROUTERnEN):
+    {
+        uint64_t irouter;
+        unsigned int offset, virq;
+
+        if ( !v->domain->arch.vgic.has_espi )
+            goto write_reserved;
+
+        if ( !vgic_reg64_check_access(dabt) )
+            goto bad_width;
+        offset = gicd_reg - GICD_IROUTERnE;
+        rank = vgic_ext_rank_offset(v, 64, offset, DABT_DOUBLE_WORD);
+        if ( rank == NULL )
+            goto write_ignore;
+        vgic_lock_rank(v, rank, flags);
+        irouter = vgic_fetch_irouter(rank, offset);
+        vreg_reg64_update(&irouter, r, info);
+        virq = ESPI_IDX2INTID(offset / NR_BYTES_PER_IROUTER);
+        vgic_store_irouter(v->domain, rank, virq, irouter);
+        vgic_unlock_rank(v, rank, flags);
+        return 1;
+    }
+
+    case VRANGE32(GICD_RESERVED_RANGE3_START, GICD_RESERVED_RANGE3_END):
+#endif
+    case VRANGE32(GICD_RESERVED_RANGE2_START, GICD_RESERVED_RANGE2_END):
         goto write_reserved;
 
     case VRANGE32(0xC000, 0xFFCC):
-- 
2.34.1

[PATCH v4 11/12] doc/man: update description for nr_spis with eSPI

[Leonid Komarianskyi]

Since eSPI support has been introduced, update the documentation with
the appropriate description.

Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>

---
Changes in V4:
- introduced this patch
---
 docs/man/xl.cfg.5.pod.in | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 5362fb0e9a..292ab10331 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -3072,11 +3072,14 @@ interval of colors (such as "0-4").
 =item B<nr_spis="NR_SPIS">
 
 An optional integer parameter specifying the number of SPIs (Shared
-Peripheral Interrupts) to allocate for the domain. Max is 960 SPIs. If
-the `nr_spis` parameter is not specified, the value calculated by the toolstack
-will be used for the domain. Otherwise, the value specified by the `nr_spis`
-parameter will be used. The number of SPIs should match the highest interrupt
-ID that will be assigned to the domain.
+Peripheral Interrupts) to allocate for the domain. Max is 960 for regular SPIs
+or 5088 for eSPIs (Extended SPIs). The eSPIs includes an additional 1024 SPIs
+from the eSPI range (4096 to 5119) if the hardware supports extended SPIs
+(GICv3.1+) and CONFIG_GICV3_ESPI is enabled. If the `nr_spis` parameter is not
+specified, the value calculated by the toolstack will be used for the domain.
+Otherwise, the value specified by the `nr_spis` parameter will be used. The
+number of SPIs should match the highest interrupt ID that will be assigned to
+the domain.
 
 =item B<trap_unmapped_accesses=BOOLEAN>
 
-- 
2.34.1

[PATCH v4 12/12] CHANGELOG.md: add mention of GICv3.1 eSPI support

[Leonid Komarianskyi]

The GICv3.1 eSPI (Extended Shared Peripheral Interrupts) range is
already supported with CONFIG_GICV3_ESPI enabled, so this feature should
be mentioned in CHANGELOG.md.

Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>

---
Changes in V3:
- introduced this patch
---
 CHANGELOG.md | 1 +
 1 file changed, 1 insertion(+)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 5f31ca08fe..dc34d29d99 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -29,6 +29,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 
  - On Arm:
     - Ability to enable stack protector
+    - GICv3.1 eSPI support
 
 ### Removed
  - On x86:
-- 
2.34.1

Re: [PATCH v4 03/12] xen/arm: vgic: implement helper functions for virq checks

[Volodymyr Babchuk]

Hi Leonid,

Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:

> Introduced two new helper functions for vGIC: vgic_is_valid_line and
> vgic_is_spi. The functions are similar to the newly introduced
> gic_is_valid_line and gic_is_spi, but they verify whether a vIRQ
> is available for a specific domain, while GIC-specific functions
> validate INTIDs for the real GIC hardware. For example, the GIC may
> support all 992 SPI lines, but the domain may use only some part of them
> (e.g., 640), depending on the highest IRQ number defined in the domain
> configuration. Therefore, for vGIC-related code and checks, the
> appropriate functions should be used. Also, updated the appropriate
> checks to use these new helper functions.
>
> The purpose of introducing new helper functions for vGIC is essentially
> the same as for GIC: to avoid potential confusion with GIC-related
> checks and to consolidate similar code into separate functions, which
> can be more easily extended by additional conditions, e.g., when
> implementing extended SPI interrupts.
>
> Only the validation change in vgic_inject_irq may affect existing
> functionality, as it currently checks whether the vIRQ is less than or
> equal to vgic_num_irqs. Since IRQ indexes start from 0 (where 32 is the
> first SPI), the check should behave consistently with similar logic in
> other places and should check if the vIRQ number is less than
> vgic_num_irqs. The remaining changes, which replace open-coded checks
> with the use of these new helper functions, do not introduce any
> functional changes, as the helper functions follow the current vIRQ
> index verification logic.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>

Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>

>
> ---
> Changes in V4:
> - removed redundant parentheses
>
> Changes in V3:
> - renamed vgic_is_valid_irq to vgic_is_valid_line and vgic_is_shared_irq
>   to vgic_is_spi
> - added vgic_is_valid_line implementation for new-vgic, because
>   vgic_is_valid_line is called from generic code. It is necessary to fix
>   the build for new-vgic.
> - updated commit message
>
> Changes in V2:
> - introduced this patch
> ---
>  xen/arch/arm/gic.c              |  3 +--
>  xen/arch/arm/include/asm/vgic.h |  7 +++++++
>  xen/arch/arm/irq.c              |  4 ++--
>  xen/arch/arm/vgic.c             | 10 ++++++++--
>  xen/arch/arm/vgic/vgic.c        |  5 +++++
>  5 files changed, 23 insertions(+), 6 deletions(-)
>
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index 9220eef6ea..b88237ccda 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -133,8 +133,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
>  
>      ASSERT(spin_is_locked(&desc->lock));
>      /* Caller has already checked that the IRQ is an SPI */
> -    ASSERT(virq >= 32);
> -    ASSERT(virq < vgic_num_irqs(d));
> +    ASSERT(vgic_is_spi(d, virq));
>      ASSERT(!is_lpi(virq));
>  
>      ret = vgic_connect_hw_irq(d, NULL, virq, desc, true);
> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
> index 35c0c6a8b0..3e7cbbb196 100644
> --- a/xen/arch/arm/include/asm/vgic.h
> +++ b/xen/arch/arm/include/asm/vgic.h
> @@ -335,6 +335,13 @@ extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
>  /* Default number of vGIC SPIs. 32 are substracted to cover local IRQs. */
>  #define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
>  
> +extern bool vgic_is_valid_line(struct domain *d, unsigned int virq);
> +
> +static inline bool vgic_is_spi(struct domain *d, unsigned int virq)
> +{
> +    return virq >= NR_LOCAL_IRQS && vgic_is_valid_line(d, virq);
> +}
> +
>  /*
>   * Allocate a guest VIRQ
>   *  - spi == 0 => allocate a PPI. It will be the same on every vCPU
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 7dd5a2a453..b8eccfc924 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -442,7 +442,7 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
>      unsigned long flags;
>      int retval = 0;
>  
> -    if ( virq >= vgic_num_irqs(d) )
> +    if ( !vgic_is_valid_line(d, virq) )
>      {
>          printk(XENLOG_G_ERR
>                 "the vIRQ number %u is too high for domain %u (max = %u)\n",
> @@ -560,7 +560,7 @@ int release_guest_irq(struct domain *d, unsigned int virq)
>      int ret;
>  
>      /* Only SPIs are supported */
> -    if ( virq < NR_LOCAL_IRQS || virq >= vgic_num_irqs(d) )
> +    if ( !vgic_is_spi(d, virq) )
>          return -EINVAL;
>  
>      desc = vgic_get_hw_irq_desc(d, NULL, virq);
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index c563ba93af..2bbf4d99aa 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -24,6 +24,12 @@
>  #include <asm/gic.h>
>  #include <asm/vgic.h>
>  
> +
> +bool vgic_is_valid_line(struct domain *d, unsigned int virq)
> +{
> +    return virq < vgic_num_irqs(d);
> +}
> +
>  static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
>                                                    unsigned int rank)
>  {
> @@ -582,7 +588,7 @@ void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
>      if ( !v )
>      {
>          /* The IRQ needs to be an SPI if no vCPU is specified. */
> -        ASSERT(virq >= 32 && virq <= vgic_num_irqs(d));
> +        ASSERT(vgic_is_spi(d, virq));
>  
>          v = vgic_get_target_vcpu(d->vcpu[0], virq);
>      };
> @@ -659,7 +665,7 @@ bool vgic_emulate(struct cpu_user_regs *regs, union hsr hsr)
>  
>  bool vgic_reserve_virq(struct domain *d, unsigned int virq)
>  {
> -    if ( virq >= vgic_num_irqs(d) )
> +    if ( !vgic_is_valid_line(d, virq) )
>          return false;
>  
>      return !test_and_set_bit(virq, d->arch.vgic.allocated_irqs);
> diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c
> index 6cabd0496d..b2c0e1873a 100644
> --- a/xen/arch/arm/vgic/vgic.c
> +++ b/xen/arch/arm/vgic/vgic.c
> @@ -718,6 +718,11 @@ bool vgic_reserve_virq(struct domain *d, unsigned int virq)
>      return !test_and_set_bit(virq, d->arch.vgic.allocated_irqs);
>  }
>  
> +bool vgic_is_valid_line(struct domain *d, unsigned int virq)
> +{
> +    return virq < vgic_num_irqs(d);
> +}
> +
>  int vgic_allocate_virq(struct domain *d, bool spi)
>  {
>      int first, end;

-- 
WBR, Volodymyr

Re: [PATCH v4 05/12] xen/arm: gicv3: implement handling of GICv3.1 eSPI

[Volodymyr Babchuk]

Hi Leonid,

Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:

> Introduced appropriate register definitions, helper macros,
> and initialization of required GICv3.1 distributor registers
> to support eSPI. This type of interrupt is handled in the
> same way as regular SPI interrupts, with the following
> differences:
>
> 1) eSPIs can have up to 1024 interrupts, starting from the
> beginning of the range, whereas regular SPIs use INTIDs from
> 32 to 1019, totaling 988 interrupts;
> 2) eSPIs start at INTID 4096, necessitating additional interrupt
> index conversion during register operations.
>
> In case if appropriate config is disabled, or GIC HW doesn't
> support eSPI, the existing functionality will remain the same.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
> Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>

With latest changes:

Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>

>
> ---
> Changes in V4:
> - added offsets for GICD_IGRPMODRnE and GICD_NSACRnE that are required
>   for vGIC emulation
> - added a log banner with eSPI information, similar to the one for
>   regular SPI
> - added newline after ifdef and before gic_is_valid_line
> - added reviewed-by from Volodymyr Babchuk
>
> Changes in V3:
> - add __init attribute to gicv3_dist_espi_common_init
> - change open-codded eSPI register initialization to the appropriate
>   gen-mask macro
> - fixed formatting for lines with more than 80 symbols
> - introduced gicv3_dist_espi_init_aff to be able to use stubs in case of
>   CONFIG_GICV3_ESPI disabled
> - renamed parameter in the GICD_TYPER_ESPI_RANGE macro to espi_range
>   (name was taken from GIC specification) to avoid confusion
> - changed type for i variable to unsigned int since it cannot be
>   negative
>
> Changes in V2:
> - move gic_number_espis function from
>   [PATCH 08/10] xen/arm: vgic: add resource management for extended SPIs
>   to use it in the newly introduced gic_is_valid_espi
> - add gic_is_valid_espi which checks if IRQ number is in supported
>   by HW eSPI range
> - update gic_is_valid_irq conditions to allow operations with eSPIs
>
> Changes for V4:
>
> Changes in V4:
> - added offsets for GICD_IGRPMODRnE and GICD_NSACRnE that are required
>   for vGIC emulation
> - added newline after ifdef and before gic_is_valid_line
> - added reviewed-by from Volodymyr Babchuk
> - added a log banner with eSPI information, similar to the one for
>   regular SPI

Looks like your changelog is doubled :)

> ---
>  xen/arch/arm/gic-v3.c                  | 82 ++++++++++++++++++++++++++
>  xen/arch/arm/include/asm/gic.h         | 22 +++++++
>  xen/arch/arm/include/asm/gic_v3_defs.h | 38 ++++++++++++
>  3 files changed, 142 insertions(+)
>
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index a959fefebe..b939a1f490 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -485,6 +485,36 @@ static void __iomem *get_addr_by_offset(struct irq_desc *irqd, u32 offset)
>          default:
>              break;
>          }
> +#ifdef CONFIG_GICV3_ESPI
> +    case ESPI_BASE_INTID ... ESPI_MAX_INTID:
> +    {
> +        u32 irq_index = ESPI_INTID2IDX(irqd->irq);
> +
> +        switch ( offset )
> +        {
> +        case GICD_ISENABLER:
> +            return (GICD + GICD_ISENABLERnE + (irq_index / 32) * 4);
> +        case GICD_ICENABLER:
> +            return (GICD + GICD_ICENABLERnE + (irq_index / 32) * 4);
> +        case GICD_ISPENDR:
> +            return (GICD + GICD_ISPENDRnE + (irq_index / 32) * 4);
> +        case GICD_ICPENDR:
> +            return (GICD + GICD_ICPENDRnE + (irq_index / 32) * 4);
> +        case GICD_ISACTIVER:
> +            return (GICD + GICD_ISACTIVERnE + (irq_index / 32) * 4);
> +        case GICD_ICACTIVER:
> +            return (GICD + GICD_ICACTIVERnE + (irq_index / 32) * 4);
> +        case GICD_ICFGR:
> +            return (GICD + GICD_ICFGRnE + (irq_index / 16) * 4);
> +        case GICD_IROUTER:
> +            return (GICD + GICD_IROUTERnE + irq_index * 8);
> +        case GICD_IPRIORITYR:
> +            return (GICD + GICD_IPRIORITYRnE + irq_index);
> +        default:
> +            break;
> +        }
> +    }
> +#endif
>      default:
>          break;
>      }
> @@ -655,6 +685,54 @@ static void gicv3_set_irq_priority(struct irq_desc *desc,
>      spin_unlock(&gicv3.lock);
>  }
>  
> +#ifdef CONFIG_GICV3_ESPI
> +unsigned int gic_number_espis(void)
> +{
> +    return gic_hw_ops->info->nr_espi;
> +}
> +
> +static void __init gicv3_dist_espi_common_init(uint32_t type)
> +{
> +    unsigned int espi_nr, i;
> +
> +    espi_nr = min(1024U, GICD_TYPER_ESPIS_NUM(type));
> +    gicv3_info.nr_espi = espi_nr;
> +    /* The GIC HW doesn't support eSPI, so we can leave from here */
> +    if ( gicv3_info.nr_espi == 0 )
> +        return;
> +
> +    printk("GICv3: %d eSPI lines\n", gicv3_info.nr_espi);
> +
> +    for ( i = 0; i < espi_nr; i += 16 )
> +        writel_relaxed(0, GICD + GICD_ICFGRnE + (i / 16) * 4);
> +
> +    for ( i = 0; i < espi_nr; i += 4 )
> +        writel_relaxed(GIC_PRI_IRQ_ALL,
> +                       GICD + GICD_IPRIORITYRnE + (i / 4) * 4);
> +
> +    for ( i = 0; i < espi_nr; i += 32 )
> +    {
> +        writel_relaxed(GENMASK(31, 0), GICD + GICD_ICENABLERnE + (i / 32) * 4);
> +        writel_relaxed(GENMASK(31, 0), GICD + GICD_ICACTIVERnE + (i / 32) * 4);
> +    }
> +
> +    for ( i = 0; i < espi_nr; i += 32 )
> +        writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPRnE + (i / 32) * 4);
> +}
> +
> +static void __init gicv3_dist_espi_init_aff(uint64_t affinity)
> +{
> +    unsigned int i;
> +
> +    for ( i = 0; i < gicv3_info.nr_espi; i++ )
> +        writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTERnE + i * 8);
> +}
> +#else
> +static void __init gicv3_dist_espi_common_init(uint32_t type) { }
> +
> +static void __init gicv3_dist_espi_init_aff(uint64_t affinity) { }
> +#endif
> +
>  static void __init gicv3_dist_init(void)
>  {
>      uint32_t type;
> @@ -700,6 +778,8 @@ static void __init gicv3_dist_init(void)
>      for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i += 32 )
>          writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPR + (i / 32) * 4);
>  
> +    gicv3_dist_espi_common_init(type);
> +
>      gicv3_dist_wait_for_rwp();
>  
>      /* Turn on the distributor */
> @@ -713,6 +793,8 @@ static void __init gicv3_dist_init(void)
>  
>      for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i++ )
>          writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTER + i * 8);
> +
> +    gicv3_dist_espi_init_aff(affinity);
>  }
>  
>  static int gicv3_enable_redist(void)
> diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/asm/gic.h
> index 3fcee42675..1e747dcd99 100644
> --- a/xen/arch/arm/include/asm/gic.h
> +++ b/xen/arch/arm/include/asm/gic.h
> @@ -306,8 +306,26 @@ extern void gic_dump_vgic_info(struct vcpu *v);
>  
>  /* Number of interrupt lines */
>  extern unsigned int gic_number_lines(void);
> +#ifdef CONFIG_GICV3_ESPI
> +extern unsigned int gic_number_espis(void);
> +
> +static inline bool gic_is_valid_espi(unsigned int irq)
> +{
> +    return (irq >= ESPI_BASE_INTID &&
> +            irq < ESPI_IDX2INTID(gic_number_espis()));
> +}
> +#else
> +static inline bool gic_is_valid_espi(unsigned int irq)
> +{
> +    return false;
> +}
> +#endif
> +
>  static inline bool gic_is_valid_line(unsigned int irq)
>  {
> +    if ( gic_is_valid_espi(irq) )
> +        return true;
> +
>      return irq < gic_number_lines();
>  }
>  
> @@ -325,6 +343,10 @@ struct gic_info {
>      enum gic_version hw_version;
>      /* Number of GIC lines supported */
>      unsigned int nr_lines;
> +#ifdef CONFIG_GICV3_ESPI
> +    /* Number of GIC eSPI supported */
> +    unsigned int nr_espi;
> +#endif
>      /* Number of LR registers */
>      uint8_t nr_lrs;
>      /* Maintenance irq number */
> diff --git a/xen/arch/arm/include/asm/gic_v3_defs.h b/xen/arch/arm/include/asm/gic_v3_defs.h
> index 2af093e774..c10db9bd05 100644
> --- a/xen/arch/arm/include/asm/gic_v3_defs.h
> +++ b/xen/arch/arm/include/asm/gic_v3_defs.h
> @@ -37,6 +37,44 @@
>  #define GICD_IROUTER1019             (0x7FD8)
>  #define GICD_PIDR2                   (0xFFE8)
>  
> +#ifdef CONFIG_GICV3_ESPI
> +/* Additional registers for GICv3.1 */
> +#define GICD_IGROUPRnE               (0x1000)
> +#define GICD_IGROUPRnEN              (0x107C)
> +#define GICD_ISENABLERnE             (0x1200)
> +#define GICD_ISENABLERnEN            (0x127C)
> +#define GICD_ICENABLERnE             (0x1400)
> +#define GICD_ICENABLERnEN            (0x147C)
> +#define GICD_ISPENDRnE               (0x1600)
> +#define GICD_ISPENDRnEN              (0x167C)
> +#define GICD_ICPENDRnE               (0x1800)
> +#define GICD_ICPENDRnEN              (0x187C)
> +#define GICD_ISACTIVERnE             (0x1A00)
> +#define GICD_ISACTIVERnEN            (0x1A7C)
> +#define GICD_ICACTIVERnE             (0x1C00)
> +#define GICD_ICACTIVERnEN            (0x1C7C)
> +#define GICD_IPRIORITYRnE            (0x2000)
> +#define GICD_IPRIORITYRnEN           (0x23FC)
> +#define GICD_ICFGRnE                 (0x3000)
> +#define GICD_ICFGRnEN                (0x30FC)
> +#define GICD_IGRPMODRnE              (0x3400)
> +#define GICD_IGRPMODRnEN             (0x347C)
> +#define GICD_NSACRnE                 (0x3600)
> +#define GICD_NSACRnEN                (0x36FC)
> +#define GICD_IROUTERnE               (0x8000)
> +#define GICD_IROUTERnEN              (0x9FFC)
> +
> +#define GICD_TYPER_ESPI_SHIFT        8
> +#define GICD_TYPER_ESPI_RANGE_SHIFT  27
> +#define GICD_TYPER_ESPI_RANGE_MASK   (0x1F)
> +#define GICD_TYPER_ESPI              (1U << GICD_TYPER_ESPI_SHIFT)
> +#define GICD_TYPER_ESPI_RANGE(espi_range) ((((espi_range) & \
> +        GICD_TYPER_ESPI_RANGE_MASK) + 1) * 32)
> +#define GICD_TYPER_ESPIS_NUM(typer)    \
> +        (((typer) & GICD_TYPER_ESPI) ? \
> +        GICD_TYPER_ESPI_RANGE((typer) >> GICD_TYPER_ESPI_RANGE_SHIFT) : 0)
> +#endif
> +
>  /* Common between GICD_PIDR2 and GICR_PIDR2 */
>  #define GIC_PIDR2_ARCH_MASK         (0xf0)
>  #define GIC_PIDR2_ARCH_GICv3        (0x30)

-- 
WBR, Volodymyr

Re: [PATCH v4 08/12] xen/arm: vgic: add resource management for extended SPIs

[Volodymyr Babchuk]

Hi Leonid,

Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:

> This change introduces resource management in the VGIC to handle
> extended SPIs introduced in GICv3.1. The pending_irqs and
> allocated_irqs arrays are resized to support the required
> number of eSPIs, based on what is supported by the hardware and
> requested by the guest. A new field, ext_shared_irqs, is added
> to the VGIC structure to store information about eSPIs, similar
> to how shared_irqs is used for regular SPIs.
>
> Since the eSPI range starts at INTID 4096 and INTIDs between 1025
> and 4095 are reserved, helper macros are introduced to simplify the
> transformation of indices and to enable easier access to eSPI-specific
> resources. These changes prepare the VGIC for processing eSPIs as
> required by future functionality.
>
> The initialization and deinitialization paths for vgic have been updated
> to allocate and free these resources appropriately. Additionally,
> updated handling of INTIDs greater than 1024, passed from the toolstack
> during domain creation, and verification logic ensures only valid SPI or
> eSPI INTIDs are used.
>
> The existing SPI behavior remains unaffected when guests do not request
> eSPIs, GIC hardware does not support them, or the CONFIG_GICV3_ESPI
> option is disabled.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>
> ---
> Changes in V4:
> - added has_espi field to simplify determining whether a domain is able
>   to operate with eSPI

I don't think that this is a good idea. You already have an invariant that
tells if domain has eSPIs: d->nr_espis != 0. If you introduce a new
field, you now have to keep these two values coherent or deal with possible cases
like d->nr_espis == 0 && d->has_espi == true

Also, this new field is not used anywhere, so why adding it in the first
place?

> - fixed formatting issues and misspellings
>
> Changes in V3:
> - fixed formatting for lines with more than 80 symbols
> - introduced helper functions to be able to use stubs in case of
>   CONFIG_GICV3_ESPI disabled, and as a result, reduce the number of
>   #ifdefs
> - fixed checks for nr_spis in domain_vgic_init
> - updated comment about nr_spis adjustments with dom0less mention
> - moved comment with additional explanations before checks
> - used unsigned int for indexes since they cannot be negative
> - removed unnecessary parentheses
> - move vgic_ext_rank_offset to the below ifdef guard, to reduce the
>   number of ifdefs
>
> Changes in V2:
> - change is_espi_rank to is_valid_espi_rank to verify whether the array
>   element ext_shared_irqs exists. The previous version, is_espi_rank,
>   only checked if the rank index was less than the maximum possible eSPI
>   rank index, but this could potentially result in accessing a
>   non-existing array element. To address this, is_valid_espi_rank was
>   introduced, which ensures that the required eSPI rank exists
> - move gic_number_espis to
>   xen/arm: gicv3: implement handling of GICv3.1 eSPI
> - update vgic_is_valid_irq checks to allow operating with eSPIs
> - remove redundant newline in vgic_allocate_virq
> ---
>  xen/arch/arm/include/asm/vgic.h |  20 +++
>  xen/arch/arm/vgic.c             | 213 +++++++++++++++++++++++++++++++-
>  2 files changed, 230 insertions(+), 3 deletions(-)
>
> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
> index 3e7cbbb196..fb4cea73eb 100644
> --- a/xen/arch/arm/include/asm/vgic.h
> +++ b/xen/arch/arm/include/asm/vgic.h
> @@ -146,6 +146,12 @@ struct vgic_dist {
>      int nr_spis; /* Number of SPIs */
>      unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
>      struct vgic_irq_rank *shared_irqs;
> +#ifdef CONFIG_GICV3_ESPI
> +    struct vgic_irq_rank *ext_shared_irqs;
> +    int nr_espis; /* Number of extended SPIs */
> +    /* To simplify determining whether a domain is able to operate with eSPI */
> +    bool has_espi;
> +#endif
>      /*
>       * SPIs are domain global, SGIs and PPIs are per-VCPU and stored in
>       * struct arch_vcpu.
> @@ -243,6 +249,14 @@ struct vgic_ops {
>  /* Number of ranks of interrupt registers for a domain */
>  #define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_spis+31)/32)
>  
> +#ifdef CONFIG_GICV3_ESPI
> +#define DOMAIN_NR_EXT_RANKS(d) (((d)->arch.vgic.nr_espis+31)/32)
> +#define EXT_RANK_MIN (ESPI_BASE_INTID/32)
> +#define EXT_RANK_MAX ((ESPI_MAX_INTID+31)/32)
> +#define EXT_RANK_NUM2IDX(num) ((num)-EXT_RANK_MIN)
> +#define EXT_RANK_IDX2NUM(idx) ((idx)+EXT_RANK_MIN)
> +#endif
> +
>  #define vgic_lock(v)   spin_lock_irq(&(v)->domain->arch.vgic.lock)
>  #define vgic_unlock(v) spin_unlock_irq(&(v)->domain->arch.vgic.lock)
>  
> @@ -302,6 +316,12 @@ extern struct vgic_irq_rank *vgic_rank_offset(struct vcpu *v,
>                                                unsigned int b,
>                                                unsigned int n,
>                                                unsigned int s);
> +#ifdef CONFIG_GICV3_ESPI
> +extern struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v,
> +                                                  unsigned int b,
> +                                                  unsigned int n,
> +                                                  unsigned int s);
> +#endif
>  extern struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq);
>  extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned int n);
>  extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, unsigned int n);
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 2bbf4d99aa..f4b80cb05f 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -27,9 +27,82 @@
>  
>  bool vgic_is_valid_line(struct domain *d, unsigned int virq)
>  {
> +#ifdef CONFIG_GICV3_ESPI
> +    if ( virq >= ESPI_BASE_INTID &&
> +         virq < ESPI_IDX2INTID(d->arch.vgic.nr_espis) )
> +        return true;
> +#endif
> +
>      return virq < vgic_num_irqs(d);
>  }
>  
> +#ifdef CONFIG_GICV3_ESPI
> +/*
> + * Since eSPI indexes start from 4096 and numbers from 1024 to
> + * 4095 are forbidden, we need to check both lower and upper
> + * limits for ranks.
> + */
> +static inline bool is_valid_espi_rank(struct domain *d, unsigned int rank)
> +{
> +    return rank >= EXT_RANK_MIN &&
> +           EXT_RANK_NUM2IDX(rank) < DOMAIN_NR_EXT_RANKS(d);
> +}
> +
> +static inline struct vgic_irq_rank *vgic_get_espi_rank(struct vcpu *v,
> +                                                       unsigned int rank)
> +{
> +    return &v->domain->arch.vgic.ext_shared_irqs[EXT_RANK_NUM2IDX(rank)];
> +}
> +
> +static inline bool vgic_reserve_espi_virq(struct domain *d, unsigned int virq)
> +{
> +    return !test_and_set_bit(ESPI_INTID2IDX(virq) + vgic_num_irqs(d),
> +                             d->arch.vgic.allocated_irqs);
> +}
> +
> +static void arch_move_espis(struct vcpu *v)
> +{
> +    const cpumask_t *cpu_mask = cpumask_of(v->processor);
> +    struct domain *d = v->domain;
> +    struct pending_irq *p;
> +    struct vcpu *v_target;
> +    unsigned int i;
> +
> +    for ( i = ESPI_BASE_INTID;
> +          i < EXT_RANK_IDX2NUM(d->arch.vgic.nr_espis); i++ )
> +    {
> +        v_target = vgic_get_target_vcpu(v, i);
> +        p = irq_to_pending(v_target, i);
> +
> +        if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) )
> +            irq_set_affinity(p->desc, cpu_mask);
> +    }
> +}
> +#else
> +static inline bool is_valid_espi_rank(struct domain *d, unsigned int rank)
> +{
> +    return false;
> +}
> +
> +/*
> + * This function is stub and will not be called if CONFIG_GICV3_ESPI=n,
> + * because in this case, is_valid_espi_rank will always return false.
> + */
> +static inline struct vgic_irq_rank *vgic_get_espi_rank(struct vcpu *v,
> +                                                       unsigned int rank)
> +{
> +    ASSERT_UNREACHABLE();
> +    return NULL;
> +}
> +
> +static inline bool vgic_reserve_espi_virq(struct domain *d, unsigned int virq)
> +{
> +    return false;
> +}
> +
> +static void arch_move_espis(struct vcpu *v) { }
> +#endif
> +
>  static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
>                                                    unsigned int rank)
>  {
> @@ -37,6 +110,8 @@ static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
>          return v->arch.vgic.private_irqs;
>      else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
>          return &v->domain->arch.vgic.shared_irqs[rank - 1];
> +    else if ( is_valid_espi_rank(v->domain, rank) )
> +        return vgic_get_espi_rank(v, rank);
>      else
>          return NULL;
>  }
> @@ -117,6 +192,74 @@ int domain_vgic_register(struct domain *d, unsigned int *mmio_count)
>      return 0;
>  }
>  
> +#ifdef CONFIG_GICV3_ESPI
> +/*
> + * The function behavior is the same as for regular SPIs (vgic_rank_offset),
> + * but it operates with extended SPI ranks.
> + */
> +struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v, unsigned int b,
> +                                           unsigned int n, unsigned int s)
> +{
> +    unsigned int rank = REG_RANK_NR(b, (n >> s));
> +
> +    return vgic_get_rank(v, rank + EXT_RANK_MIN);
> +}
> +
> +static unsigned int vgic_num_spi_lines(struct domain *d)
> +{
> +    return d->arch.vgic.nr_spis + d->arch.vgic.nr_espis;
> +}
> +
> +static int init_vgic_espi(struct domain *d)
> +{
> +    unsigned int i, idx;
> +
> +    if ( d->arch.vgic.nr_espis == 0 )
> +        return 0;
> +
> +    d->arch.vgic.ext_shared_irqs =
> +        xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_EXT_RANKS(d));
> +    if ( d->arch.vgic.ext_shared_irqs == NULL )
> +        return -ENOMEM;
> +
> +    for ( i = d->arch.vgic.nr_spis, idx = 0;
> +          i < vgic_num_spi_lines(d); i++, idx++ )
> +        vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i],
> +                              ESPI_IDX2INTID(idx));
> +
> +    for ( i = 0; i < DOMAIN_NR_EXT_RANKS(d); i++ )
> +        vgic_rank_init(&d->arch.vgic.ext_shared_irqs[i], i, 0);
> +
> +    return 0;
> +}
> +
> +struct pending_irq *espi_to_pending(struct domain *d, unsigned int irq)
> +{
> +    irq = ESPI_INTID2IDX(irq) + d->arch.vgic.nr_spis;
> +    return &d->arch.vgic.pending_irqs[irq];
> +}
> +#else
> +static unsigned int init_vgic_espi(struct domain *d)
> +{
> +    return 0;
> +}
> +
> +static unsigned int vgic_num_spi_lines(struct domain *d)
> +{
> +    return d->arch.vgic.nr_spis;
> +}
> +
> +struct pending_irq *espi_to_pending(struct domain *d, unsigned int irq)
> +{
> +    return NULL;
> +}
> +#endif
> +
> +static unsigned int vgic_num_alloc_irqs(struct domain *d)
> +{
> +    return vgic_num_spi_lines(d) + NR_LOCAL_IRQS;
> +}
> +
>  int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>  {
>      int i;
> @@ -131,6 +274,38 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>       */
>      nr_spis = ROUNDUP(nr_spis, 32);
>  
> +#ifdef CONFIG_GICV3_ESPI
> +    /*
> +     * During domain creation, the dom0less DomUs code or toolstack specifies
> +     * the maximum INTID, which is defined in the domain config subtracted by
> +     * 32 to cover the local IRQs (please see the comment to VGIC_DEF_NR_SPIS).
> +     * To compute the actual number of eSPI that will be usable for,
> +     * add back 32.
> +     */
> +    if ( nr_spis + 32 > ESPI_IDX2INTID(NR_ESPI_IRQS) )
> +        return -EINVAL;
> +
> +    if ( nr_spis + 32 >= ESPI_BASE_INTID )
> +    {
> +        d->arch.vgic.nr_espis = min(nr_spis - ESPI_BASE_INTID + 32, 1024U);
> +        /* Verify if GIC HW can handle provided INTID */
> +        if ( d->arch.vgic.nr_espis > gic_number_espis() )
> +            return -EINVAL;
> +        /*
> +         * Set the maximum available number for regular
> +         * SPI to pass the next check
> +         */
> +        nr_spis = VGIC_DEF_NR_SPIS;
> +        d->arch.vgic.has_espi = true;
> +    }
> +    else
> +    {
> +        /* Domain will use the regular SPI range */
> +        d->arch.vgic.nr_espis = 0;
> +        d->arch.vgic.has_espi = false;
> +    }
> +#endif
> +
>      /* Limit the number of virtual SPIs supported to (1020 - 32) = 988  */
>      if ( nr_spis > (1020 - NR_LOCAL_IRQS) )
>          return -EINVAL;
> @@ -145,7 +320,7 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>          return -ENOMEM;
>  
>      d->arch.vgic.pending_irqs =
> -        xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis);
> +        xzalloc_array(struct pending_irq, vgic_num_spi_lines(d));
>      if ( d->arch.vgic.pending_irqs == NULL )
>          return -ENOMEM;
>  
> @@ -156,12 +331,16 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>      for ( i = 0; i < DOMAIN_NR_RANKS(d); i++ )
>          vgic_rank_init(&d->arch.vgic.shared_irqs[i], i + 1, 0);
>  
> +    ret = init_vgic_espi(d);
> +    if ( ret )
> +        return ret;
> +
>      ret = d->arch.vgic.handler->domain_init(d);
>      if ( ret )
>          return ret;
>  
>      d->arch.vgic.allocated_irqs =
> -        xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d)));
> +        xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_alloc_irqs(d)));
>      if ( !d->arch.vgic.allocated_irqs )
>          return -ENOMEM;
>  
> @@ -195,9 +374,27 @@ void domain_vgic_free(struct domain *d)
>          }
>      }
>  
> +#ifdef CONFIG_GICV3_ESPI
> +    for ( i = 0; i < d->arch.vgic.nr_espis; i++ )
> +    {
> +        struct pending_irq *p = spi_to_pending(d, ESPI_IDX2INTID(i));
> +
> +        if ( p->desc )
> +        {
> +            ret = release_guest_irq(d, p->irq);
> +            if ( ret )
> +                dprintk(XENLOG_G_WARNING, "d%u: Failed to release virq %u ret = %d\n",
> +                        d->domain_id, p->irq, ret);
> +        }
> +    }
> +#endif
> +
>      if ( d->arch.vgic.handler )
>          d->arch.vgic.handler->domain_free(d);
>      xfree(d->arch.vgic.shared_irqs);
> +#ifdef CONFIG_GICV3_ESPI
> +    xfree(d->arch.vgic.ext_shared_irqs);
> +#endif
>      xfree(d->arch.vgic.pending_irqs);
>      xfree(d->arch.vgic.allocated_irqs);
>  }
> @@ -331,6 +528,8 @@ void arch_move_irqs(struct vcpu *v)
>          if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) )
>              irq_set_affinity(p->desc, cpu_mask);
>      }
> +
> +    arch_move_espis(v);
>  }
>  
>  void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned int n)
> @@ -538,6 +737,8 @@ struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
>          n = &v->arch.vgic.pending_irqs[irq];
>      else if ( is_lpi(irq) )
>          n = v->domain->arch.vgic.handler->lpi_to_pending(v->domain, irq);
> +    else if ( is_espi(irq) )
> +        n = espi_to_pending(v->domain, irq);
>      else
>          n = &v->domain->arch.vgic.pending_irqs[irq - 32];
>      return n;
> @@ -547,6 +748,9 @@ struct pending_irq *spi_to_pending(struct domain *d, unsigned int irq)
>  {
>      ASSERT(irq >= NR_LOCAL_IRQS);
>  
> +    if ( is_espi(irq) )
> +        return espi_to_pending(d, irq);
> +
>      return &d->arch.vgic.pending_irqs[irq - 32];
>  }
>  
> @@ -668,6 +872,9 @@ bool vgic_reserve_virq(struct domain *d, unsigned int virq)
>      if ( !vgic_is_valid_line(d, virq) )
>          return false;
>  
> +    if ( is_espi(virq) )
> +        return vgic_reserve_espi_virq(d, virq);
> +
>      return !test_and_set_bit(virq, d->arch.vgic.allocated_irqs);
>  }
>  
> @@ -685,7 +892,7 @@ int vgic_allocate_virq(struct domain *d, bool spi)
>      else
>      {
>          first = 32;
> -        end = vgic_num_irqs(d);
> +        end = vgic_num_alloc_irqs(d);
>      }
>  
>      /*

-- 
WBR, Volodymyr

Re: [PATCH v4 09/12] xen/arm: domain_build/dom0less-build: adjust domains config to support eSPIs

[Volodymyr Babchuk]

Hi Loenid,

Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:

> The Dom0 and DomUs logic for the dom0less configuration in create_dom0() and
> arch_create_domUs() has been updated to account for extended SPIs when
> supported by the hardware and enabled with CONFIG_GICV3_ESPI. These changes
> ensure the proper calculation of the maximum number of SPIs and eSPIs available
> to Dom0 and DomUs in dom0less setups.
>
> When eSPIs are supported by the hardware and CONFIG_GICV3_ESPI is enabled, the
> maximum number of eSPI interrupts is calculated using the ESPI_BASE_INTID
> offset (4096) and is limited to 1024, with 32 IRQs subtracted. To ensure
> compatibility with non-Dom0 domains, this adjustment is applied by the
> toolstack during domain creation, while for Dom0 or DomUs in Dom0, it is
> handled directly during VGIC initialization. If eSPIs are not supported, the
> calculation defaults to using the standard SPI range, with a maximum value of
> 992 interrupt lines, as it works currently.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>

Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>

>
> ---
> Changes in V4:
> - consolidated the eSPI and SPI logic into a new inline function,
>   vgic_def_nr_spis. Without eSPI support (either due to config being
>   disabled or hardware not supporting it), it will return the regular SPI
>   range, as it works currently. There are no functional changes compared
>   with the previous patch version
> - removed VGIC_DEF_MAX_SPI macro, to reduce the number of ifdefs
>
> Changes in V3:
> - renamed macro VGIC_DEF_NR_ESPIS to more appropriate VGIC_DEF_MAX_SPI
> - added eSPI initialization for dom0less setups
> - fixed comment with mentions about dom0less builds
> - fixed formatting for lines with more than 80 symbols
> - updated commit message
>
> Changes in V2:
> - no changes
> ---
>  xen/arch/arm/dom0less-build.c   |  2 +-
>  xen/arch/arm/domain_build.c     |  2 +-
>  xen/arch/arm/include/asm/vgic.h | 21 +++++++++++++++++++++
>  3 files changed, 23 insertions(+), 2 deletions(-)
>
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index 69b9ea22ce..02d5559102 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -285,7 +285,7 @@ void __init arch_create_domUs(struct dt_device_node *node,
>      {
>          int vpl011_virq = GUEST_VPL011_SPI;
>  
> -        d_cfg->arch.nr_spis = VGIC_DEF_NR_SPIS;
> +        d_cfg->arch.nr_spis = vgic_def_nr_spis();
>  
>          /*
>           * The VPL011 virq is GUEST_VPL011_SPI, unless direct-map is
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index d91a71acfd..39eea0be00 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -2054,7 +2054,7 @@ void __init create_dom0(void)
>  
>      /* The vGIC for DOM0 is exactly emulating the hardware GIC */
>      dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
> -    dom0_cfg.arch.nr_spis = VGIC_DEF_NR_SPIS;
> +    dom0_cfg.arch.nr_spis = vgic_def_nr_spis();
>      dom0_cfg.arch.tee_type = tee_get_type();
>      dom0_cfg.max_vcpus = dom0_max_vcpus();
>  
> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
> index fb4cea73eb..11f9d216eb 100644
> --- a/xen/arch/arm/include/asm/vgic.h
> +++ b/xen/arch/arm/include/asm/vgic.h
> @@ -355,6 +355,27 @@ extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
>  /* Default number of vGIC SPIs. 32 are substracted to cover local IRQs. */
>  #define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
>  
> +static inline unsigned int vgic_def_nr_spis(void)
> +{
> +#ifdef CONFIG_GICV3_ESPI
> +    /*
> +     * Check if the hardware supports extended SPIs (even if the appropriate
> +     * config is set). If not, the common SPI range will be used. Otherwise
> +     * returns the maximum eSPI INTID, supported by HW GIC, subtracted by 32.
> +     * For non-Dom0 domains, the toolstack or arch_create_domUs function
> +     * applies the same adjustment to cover local IRQs (please, see comment
> +     * for macro that is used for regular SPIs - VGIC_DEF_NR_SPIS). We will
> +     * add back this value during VGIC initialization. This ensures consistent
> +     * handling for Dom0 and other domains. For the regular SPI range interrupts
> +     * in this case, the maximum value of VGIC_DEF_NR_SPIS will be used.
> +     */
> +    if ( gic_number_espis() > 0 )
> +        return ESPI_BASE_INTID + min(gic_number_espis(), 1024U) - 32;
> +#endif
> +
> +    return VGIC_DEF_NR_SPIS;
> +}
> +
>  extern bool vgic_is_valid_line(struct domain *d, unsigned int virq);
>  
>  static inline bool vgic_is_spi(struct domain *d, unsigned int virq)

-- 
WBR, Volodymyr

Re: [PATCH v4 11/12] doc/man: update description for nr_spis with eSPI

[Volodymyr Babchuk]

Hi Leonid,

Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:

> Since eSPI support has been introduced, update the documentation with
> the appropriate description.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>
> ---
> Changes in V4:
> - introduced this patch
> ---
>  docs/man/xl.cfg.5.pod.in | 13 ++++++++-----
>  1 file changed, 8 insertions(+), 5 deletions(-)
>
> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> index 5362fb0e9a..292ab10331 100644
> --- a/docs/man/xl.cfg.5.pod.in
> +++ b/docs/man/xl.cfg.5.pod.in
> @@ -3072,11 +3072,14 @@ interval of colors (such as "0-4").
>  =item B<nr_spis="NR_SPIS">
>  
>  An optional integer parameter specifying the number of SPIs (Shared
> -Peripheral Interrupts) to allocate for the domain. Max is 960 SPIs. If
> -the `nr_spis` parameter is not specified, the value calculated by the toolstack
> -will be used for the domain. Otherwise, the value specified by the `nr_spis`
> -parameter will be used. The number of SPIs should match the highest interrupt
> -ID that will be assigned to the domain.
> +Peripheral Interrupts) to allocate for the domain. Max is 960 for regular SPIs
> +or 5088 for eSPIs (Extended SPIs). The eSPIs includes an additional 1024 SPIs

Well, this is awkward and confusing now. You can't allocate 5088 SPIs,
because no GIC does not support this. Number of SPIs is
960 (regular SPIs) + 1024 (eSPIs) = 1984.

What you are describing is "max SPI number", but the parameter is called
"nr_spis". If we want to leave things consistent, we either need to
deprecate nr_spis in favor of max_spi_nr, or severely rework nr_spi
logic, so it can have values from 0 to 1984...

Personally, I'd prefer max_spi_nr approach, because it will be less
confusing for a user. I am curios what ARM maintainers think about this.


> +from the eSPI range (4096 to 5119) if the hardware supports extended SPIs
> +(GICv3.1+) and CONFIG_GICV3_ESPI is enabled. If the `nr_spis` parameter is not
> +specified, the value calculated by the toolstack will be used for the domain.
> +Otherwise, the value specified by the `nr_spis` parameter will be used. The
> +number of SPIs should match the highest interrupt ID that will be assigned to
> +the domain.
>  
>  =item B<trap_unmapped_accesses=BOOLEAN>

-- 
WBR, Volodymyr

Re: [PATCH v4 08/12] xen/arm: vgic: add resource management for extended SPIs

[Leonid Komarianskyi]

Hi Volodymyr,

Thank you for your review comment.

On 28.08.25 02:01, Volodymyr Babchuk wrote:
> Hi Leonid,
> 
> Leonid Komarianskyi<Leonid_Komarianskyi@epam.com> writes:
> 
>> This change introduces resource management in the VGIC to handle
>> extended SPIs introduced in GICv3.1. The pending_irqs and
>> allocated_irqs arrays are resized to support the required
>> number of eSPIs, based on what is supported by the hardware and
>> requested by the guest. A new field, ext_shared_irqs, is added
>> to the VGIC structure to store information about eSPIs, similar
>> to how shared_irqs is used for regular SPIs.
>>
>> Since the eSPI range starts at INTID 4096 and INTIDs between 1025
>> and 4095 are reserved, helper macros are introduced to simplify the
>> transformation of indices and to enable easier access to eSPI-specific
>> resources. These changes prepare the VGIC for processing eSPIs as
>> required by future functionality.
>>
>> The initialization and deinitialization paths for vgic have been updated
>> to allocate and free these resources appropriately. Additionally,
>> updated handling of INTIDs greater than 1024, passed from the toolstack
>> during domain creation, and verification logic ensures only valid SPI or
>> eSPI INTIDs are used.
>>
>> The existing SPI behavior remains unaffected when guests do not request
>> eSPIs, GIC hardware does not support them, or the CONFIG_GICV3_ESPI
>> option is disabled.
>>
>> Signed-off-by: Leonid Komarianskyi<leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V4:
>> - added has_espi field to simplify determining whether a domain is able
>>    to operate with eSPI
> I don't think that this is a good idea. You already have an invariant that
> tells if domain has eSPIs: d->nr_espis != 0. If you introduce a new
> field, you now have to keep these two values coherent or deal with possible cases
> like d->nr_espis == 0 && d->has_espi == true
> 
> Also, this new field is not used anywhere, so why adding it in the first
> place?

I just wanted to simplify the checks in the next patch:
https://patchew.org/Xen/cover.1756317702.git.leonid._5Fkomarianskyi@epam.com/6b312e1997da5abdf592f66d16067f4330431ded.1756317702.git.leonid._5Fkomarianskyi@epam.com/
e.g.:

+        if ( !v->domain->arch.vgic.has_espi )
+            goto read_reserved;

But yes, I agree that it looks redundant. Would it be okay if I drop 
this change in V5 and modify the checks in the next patch to something 
like this?

+        if ( v->domain->arch.vgic.nr_espis == 0 )
+            goto read_reserved;

Best regards,
Leonid

Re: [PATCH v4 10/12] xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers

[Leonid Komarianskyi]

Hello everyone,

Sorry for spamming, but according to the comments on the 8th patch of 
the series:
https://patchew.org/Xen/cover.1756317702.git.leonid._5Fkomarianskyi@epam.com/9e8a11b024833c1b91b8806e7708bf35b04a8f6e.1756317702.git.leonid._5Fkomarianskyi@epam.com/

As has_espi field will be removed, the following:

On 27.08.25 21:24, Leonid Komarianskyi wrote:
> @@ -1184,6 +1396,21 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info,
>       case VRANGE32(0x005C, 0x007C):
>           goto read_reserved;
>   
> +#ifdef CONFIG_GICV3_ESPI
> +    case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
> +    case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
> +    case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
> +    case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
> +    case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
> +    case VRANGE32(GICD_ISACTIVERnE, GICD_ISACTIVERnEN):
> +    case VRANGE32(GICD_ICACTIVERnE, GICD_ICACTIVERnEN):
> +    case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
> +    case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
> +    case VRANGE32(GICD_IGRPMODRnE, GICD_IGRPMODRnEN):
> +        if ( !v->domain->arch.vgic.has_espi )

...and similar code will be changed to the macro:

#define has_espi(v) ((v)->domain->arch.vgic.nr_espis != 0)

It will behave in a similar way, without any functional changes, 
assuming this solution is acceptable.

Best regards,
Leonid

Re: [PATCH v4 01/12] xen/arm: gicv3: refactor obtaining GIC addresses for common operations

[Julien Grall]

Hi Leonid,

On 27/08/2025 19:24, Leonid Komarianskyi wrote:
> Currently, many common functions perform the same operations to calculate
> GIC register addresses. This patch consolidates the similar code into
> a separate helper function to improve maintainability and reduce duplication.
> This refactoring also simplifies the implementation of eSPI support in future
> changes.
> 
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
> Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> 
> ---
> Changes in V4:
> - no changes
> 
> Changes in V3:
> - changed panic() in get_addr_by_offset() to printing warning and
>    ASSERT_UNREACHABLE()
> - added verification of return pointer from get_addr_by_offset() in the
>    callers
> - moved invocation of get_addr_by_offset() from spinlock guards, since
>    it is not necessarry
> - added RB from Volodymyr Babchuk

Procces remark, here you said the Reviewed-by from Volodymyr was added 
in v3. However, given the changes you made this should have been 
invalidated (reviewed-by means the person read the code and confirmed it 
is correct).

I see Volodymyr confirmed his reviewed-by on v3. So no issue, but this 
should have been clarified in the changelog.

> 
> Changes in V2:
> - no changes
> ---
>   xen/arch/arm/gic-v3.c          | 114 +++++++++++++++++++++++----------
>   xen/arch/arm/include/asm/irq.h |   1 +
>   2 files changed, 81 insertions(+), 34 deletions(-)
> 
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index cd3e1acf79..a959fefebe 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -445,17 +445,67 @@ static void gicv3_dump_state(const struct vcpu *v)
>       }
>   }
>   
> +static void __iomem *get_addr_by_offset(struct irq_desc *irqd, u32 offset)
> +{
> +    switch ( irqd->irq )
> +    {
> +    case 0 ... (NR_GIC_LOCAL_IRQS - 1):
> +        switch ( offset )
> +        {
> +        case GICD_ISENABLER:
> +        case GICD_ICENABLER:
> +        case GICD_ISPENDR:
> +        case GICD_ICPENDR:
> +        case GICD_ISACTIVER:
> +        case GICD_ICACTIVER:
> +            return (GICD_RDIST_SGI_BASE + offset);
> +        case GICD_ICFGR:
> +            return (GICD_RDIST_SGI_BASE + GICR_ICFGR1);
> +        case GICD_IPRIORITYR:
> +            return (GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + irqd->irq);
> +        default:
> +            break;
> +        }
> +    case NR_GIC_LOCAL_IRQS ... SPI_MAX_INTID:
> +        switch ( offset )
> +        {
> +        case GICD_ISENABLER:
> +        case GICD_ICENABLER:
> +        case GICD_ISPENDR:
> +        case GICD_ICPENDR:
> +        case GICD_ISACTIVER:
> +        case GICD_ICACTIVER:
> +            return (GICD + offset + (irqd->irq / 32) * 4);
> +        case GICD_ICFGR:
> +            return (GICD + GICD_ICFGR + (irqd->irq / 16) * 4);
> +        case GICD_IROUTER:
> +            return (GICD + GICD_IROUTER + irqd->irq * 8);
> +        case GICD_IPRIORITYR:
> +            return (GICD + GICD_IPRIORITYR + irqd->irq);
> +        default:
> +            break;
> +        }
> +    default:
> +        break;
> +    }
> +
> +    /* Something went wrong, we shouldn't be able to reach here */
> +    printk(XENLOG_WARNING "GICv3: WARNING: Invalid offset 0x%x for IRQ#%d",

NIT: I am not expecting the interrupt to be < 0. So it would be 
preferable to use %u.

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall

Re: [PATCH v4 11/12] doc/man: update description for nr_spis with eSPI

[Leonid Komarianskyi]

Hi Volodymyr,

Thank you for your review comment and suggestions.

On 28.08.25 02:20, Volodymyr Babchuk wrote:
> Hi Leonid,
> 
> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
> 
>> Since eSPI support has been introduced, update the documentation with
>> the appropriate description.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V4:
>> - introduced this patch
>> ---
>>   docs/man/xl.cfg.5.pod.in | 13 ++++++++-----
>>   1 file changed, 8 insertions(+), 5 deletions(-)
>>
>> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
>> index 5362fb0e9a..292ab10331 100644
>> --- a/docs/man/xl.cfg.5.pod.in
>> +++ b/docs/man/xl.cfg.5.pod.in
>> @@ -3072,11 +3072,14 @@ interval of colors (such as "0-4").
>>   =item B<nr_spis="NR_SPIS">
>>   
>>   An optional integer parameter specifying the number of SPIs (Shared
>> -Peripheral Interrupts) to allocate for the domain. Max is 960 SPIs. If
>> -the `nr_spis` parameter is not specified, the value calculated by the toolstack
>> -will be used for the domain. Otherwise, the value specified by the `nr_spis`
>> -parameter will be used. The number of SPIs should match the highest interrupt
>> -ID that will be assigned to the domain.
>> +Peripheral Interrupts) to allocate for the domain. Max is 960 for regular SPIs
>> +or 5088 for eSPIs (Extended SPIs). The eSPIs includes an additional 1024 SPIs
> 
> Well, this is awkward and confusing now. You can't allocate 5088 SPIs,
> because no GIC does not support this. Number of SPIs is
> 960 (regular SPIs) + 1024 (eSPIs) = 1984.
> 

I agree with this, but even if I were to update the toolstack logic with 
parsing IRQ part of domain config, it would still be confusing and 
awkward. This is because, in that case, we would need to define some 
range for eSPIs, such as a number from 960 to 1984, while in the DTS 
file, we would define the INTID of eSPIs as being from 4064 (0xFE0) to 
5088 (0x13DF). Additionally, the current mainline description:

 > The number of SPIs should match the highest interrupt ID that will be
 > assigned to the domain.

will also be incorrect in this case.

> What you are describing is "max SPI number", but the parameter is called
> "nr_spis". If we want to leave things consistent, we either need to
> deprecate nr_spis in favor of max_spi_nr, or severely rework nr_spi
> logic, so it can have values from 0 to 1984...
> 
> Personally, I'd prefer max_spi_nr approach, because it will be less
> confusing for a user. I am curios what ARM maintainers think about this.
> 

I also agree that nr_spis is not proper naming in the context of eSPIs. 
Introducing max_spi_nr and deprecating nr_spis is a good option, but 
this would require additional updates to the toolstack, documentation, etc..

Thus, I provided additional technical details in the context of eSPIs, 
which will be helpful for users even while using the current nr_spis:

 > The eSPIs includes an additional 1024 SPIs from the eSPI range (4096
 > to 5119) if the hardware supports extended SPIs (GICv3.1+) and
 > CONFIG_GICV3_ESPI is enabled.

This definitely needs to be agreed upon with ARM maintainers.

> 
>> +from the eSPI range (4096 to 5119) if the hardware supports extended SPIs
>> +(GICv3.1+) and CONFIG_GICV3_ESPI is enabled. If the `nr_spis` parameter is not
>> +specified, the value calculated by the toolstack will be used for the domain.
>> +Otherwise, the value specified by the `nr_spis` parameter will be used. The
>> +number of SPIs should match the highest interrupt ID that will be assigned to
>> +the domain.
>>   
>>   =item B<trap_unmapped_accesses=BOOLEAN>
> 

Best regards,
Leonid

Re: [PATCH v4 02/12] xen/arm: gic: implement helper functions for INTID checks

[Julien Grall]

Hi Leonid,

On 27/08/2025 19:24, Leonid Komarianskyi wrote:
> Introduced two new helper functions: gic_is_valid_line and
> gic_is_spi. The first function helps determine whether an IRQ
> number is less than the number of lines supported by hardware. The
> second function additionally checks if the IRQ number falls within the
> SPI range. Also, updated the appropriate checks to use these new helper
> functions.
> 
> The current checks for the real GIC are very similar to those for the
> vGIC but serve a different purpose. For GIC-related code, the interrupt
> numbers should be validated based on whether the hardware can operate
> with such interrupts. On the other hand, for the vGIC, the indexes must
> also be verified to ensure they are available for a specific domain. The
> first reason for introducing these helper functions is to avoid
> potential confusion with vGIC-related checks. The second reason is to
> consolidate similar code into separate functions, which can be more
> easily extended by additional conditions, e.g., when implementing
> extended SPI interrupts.
> 
> The changes, which replace open-coded checks with the use of the new
> helper functions, do not introduce any functional changes, as the helper
> functions follow the current IRQ index verification logic.
> 
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
> Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>

With one remark below:

Acked-by: Julien Grall <jgrall@amazon.com>

> ---
> Changes in V4:
> - removed redundant parentheses
> - added reviewed-by from Volodymyr Babchuk
> 
> Changes in V3:
> - renamed gic_is_valid_irq to gic_is_valid_line and gic_is_shared_irq to
>    gic_is_spi
> - updated commit message
> 
> Changes in V2:
> - introduced this patch
> 
> Changes for V4:
> 
> Changes in V4:
> - removed redundant parentheses
> - added reviewed-by from Volodymyr Babchuk
> ---
>   xen/arch/arm/gic.c             | 2 +-
>   xen/arch/arm/include/asm/gic.h | 9 +++++++++
>   xen/arch/arm/irq.c             | 2 +-
>   3 files changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index e80fe0ca24..9220eef6ea 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -111,7 +111,7 @@ static void gic_set_irq_priority(struct irq_desc *desc, unsigned int priority)
>   void gic_route_irq_to_xen(struct irq_desc *desc, unsigned int priority)
>   {
>       ASSERT(priority <= 0xff);     /* Only 8 bits of priority */
> -    ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
> +    ASSERT(gic_is_valid_line(desc->irq));/* Can't route interrupts that don't exist */

As you are touching the line. It is over 80 characters. Can you move the 
command in a separate line?

>       ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
>       ASSERT(spin_is_locked(&desc->lock));

Cheers,

-- 
Julien Grall

Re: [PATCH v4 03/12] xen/arm: vgic: implement helper functions for virq checks

[Julien Grall]

Hi Leonid,

On 27/08/2025 19:24, Leonid Komarianskyi wrote:
> Introduced two new helper functions for vGIC: vgic_is_valid_line and
> vgic_is_spi. The functions are similar to the newly introduced
> gic_is_valid_line and gic_is_spi, but they verify whether a vIRQ
> is available for a specific domain, while GIC-specific functions
> validate INTIDs for the real GIC hardware. For example, the GIC may
> support all 992 SPI lines, but the domain may use only some part of them
> (e.g., 640), depending on the highest IRQ number defined in the domain
> configuration. Therefore, for vGIC-related code and checks, the
> appropriate functions should be used. Also, updated the appropriate
> checks to use these new helper functions.
> 
> The purpose of introducing new helper functions for vGIC is essentially
> the same as for GIC: to avoid potential confusion with GIC-related
> checks and to consolidate similar code into separate functions, which
> can be more easily extended by additional conditions, e.g., when
> implementing extended SPI interrupts.
> 
> Only the validation change in vgic_inject_irq may affect existing
> functionality, as it currently checks whether the vIRQ is less than or
> equal to vgic_num_irqs. Since IRQ indexes start from 0 (where 32 is the
> first SPI), the check should behave consistently with similar logic in
> other places and should check if the vIRQ number is less than
> vgic_num_irqs. The remaining changes, which replace open-coded checks
> with the use of these new helper functions, do not introduce any
> functional changes, as the helper functions follow the current vIRQ
> index verification logic.
> 
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall

Re: [PATCH v4 03/12] xen/arm: vgic: implement helper functions for virq checks

[Oleksandr Tyshchenko]

On 27.08.25 21:24, Leonid Komarianskyi wrote:

Hello Leonid


> Introduced two new helper functions for vGIC: vgic_is_valid_line and
> vgic_is_spi. The functions are similar to the newly introduced
> gic_is_valid_line and gic_is_spi, but they verify whether a vIRQ
> is available for a specific domain, while GIC-specific functions
> validate INTIDs for the real GIC hardware. For example, the GIC may
> support all 992 SPI lines, but the domain may use only some part of them
> (e.g., 640), depending on the highest IRQ number defined in the domain
> configuration. Therefore, for vGIC-related code and checks, the
> appropriate functions should be used. Also, updated the appropriate
> checks to use these new helper functions.
> 
> The purpose of introducing new helper functions for vGIC is essentially
> the same as for GIC: to avoid potential confusion with GIC-related
> checks and to consolidate similar code into separate functions, which
> can be more easily extended by additional conditions, e.g., when
> implementing extended SPI interrupts.
> 
> Only the validation change in vgic_inject_irq may affect existing
> functionality, as it currently checks whether the vIRQ is less than or
> equal to vgic_num_irqs. Since IRQ indexes start from 0 (where 32 is the
> first SPI), the check should behave consistently with similar logic in
> other places and should check if the vIRQ number is less than
> vgic_num_irqs. The remaining changes, which replace open-coded checks
> with the use of these new helper functions, do not introduce any
> functional changes, as the helper functions follow the current vIRQ
> index verification logic.
> 
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
> 
> ---
> Changes in V4:
> - removed redundant parentheses
> 
> Changes in V3:
> - renamed vgic_is_valid_irq to vgic_is_valid_line and vgic_is_shared_irq
>    to vgic_is_spi
> - added vgic_is_valid_line implementation for new-vgic, because
>    vgic_is_valid_line is called from generic code. It is necessary to fix
>    the build for new-vgic.

The comments I provided are addressed (thanks), so you can add my:

Reviewed-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

[snip]

Re: [PATCH v4 09/12] xen/arm: domain_build/dom0less-build: adjust domains config to support eSPIs

[Oleksandr Tyshchenko]

On 27.08.25 21:24, Leonid Komarianskyi wrote:

Hello Leonid

> The Dom0 and DomUs logic for the dom0less configuration in create_dom0() and
> arch_create_domUs() has been updated to account for extended SPIs when
> supported by the hardware and enabled with CONFIG_GICV3_ESPI. These changes
> ensure the proper calculation of the maximum number of SPIs and eSPIs available
> to Dom0 and DomUs in dom0less setups.
> 
> When eSPIs are supported by the hardware and CONFIG_GICV3_ESPI is enabled, the
> maximum number of eSPI interrupts is calculated using the ESPI_BASE_INTID
> offset (4096) and is limited to 1024, with 32 IRQs subtracted. To ensure
> compatibility with non-Dom0 domains, this adjustment is applied by the
> toolstack during domain creation, while for Dom0 or DomUs in Dom0, it is
> handled directly during VGIC initialization. If eSPIs are not supported, the
> calculation defaults to using the standard SPI range, with a maximum value of
> 992 interrupt lines, as it works currently.
> 
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
> 
> ---
> Changes in V4:
> - consolidated the eSPI and SPI logic into a new inline function,
>    vgic_def_nr_spis. Without eSPI support (either due to config being
>    disabled or hardware not supporting it), it will return the regular SPI
>    range, as it works currently. There are no functional changes compared
>    with the previous patch version
> - removed VGIC_DEF_MAX_SPI macro, to reduce the number of ifdefs

very good, thanks, only NIT below

Reviewed-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>



> 
> Changes in V3:
> - renamed macro VGIC_DEF_NR_ESPIS to more appropriate VGIC_DEF_MAX_SPI
> - added eSPI initialization for dom0less setups
> - fixed comment with mentions about dom0less builds
> - fixed formatting for lines with more than 80 symbols
> - updated commit message
> 
> Changes in V2:
> - no changes
> ---
>   xen/arch/arm/dom0less-build.c   |  2 +-
>   xen/arch/arm/domain_build.c     |  2 +-
>   xen/arch/arm/include/asm/vgic.h | 21 +++++++++++++++++++++
>   3 files changed, 23 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c
> index 69b9ea22ce..02d5559102 100644
> --- a/xen/arch/arm/dom0less-build.c
> +++ b/xen/arch/arm/dom0less-build.c
> @@ -285,7 +285,7 @@ void __init arch_create_domUs(struct dt_device_node *node,
>       {
>           int vpl011_virq = GUEST_VPL011_SPI;
>   
> -        d_cfg->arch.nr_spis = VGIC_DEF_NR_SPIS;
> +        d_cfg->arch.nr_spis = vgic_def_nr_spis();
>   
>           /*
>            * The VPL011 virq is GUEST_VPL011_SPI, unless direct-map is
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index d91a71acfd..39eea0be00 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -2054,7 +2054,7 @@ void __init create_dom0(void)
>   
>       /* The vGIC for DOM0 is exactly emulating the hardware GIC */
>       dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
> -    dom0_cfg.arch.nr_spis = VGIC_DEF_NR_SPIS;
> +    dom0_cfg.arch.nr_spis = vgic_def_nr_spis();
>       dom0_cfg.arch.tee_type = tee_get_type();
>       dom0_cfg.max_vcpus = dom0_max_vcpus();
>   
> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
> index fb4cea73eb..11f9d216eb 100644
> --- a/xen/arch/arm/include/asm/vgic.h
> +++ b/xen/arch/arm/include/asm/vgic.h
> @@ -355,6 +355,27 @@ extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
>   /* Default number of vGIC SPIs. 32 are substracted to cover local IRQs. */
>   #define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
>   
> +static inline unsigned int vgic_def_nr_spis(void)
> +{
> +#ifdef CONFIG_GICV3_ESPI
> +    /*
> +     * Check if the hardware supports extended SPIs (even if the appropriate
> +     * config is set). If not, the common SPI range will be used. Otherwise
> +     * returns the maximum eSPI INTID, supported by HW GIC, subtracted by 32.

NIT: s/returns/return ...


> +     * For non-Dom0 domains, the toolstack or arch_create_domUs function
> +     * applies the same adjustment to cover local IRQs (please, see comment
> +     * for macro that is used for regular SPIs - VGIC_DEF_NR_SPIS).

  ... I am not 100% sure, that I follow the wording "For non-Dom0 
domains, the toolstack or arch_create_domUs ...". Did you perhaps mean:

For Dom0 and started at boot time DomUs ...?

  We will
> +     * add back this value during VGIC initialization. This ensures consistent
> +     * handling for Dom0 and other domains. For the regular SPI range interrupts
> +     * in this case, the maximum value of VGIC_DEF_NR_SPIS will be used.
> +     */
> +    if ( gic_number_espis() > 0 )
> +        return ESPI_BASE_INTID + min(gic_number_espis(), 1024U) - 32;
> +#endif
> +
> +    return VGIC_DEF_NR_SPIS;
> +}
> +
>   extern bool vgic_is_valid_line(struct domain *d, unsigned int virq);
>   
>   static inline bool vgic_is_spi(struct domain *d, unsigned int virq)

Re: [PATCH v4 05/12] xen/arm: gicv3: implement handling of GICv3.1 eSPI

[Oleksandr Tyshchenko]

On 27.08.25 21:24, Leonid Komarianskyi wrote:

Hello Leonid


> Introduced appropriate register definitions, helper macros,
> and initialization of required GICv3.1 distributor registers
> to support eSPI. This type of interrupt is handled in the
> same way as regular SPI interrupts, with the following
> differences:
> 
> 1) eSPIs can have up to 1024 interrupts, starting from the
> beginning of the range, whereas regular SPIs use INTIDs from
> 32 to 1019, totaling 988 interrupts;
> 2) eSPIs start at INTID 4096, necessitating additional interrupt
> index conversion during register operations.
> 
> In case if appropriate config is disabled, or GIC HW doesn't
> support eSPI, the existing functionality will remain the same.
> 
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
> Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> 
> ---
> Changes in V4:
> - added offsets for GICD_IGRPMODRnE and GICD_NSACRnE that are required
>    for vGIC emulation
> - added a log banner with eSPI information, similar to the one for
>    regular SPI
> - added newline after ifdef and before gic_is_valid_line
> - added reviewed-by from Volodymyr Babchuk

only NITs below

Reviewed-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

> 
> Changes in V3:
> - add __init attribute to gicv3_dist_espi_common_init
> - change open-codded eSPI register initialization to the appropriate
>    gen-mask macro
> - fixed formatting for lines with more than 80 symbols
> - introduced gicv3_dist_espi_init_aff to be able to use stubs in case of
>    CONFIG_GICV3_ESPI disabled
> - renamed parameter in the GICD_TYPER_ESPI_RANGE macro to espi_range
>    (name was taken from GIC specification) to avoid confusion
> - changed type for i variable to unsigned int since it cannot be
>    negative
> 
> Changes in V2:
> - move gic_number_espis function from
>    [PATCH 08/10] xen/arm: vgic: add resource management for extended SPIs
>    to use it in the newly introduced gic_is_valid_espi
> - add gic_is_valid_espi which checks if IRQ number is in supported
>    by HW eSPI range
> - update gic_is_valid_irq conditions to allow operations with eSPIs
> 
> Changes for V4:
> 
> Changes in V4:
> - added offsets for GICD_IGRPMODRnE and GICD_NSACRnE that are required
>    for vGIC emulation
> - added newline after ifdef and before gic_is_valid_line
> - added reviewed-by from Volodymyr Babchuk
> - added a log banner with eSPI information, similar to the one for
>    regular SPI
> ---
>   xen/arch/arm/gic-v3.c                  | 82 ++++++++++++++++++++++++++
>   xen/arch/arm/include/asm/gic.h         | 22 +++++++
>   xen/arch/arm/include/asm/gic_v3_defs.h | 38 ++++++++++++
>   3 files changed, 142 insertions(+)
> 
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index a959fefebe..b939a1f490 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -485,6 +485,36 @@ static void __iomem *get_addr_by_offset(struct irq_desc *irqd, u32 offset)
>           default:
>               break;
>           }
> +#ifdef CONFIG_GICV3_ESPI
> +    case ESPI_BASE_INTID ... ESPI_MAX_INTID:
> +    {
> +        u32 irq_index = ESPI_INTID2IDX(irqd->irq);

NIT: I have heard that no uN for new code, but uintN_t (sorry for didn't 
spot this before), so I would use uint32_t


> +
> +        switch ( offset )
> +        {
> +        case GICD_ISENABLER:
> +            return (GICD + GICD_ISENABLERnE + (irq_index / 32) * 4);
> +        case GICD_ICENABLER:
> +            return (GICD + GICD_ICENABLERnE + (irq_index / 32) * 4);
> +        case GICD_ISPENDR:
> +            return (GICD + GICD_ISPENDRnE + (irq_index / 32) * 4);
> +        case GICD_ICPENDR:
> +            return (GICD + GICD_ICPENDRnE + (irq_index / 32) * 4);
> +        case GICD_ISACTIVER:
> +            return (GICD + GICD_ISACTIVERnE + (irq_index / 32) * 4);
> +        case GICD_ICACTIVER:
> +            return (GICD + GICD_ICACTIVERnE + (irq_index / 32) * 4);
> +        case GICD_ICFGR:
> +            return (GICD + GICD_ICFGRnE + (irq_index / 16) * 4);
> +        case GICD_IROUTER:
> +            return (GICD + GICD_IROUTERnE + irq_index * 8);
> +        case GICD_IPRIORITYR:
> +            return (GICD + GICD_IPRIORITYRnE + irq_index);
> +        default:
> +            break;
> +        }
> +    }
> +#endif
>       default:
>           break;
>       }
> @@ -655,6 +685,54 @@ static void gicv3_set_irq_priority(struct irq_desc *desc,
>       spin_unlock(&gicv3.lock);
>   }
>   
> +#ifdef CONFIG_GICV3_ESPI
> +unsigned int gic_number_espis(void)
> +{
> +    return gic_hw_ops->info->nr_espi;
> +}
> +
> +static void __init gicv3_dist_espi_common_init(uint32_t type)
> +{
> +    unsigned int espi_nr, i;
> +
> +    espi_nr = min(1024U, GICD_TYPER_ESPIS_NUM(type));
> +    gicv3_info.nr_espi = espi_nr;
> +    /* The GIC HW doesn't support eSPI, so we can leave from here */
> +    if ( gicv3_info.nr_espi == 0 )
> +        return;
> +
> +    printk("GICv3: %d eSPI lines\n", gicv3_info.nr_espi);
> +
> +    for ( i = 0; i < espi_nr; i += 16 )
> +        writel_relaxed(0, GICD + GICD_ICFGRnE + (i / 16) * 4);
> +
> +    for ( i = 0; i < espi_nr; i += 4 )
> +        writel_relaxed(GIC_PRI_IRQ_ALL,
> +                       GICD + GICD_IPRIORITYRnE + (i / 4) * 4);
> +
> +    for ( i = 0; i < espi_nr; i += 32 )
> +    {
> +        writel_relaxed(GENMASK(31, 0), GICD + GICD_ICENABLERnE + (i / 32) * 4);
> +        writel_relaxed(GENMASK(31, 0), GICD + GICD_ICACTIVERnE + (i / 32) * 4);
> +    }
> +
> +    for ( i = 0; i < espi_nr; i += 32 )
> +        writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPRnE + (i / 32) * 4);

NIT: From what I see, the eSPIs are configured exactly as regular SPIs 
in gicv3_dist_init() (i.e. the eSPIs are level-triggered, disabled and 
deactivated, belong to the same group, etc). In gicv3_dist_init() we 
have comments clarying the actions, but here we do not. I would at least 
write a sentence in patch description/in-code comment saying that eSPIs 
configuration is the same as for regular SPIs.


> +}
> +
> +static void __init gicv3_dist_espi_init_aff(uint64_t affinity)
> +{
> +    unsigned int i;
> +
> +    for ( i = 0; i < gicv3_info.nr_espi; i++ )
> +        writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTERnE + i * 8);
> +}
> +#else
> +static void __init gicv3_dist_espi_common_init(uint32_t type) { }
> +
> +static void __init gicv3_dist_espi_init_aff(uint64_t affinity) { }
> +#endif
> +
>   static void __init gicv3_dist_init(void)
>   {
>       uint32_t type;
> @@ -700,6 +778,8 @@ static void __init gicv3_dist_init(void)
>       for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i += 32 )
>           writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPR + (i / 32) * 4);
>   
> +    gicv3_dist_espi_common_init(type);
> +
>       gicv3_dist_wait_for_rwp();
>   
>       /* Turn on the distributor */
> @@ -713,6 +793,8 @@ static void __init gicv3_dist_init(void)
>   
>       for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i++ )
>           writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTER + i * 8);
> +
> +    gicv3_dist_espi_init_aff(affinity);
>   }
>   
>   static int gicv3_enable_redist(void)


[snip]

Re: [PATCH v4 01/12] xen/arm: gicv3: refactor obtaining GIC addresses for common operations

[Leonid Komarianskyi]

Hi Julien,


On 28.08.25 15:00, Julien Grall wrote:
> Hi Leonid,
> 
> On 27/08/2025 19:24, Leonid Komarianskyi wrote:
>> Currently, many common functions perform the same operations to calculate
>> GIC register addresses. This patch consolidates the similar code into
>> a separate helper function to improve maintainability and reduce 
>> duplication.
>> This refactoring also simplifies the implementation of eSPI support in 
>> future
>> changes.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>> Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
>>
>> ---
>> Changes in V4:
>> - no changes
>>
>> Changes in V3:
>> - changed panic() in get_addr_by_offset() to printing warning and
>>    ASSERT_UNREACHABLE()
>> - added verification of return pointer from get_addr_by_offset() in the
>>    callers
>> - moved invocation of get_addr_by_offset() from spinlock guards, since
>>    it is not necessarry
>> - added RB from Volodymyr Babchuk
> 
> Procces remark, here you said the Reviewed-by from Volodymyr was added 
> in v3. However, given the changes you made this should have been 
> invalidated (reviewed-by means the person read the code and confirmed it 
> is correct).
> 
> I see Volodymyr confirmed his reviewed-by on v3. So no issue, but this 
> should have been clarified in the changelog.
> 

Thank you for your explanation.
Just to clarify: would it be okay to leave the RB tag (with appropriate 
text in the changelog) if I fix some minor nit from another reviewer in 
the next version, like in this patch?

>>
>> Changes in V2:
>> - no changes
>> ---
>>   xen/arch/arm/gic-v3.c          | 114 +++++++++++++++++++++++----------
>>   xen/arch/arm/include/asm/irq.h |   1 +
>>   2 files changed, 81 insertions(+), 34 deletions(-)
>>
>> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
>> index cd3e1acf79..a959fefebe 100644
>> --- a/xen/arch/arm/gic-v3.c
>> +++ b/xen/arch/arm/gic-v3.c
>> @@ -445,17 +445,67 @@ static void gicv3_dump_state(const struct vcpu *v)
>>       }
>>   }
>> +static void __iomem *get_addr_by_offset(struct irq_desc *irqd, u32 
>> offset)
>> +{
>> +    switch ( irqd->irq )
>> +    {
>> +    case 0 ... (NR_GIC_LOCAL_IRQS - 1):
>> +        switch ( offset )
>> +        {
>> +        case GICD_ISENABLER:
>> +        case GICD_ICENABLER:
>> +        case GICD_ISPENDR:
>> +        case GICD_ICPENDR:
>> +        case GICD_ISACTIVER:
>> +        case GICD_ICACTIVER:
>> +            return (GICD_RDIST_SGI_BASE + offset);
>> +        case GICD_ICFGR:
>> +            return (GICD_RDIST_SGI_BASE + GICR_ICFGR1);
>> +        case GICD_IPRIORITYR:
>> +            return (GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + irqd->irq);
>> +        default:
>> +            break;
>> +        }
>> +    case NR_GIC_LOCAL_IRQS ... SPI_MAX_INTID:
>> +        switch ( offset )
>> +        {
>> +        case GICD_ISENABLER:
>> +        case GICD_ICENABLER:
>> +        case GICD_ISPENDR:
>> +        case GICD_ICPENDR:
>> +        case GICD_ISACTIVER:
>> +        case GICD_ICACTIVER:
>> +            return (GICD + offset + (irqd->irq / 32) * 4);
>> +        case GICD_ICFGR:
>> +            return (GICD + GICD_ICFGR + (irqd->irq / 16) * 4);
>> +        case GICD_IROUTER:
>> +            return (GICD + GICD_IROUTER + irqd->irq * 8);
>> +        case GICD_IPRIORITYR:
>> +            return (GICD + GICD_IPRIORITYR + irqd->irq);
>> +        default:
>> +            break;
>> +        }
>> +    default:
>> +        break;
>> +    }
>> +
>> +    /* Something went wrong, we shouldn't be able to reach here */
>> +    printk(XENLOG_WARNING "GICv3: WARNING: Invalid offset 0x%x for 
>> IRQ#%d",
> 
> NIT: I am not expecting the interrupt to be < 0. So it would be 
> preferable to use %u.
> 
> Acked-by: Julien Grall <jgrall@amazon.com>
> 
> Cheers,
> 

Thank you for your AB. I will fix the nit in V5.

Best regards,
Leonid

Re: [PATCH v4 02/12] xen/arm: gic: implement helper functions for INTID checks

[Leonid Komarianskyi]

Hi Julien,

Thank you for your review.

On 28.08.25 15:10, Julien Grall wrote:
> Hi Leonid,
> 
> On 27/08/2025 19:24, Leonid Komarianskyi wrote:
>> Introduced two new helper functions: gic_is_valid_line and
>> gic_is_spi. The first function helps determine whether an IRQ
>> number is less than the number of lines supported by hardware. The
>> second function additionally checks if the IRQ number falls within the
>> SPI range. Also, updated the appropriate checks to use these new helper
>> functions.
>>
>> The current checks for the real GIC are very similar to those for the
>> vGIC but serve a different purpose. For GIC-related code, the interrupt
>> numbers should be validated based on whether the hardware can operate
>> with such interrupts. On the other hand, for the vGIC, the indexes must
>> also be verified to ensure they are available for a specific domain. The
>> first reason for introducing these helper functions is to avoid
>> potential confusion with vGIC-related checks. The second reason is to
>> consolidate similar code into separate functions, which can be more
>> easily extended by additional conditions, e.g., when implementing
>> extended SPI interrupts.
>>
>> The changes, which replace open-coded checks with the use of the new
>> helper functions, do not introduce any functional changes, as the helper
>> functions follow the current IRQ index verification logic.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>> Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> 
> With one remark below:
> 
> Acked-by: Julien Grall <jgrall@amazon.com>
> 
>> ---
>> Changes in V4:
>> - removed redundant parentheses
>> - added reviewed-by from Volodymyr Babchuk
>>
>> Changes in V3:
>> - renamed gic_is_valid_irq to gic_is_valid_line and gic_is_shared_irq to
>>    gic_is_spi
>> - updated commit message
>>
>> Changes in V2:
>> - introduced this patch
>>
>> Changes for V4:
>>
>> Changes in V4:
>> - removed redundant parentheses
>> - added reviewed-by from Volodymyr Babchuk
>> ---
>>   xen/arch/arm/gic.c             | 2 +-
>>   xen/arch/arm/include/asm/gic.h | 9 +++++++++
>>   xen/arch/arm/irq.c             | 2 +-
>>   3 files changed, 11 insertions(+), 2 deletions(-)
>>
>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>> index e80fe0ca24..9220eef6ea 100644
>> --- a/xen/arch/arm/gic.c
>> +++ b/xen/arch/arm/gic.c
>> @@ -111,7 +111,7 @@ static void gic_set_irq_priority(struct irq_desc 
>> *desc, unsigned int priority)
>>   void gic_route_irq_to_xen(struct irq_desc *desc, unsigned int priority)
>>   {
>>       ASSERT(priority <= 0xff);     /* Only 8 bits of priority */
>> -    ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts 
>> that don't exist */
>> +    ASSERT(gic_is_valid_line(desc->irq));/* Can't route interrupts 
>> that don't exist */
> 
> As you are touching the line. It is over 80 characters. Can you move the 
> command in a separate line?

Sure, I will fix formatting in V5.

> 
>>       ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
>>       ASSERT(spin_is_locked(&desc->lock));
> 
> Cheers,
> 

Best regards,
Leonid

Re: [PATCH v4 05/12] xen/arm: gicv3: implement handling of GICv3.1 eSPI

[Leonid Komarianskyi]

Hi Oleksandr,

Thank you for your review.

On 28.08.25 17:15, Oleksandr Tyshchenko wrote:
> 
> 
> On 27.08.25 21:24, Leonid Komarianskyi wrote:
> 
> Hello Leonid
> 
> 
>> Introduced appropriate register definitions, helper macros,
>> and initialization of required GICv3.1 distributor registers
>> to support eSPI. This type of interrupt is handled in the
>> same way as regular SPI interrupts, with the following
>> differences:
>>
>> 1) eSPIs can have up to 1024 interrupts, starting from the
>> beginning of the range, whereas regular SPIs use INTIDs from
>> 32 to 1019, totaling 988 interrupts;
>> 2) eSPIs start at INTID 4096, necessitating additional interrupt
>> index conversion during register operations.
>>
>> In case if appropriate config is disabled, or GIC HW doesn't
>> support eSPI, the existing functionality will remain the same.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>> Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
>>
>> ---
>> Changes in V4:
>> - added offsets for GICD_IGRPMODRnE and GICD_NSACRnE that are required
>>    for vGIC emulation
>> - added a log banner with eSPI information, similar to the one for
>>    regular SPI
>> - added newline after ifdef and before gic_is_valid_line
>> - added reviewed-by from Volodymyr Babchuk
> 
> only NITs below
> 
> Reviewed-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
>>
>> Changes in V3:
>> - add __init attribute to gicv3_dist_espi_common_init
>> - change open-codded eSPI register initialization to the appropriate
>>    gen-mask macro
>> - fixed formatting for lines with more than 80 symbols
>> - introduced gicv3_dist_espi_init_aff to be able to use stubs in case of
>>    CONFIG_GICV3_ESPI disabled
>> - renamed parameter in the GICD_TYPER_ESPI_RANGE macro to espi_range
>>    (name was taken from GIC specification) to avoid confusion
>> - changed type for i variable to unsigned int since it cannot be
>>    negative
>>
>> Changes in V2:
>> - move gic_number_espis function from
>>    [PATCH 08/10] xen/arm: vgic: add resource management for extended SPIs
>>    to use it in the newly introduced gic_is_valid_espi
>> - add gic_is_valid_espi which checks if IRQ number is in supported
>>    by HW eSPI range
>> - update gic_is_valid_irq conditions to allow operations with eSPIs
>>
>> Changes for V4:
>>
>> Changes in V4:
>> - added offsets for GICD_IGRPMODRnE and GICD_NSACRnE that are required
>>    for vGIC emulation
>> - added newline after ifdef and before gic_is_valid_line
>> - added reviewed-by from Volodymyr Babchuk
>> - added a log banner with eSPI information, similar to the one for
>>    regular SPI
>> ---
>>   xen/arch/arm/gic-v3.c                  | 82 ++++++++++++++++++++++++++
>>   xen/arch/arm/include/asm/gic.h         | 22 +++++++
>>   xen/arch/arm/include/asm/gic_v3_defs.h | 38 ++++++++++++
>>   3 files changed, 142 insertions(+)
>>
>> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
>> index a959fefebe..b939a1f490 100644
>> --- a/xen/arch/arm/gic-v3.c
>> +++ b/xen/arch/arm/gic-v3.c
>> @@ -485,6 +485,36 @@ static void __iomem *get_addr_by_offset(struct 
>> irq_desc *irqd, u32 offset)
>>           default:
>>               break;
>>           }
>> +#ifdef CONFIG_GICV3_ESPI
>> +    case ESPI_BASE_INTID ... ESPI_MAX_INTID:
>> +    {
>> +        u32 irq_index = ESPI_INTID2IDX(irqd->irq);
> 
> NIT: I have heard that no uN for new code, but uintN_t (sorry for didn't 
> spot this before), so I would use uint32_t
> 

Okay, I will change u32 to uint32_t in V5.

> 
>> +
>> +        switch ( offset )
>> +        {
>> +        case GICD_ISENABLER:
>> +            return (GICD + GICD_ISENABLERnE + (irq_index / 32) * 4);
>> +        case GICD_ICENABLER:
>> +            return (GICD + GICD_ICENABLERnE + (irq_index / 32) * 4);
>> +        case GICD_ISPENDR:
>> +            return (GICD + GICD_ISPENDRnE + (irq_index / 32) * 4);
>> +        case GICD_ICPENDR:
>> +            return (GICD + GICD_ICPENDRnE + (irq_index / 32) * 4);
>> +        case GICD_ISACTIVER:
>> +            return (GICD + GICD_ISACTIVERnE + (irq_index / 32) * 4);
>> +        case GICD_ICACTIVER:
>> +            return (GICD + GICD_ICACTIVERnE + (irq_index / 32) * 4);
>> +        case GICD_ICFGR:
>> +            return (GICD + GICD_ICFGRnE + (irq_index / 16) * 4);
>> +        case GICD_IROUTER:
>> +            return (GICD + GICD_IROUTERnE + irq_index * 8);
>> +        case GICD_IPRIORITYR:
>> +            return (GICD + GICD_IPRIORITYRnE + irq_index);
>> +        default:
>> +            break;
>> +        }
>> +    }
>> +#endif
>>       default:
>>           break;
>>       }
>> @@ -655,6 +685,54 @@ static void gicv3_set_irq_priority(struct 
>> irq_desc *desc,
>>       spin_unlock(&gicv3.lock);
>>   }
>> +#ifdef CONFIG_GICV3_ESPI
>> +unsigned int gic_number_espis(void)
>> +{
>> +    return gic_hw_ops->info->nr_espi;
>> +}
>> +
>> +static void __init gicv3_dist_espi_common_init(uint32_t type)
>> +{
>> +    unsigned int espi_nr, i;
>> +
>> +    espi_nr = min(1024U, GICD_TYPER_ESPIS_NUM(type));
>> +    gicv3_info.nr_espi = espi_nr;
>> +    /* The GIC HW doesn't support eSPI, so we can leave from here */
>> +    if ( gicv3_info.nr_espi == 0 )
>> +        return;
>> +
>> +    printk("GICv3: %d eSPI lines\n", gicv3_info.nr_espi);
>> +
>> +    for ( i = 0; i < espi_nr; i += 16 )
>> +        writel_relaxed(0, GICD + GICD_ICFGRnE + (i / 16) * 4);
>> +
>> +    for ( i = 0; i < espi_nr; i += 4 )
>> +        writel_relaxed(GIC_PRI_IRQ_ALL,
>> +                       GICD + GICD_IPRIORITYRnE + (i / 4) * 4);
>> +
>> +    for ( i = 0; i < espi_nr; i += 32 )
>> +    {
>> +        writel_relaxed(GENMASK(31, 0), GICD + GICD_ICENABLERnE + (i / 
>> 32) * 4);
>> +        writel_relaxed(GENMASK(31, 0), GICD + GICD_ICACTIVERnE + (i / 
>> 32) * 4);
>> +    }
>> +
>> +    for ( i = 0; i < espi_nr; i += 32 )
>> +        writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPRnE + (i / 
>> 32) * 4);
> 
> NIT: From what I see, the eSPIs are configured exactly as regular SPIs 
> in gicv3_dist_init() (i.e. the eSPIs are level-triggered, disabled and 
> deactivated, belong to the same group, etc). In gicv3_dist_init() we 
> have comments clarying the actions, but here we do not. I would at least 
> write a sentence in patch description/in-code comment saying that eSPIs 
> configuration is the same as for regular SPIs.
> 

Sure, I will add comments in the code, similar to how it is done for 
regular SPIs.

> 
>> +}
>> +
>> +static void __init gicv3_dist_espi_init_aff(uint64_t affinity)
>> +{
>> +    unsigned int i;
>> +
>> +    for ( i = 0; i < gicv3_info.nr_espi; i++ )
>> +        writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTERnE + i 
>> * 8);
>> +}
>> +#else
>> +static void __init gicv3_dist_espi_common_init(uint32_t type) { }
>> +
>> +static void __init gicv3_dist_espi_init_aff(uint64_t affinity) { }
>> +#endif
>> +
>>   static void __init gicv3_dist_init(void)
>>   {
>>       uint32_t type;
>> @@ -700,6 +778,8 @@ static void __init gicv3_dist_init(void)
>>       for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i += 32 )
>>           writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPR + (i / 
>> 32) * 4);
>> +    gicv3_dist_espi_common_init(type);
>> +
>>       gicv3_dist_wait_for_rwp();
>>       /* Turn on the distributor */
>> @@ -713,6 +793,8 @@ static void __init gicv3_dist_init(void)
>>       for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i++ )
>>           writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTER + i 
>> * 8);
>> +
>> +    gicv3_dist_espi_init_aff(affinity);
>>   }
>>   static int gicv3_enable_redist(void)
> 
> 
> [snip]
> 

Best regards,
Leonid

Re: [PATCH v4 09/12] xen/arm: domain_build/dom0less-build: adjust domains config to support eSPIs

[Leonid Komarianskyi]

Hi Oleksandr,

Thank you for your review.

On 28.08.25 16:19, Oleksandr Tyshchenko wrote:
> 
> 
> On 27.08.25 21:24, Leonid Komarianskyi wrote:
> 
> Hello Leonid
> 
>> The Dom0 and DomUs logic for the dom0less configuration in 
>> create_dom0() and
>> arch_create_domUs() has been updated to account for extended SPIs when
>> supported by the hardware and enabled with CONFIG_GICV3_ESPI. These 
>> changes
>> ensure the proper calculation of the maximum number of SPIs and eSPIs 
>> available
>> to Dom0 and DomUs in dom0less setups.
>>
>> When eSPIs are supported by the hardware and CONFIG_GICV3_ESPI is 
>> enabled, the
>> maximum number of eSPI interrupts is calculated using the ESPI_BASE_INTID
>> offset (4096) and is limited to 1024, with 32 IRQs subtracted. To ensure
>> compatibility with non-Dom0 domains, this adjustment is applied by the
>> toolstack during domain creation, while for Dom0 or DomUs in Dom0, it is
>> handled directly during VGIC initialization. If eSPIs are not 
>> supported, the
>> calculation defaults to using the standard SPI range, with a maximum 
>> value of
>> 992 interrupt lines, as it works currently.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V4:
>> - consolidated the eSPI and SPI logic into a new inline function,
>>    vgic_def_nr_spis. Without eSPI support (either due to config being
>>    disabled or hardware not supporting it), it will return the regular 
>> SPI
>>    range, as it works currently. There are no functional changes compared
>>    with the previous patch version
>> - removed VGIC_DEF_MAX_SPI macro, to reduce the number of ifdefs
> 
> very good, thanks, only NIT below
> 
> Reviewed-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> 
> 
>>
>> Changes in V3:
>> - renamed macro VGIC_DEF_NR_ESPIS to more appropriate VGIC_DEF_MAX_SPI
>> - added eSPI initialization for dom0less setups
>> - fixed comment with mentions about dom0less builds
>> - fixed formatting for lines with more than 80 symbols
>> - updated commit message
>>
>> Changes in V2:
>> - no changes
>> ---
>>   xen/arch/arm/dom0less-build.c   |  2 +-
>>   xen/arch/arm/domain_build.c     |  2 +-
>>   xen/arch/arm/include/asm/vgic.h | 21 +++++++++++++++++++++
>>   3 files changed, 23 insertions(+), 2 deletions(-)
>>
>> diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less- 
>> build.c
>> index 69b9ea22ce..02d5559102 100644
>> --- a/xen/arch/arm/dom0less-build.c
>> +++ b/xen/arch/arm/dom0less-build.c
>> @@ -285,7 +285,7 @@ void __init arch_create_domUs(struct 
>> dt_device_node *node,
>>       {
>>           int vpl011_virq = GUEST_VPL011_SPI;
>> -        d_cfg->arch.nr_spis = VGIC_DEF_NR_SPIS;
>> +        d_cfg->arch.nr_spis = vgic_def_nr_spis();
>>           /*
>>            * The VPL011 virq is GUEST_VPL011_SPI, unless direct-map is
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index d91a71acfd..39eea0be00 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -2054,7 +2054,7 @@ void __init create_dom0(void)
>>       /* The vGIC for DOM0 is exactly emulating the hardware GIC */
>>       dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
>> -    dom0_cfg.arch.nr_spis = VGIC_DEF_NR_SPIS;
>> +    dom0_cfg.arch.nr_spis = vgic_def_nr_spis();
>>       dom0_cfg.arch.tee_type = tee_get_type();
>>       dom0_cfg.max_vcpus = dom0_max_vcpus();
>> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/ 
>> asm/vgic.h
>> index fb4cea73eb..11f9d216eb 100644
>> --- a/xen/arch/arm/include/asm/vgic.h
>> +++ b/xen/arch/arm/include/asm/vgic.h
>> @@ -355,6 +355,27 @@ extern void 
>> vgic_check_inflight_irqs_pending(struct vcpu *v,
>>   /* Default number of vGIC SPIs. 32 are substracted to cover local 
>> IRQs. */
>>   #define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
>> +static inline unsigned int vgic_def_nr_spis(void)
>> +{
>> +#ifdef CONFIG_GICV3_ESPI
>> +    /*
>> +     * Check if the hardware supports extended SPIs (even if the 
>> appropriate
>> +     * config is set). If not, the common SPI range will be used. 
>> Otherwise
>> +     * returns the maximum eSPI INTID, supported by HW GIC, 
>> subtracted by 32.
> 
> NIT: s/returns/return ...
> 

I will fix this in V5.

> 
>> +     * For non-Dom0 domains, the toolstack or arch_create_domUs function
>> +     * applies the same adjustment to cover local IRQs (please, see 
>> comment
>> +     * for macro that is used for regular SPIs - VGIC_DEF_NR_SPIS).
> 
>   ... I am not 100% sure, that I follow the wording "For non-Dom0 
> domains, the toolstack or arch_create_domUs ...". Did you perhaps mean:
> 
> For Dom0 and started at boot time DomUs ...?
> 
>   We will

Yes, I agree that 'For Dom0 and started at boot time DomUs, we will add 
back...' sounds better :)

>> +     * add back this value during VGIC initialization. This ensures 
>> consistent
>> +     * handling for Dom0 and other domains. For the regular SPI range 
>> interrupts
>> +     * in this case, the maximum value of VGIC_DEF_NR_SPIS will be used.
>> +     */
>> +    if ( gic_number_espis() > 0 )
>> +        return ESPI_BASE_INTID + min(gic_number_espis(), 1024U) - 32;
>> +#endif
>> +
>> +    return VGIC_DEF_NR_SPIS;
>> +}
>> +
>>   extern bool vgic_is_valid_line(struct domain *d, unsigned int virq);
>>   static inline bool vgic_is_spi(struct domain *d, unsigned int virq)
> 

Best regards,
Leonid

Re: [PATCH v4 08/12] xen/arm: vgic: add resource management for extended SPIs

[Oleksandr Tyshchenko]

On 27.08.25 21:24, Leonid Komarianskyi wrote:

Hello Leonid

> This change introduces resource management in the VGIC to handle
> extended SPIs introduced in GICv3.1. The pending_irqs and
> allocated_irqs arrays are resized to support the required
> number of eSPIs, based on what is supported by the hardware and
> requested by the guest. A new field, ext_shared_irqs, is added
> to the VGIC structure to store information about eSPIs, similar
> to how shared_irqs is used for regular SPIs.
> 
> Since the eSPI range starts at INTID 4096 and INTIDs between 1025
> and 4095 are reserved, helper macros are introduced to simplify the
> transformation of indices and to enable easier access to eSPI-specific
> resources. These changes prepare the VGIC for processing eSPIs as
> required by future functionality.
> 
> The initialization and deinitialization paths for vgic have been updated
> to allocate and free these resources appropriately. Additionally,
> updated handling of INTIDs greater than 1024, passed from the toolstack
> during domain creation, and verification logic ensures only valid SPI or
> eSPI INTIDs are used.
> 
> The existing SPI behavior remains unaffected when guests do not request
> eSPIs, GIC hardware does not support them, or the CONFIG_GICV3_ESPI
> option is disabled.
> 
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
> 
> ---
> Changes in V4:
> - added has_espi field to simplify determining whether a domain is able
>    to operate with eSPI
> - fixed formatting issues and misspellings
> 
> Changes in V3:
> - fixed formatting for lines with more than 80 symbols
> - introduced helper functions to be able to use stubs in case of
>    CONFIG_GICV3_ESPI disabled, and as a result, reduce the number of
>    #ifdefs
> - fixed checks for nr_spis in domain_vgic_init
> - updated comment about nr_spis adjustments with dom0less mention
> - moved comment with additional explanations before checks
> - used unsigned int for indexes since they cannot be negative
> - removed unnecessary parentheses
> - move vgic_ext_rank_offset to the below ifdef guard, to reduce the
>    number of ifdefs
> 
> Changes in V2:
> - change is_espi_rank to is_valid_espi_rank to verify whether the array
>    element ext_shared_irqs exists. The previous version, is_espi_rank,
>    only checked if the rank index was less than the maximum possible eSPI
>    rank index, but this could potentially result in accessing a
>    non-existing array element. To address this, is_valid_espi_rank was
>    introduced, which ensures that the required eSPI rank exists
> - move gic_number_espis to
>    xen/arm: gicv3: implement handling of GICv3.1 eSPI
> - update vgic_is_valid_irq checks to allow operating with eSPIs
> - remove redundant newline in vgic_allocate_virq
> ---
>   xen/arch/arm/include/asm/vgic.h |  20 +++
>   xen/arch/arm/vgic.c             | 213 +++++++++++++++++++++++++++++++-
>   2 files changed, 230 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
> index 3e7cbbb196..fb4cea73eb 100644
> --- a/xen/arch/arm/include/asm/vgic.h
> +++ b/xen/arch/arm/include/asm/vgic.h
> @@ -146,6 +146,12 @@ struct vgic_dist {
>       int nr_spis; /* Number of SPIs */
>       unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
>       struct vgic_irq_rank *shared_irqs;
> +#ifdef CONFIG_GICV3_ESPI
> +    struct vgic_irq_rank *ext_shared_irqs;
> +    int nr_espis; /* Number of extended SPIs */

Can nr_espis be negative?

> +    /* To simplify determining whether a domain is able to operate with eSPI */
> +    bool has_espi;

I agree with the Volodymyr's comment provided in separate letter that we 
could avoid adding an extra var.

> +#endif
>       /*
>        * SPIs are domain global, SGIs and PPIs are per-VCPU and stored in
>        * struct arch_vcpu.
> @@ -243,6 +249,14 @@ struct vgic_ops {
>   /* Number of ranks of interrupt registers for a domain */
>   #define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_spis+31)/32)
>   
> +#ifdef CONFIG_GICV3_ESPI
> +#define DOMAIN_NR_EXT_RANKS(d) (((d)->arch.vgic.nr_espis+31)/32)
> +#define EXT_RANK_MIN (ESPI_BASE_INTID/32)
> +#define EXT_RANK_MAX ((ESPI_MAX_INTID+31)/32)
> +#define EXT_RANK_NUM2IDX(num) ((num)-EXT_RANK_MIN)
> +#define EXT_RANK_IDX2NUM(idx) ((idx)+EXT_RANK_MIN)
> +#endif
> +
>   #define vgic_lock(v)   spin_lock_irq(&(v)->domain->arch.vgic.lock)
>   #define vgic_unlock(v) spin_unlock_irq(&(v)->domain->arch.vgic.lock)
>   
> @@ -302,6 +316,12 @@ extern struct vgic_irq_rank *vgic_rank_offset(struct vcpu *v,
>                                                 unsigned int b,
>                                                 unsigned int n,
>                                                 unsigned int s);
> +#ifdef CONFIG_GICV3_ESPI
> +extern struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v,
> +                                                  unsigned int b,
> +                                                  unsigned int n,
> +                                                  unsigned int s);
> +#endif
>   extern struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq);
>   extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned int n);
>   extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, unsigned int n);
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 2bbf4d99aa..f4b80cb05f 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -27,9 +27,82 @@
>   
>   bool vgic_is_valid_line(struct domain *d, unsigned int virq)
>   {
> +#ifdef CONFIG_GICV3_ESPI
> +    if ( virq >= ESPI_BASE_INTID &&
> +         virq < ESPI_IDX2INTID(d->arch.vgic.nr_espis) )
> +        return true;
> +#endif
> +
>       return virq < vgic_num_irqs(d);
>   }
>   
> +#ifdef CONFIG_GICV3_ESPI
> +/*
> + * Since eSPI indexes start from 4096 and numbers from 1024 to
> + * 4095 are forbidden, we need to check both lower and upper
> + * limits for ranks.
> + */
> +static inline bool is_valid_espi_rank(struct domain *d, unsigned int rank)
> +{
> +    return rank >= EXT_RANK_MIN &&
> +           EXT_RANK_NUM2IDX(rank) < DOMAIN_NR_EXT_RANKS(d);
> +}
> +
> +static inline struct vgic_irq_rank *vgic_get_espi_rank(struct vcpu *v,
> +                                                       unsigned int rank)
> +{
> +    return &v->domain->arch.vgic.ext_shared_irqs[EXT_RANK_NUM2IDX(rank)];
> +}
> +
> +static inline bool vgic_reserve_espi_virq(struct domain *d, unsigned int virq)
> +{
> +    return !test_and_set_bit(ESPI_INTID2IDX(virq) + vgic_num_irqs(d),
> +                             d->arch.vgic.allocated_irqs);
> +}
> +
> +static void arch_move_espis(struct vcpu *v)
> +{
> +    const cpumask_t *cpu_mask = cpumask_of(v->processor);
> +    struct domain *d = v->domain;
> +    struct pending_irq *p;
> +    struct vcpu *v_target;
> +    unsigned int i;
> +
> +    for ( i = ESPI_BASE_INTID;
> +          i < EXT_RANK_IDX2NUM(d->arch.vgic.nr_espis); i++ )
> +    {
> +        v_target = vgic_get_target_vcpu(v, i);
> +        p = irq_to_pending(v_target, i);
> +
> +        if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) )
> +            irq_set_affinity(p->desc, cpu_mask);
> +    }
> +}
> +#else
> +static inline bool is_valid_espi_rank(struct domain *d, unsigned int rank)
> +{
> +    return false;
> +}
> +
> +/*
> + * This function is stub and will not be called if CONFIG_GICV3_ESPI=n,
> + * because in this case, is_valid_espi_rank will always return false.
> + */
> +static inline struct vgic_irq_rank *vgic_get_espi_rank(struct vcpu *v,
> +                                                       unsigned int rank)
> +{
> +    ASSERT_UNREACHABLE();
> +    return NULL;
> +}
> +
> +static inline bool vgic_reserve_espi_virq(struct domain *d, unsigned int virq)
> +{
> +    return false;
> +}
> +
> +static void arch_move_espis(struct vcpu *v) { }
> +#endif
> +
>   static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
>                                                     unsigned int rank)
>   {
> @@ -37,6 +110,8 @@ static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
>           return v->arch.vgic.private_irqs;
>       else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
>           return &v->domain->arch.vgic.shared_irqs[rank - 1];
> +    else if ( is_valid_espi_rank(v->domain, rank) )
> +        return vgic_get_espi_rank(v, rank);
>       else
>           return NULL;
>   }
> @@ -117,6 +192,74 @@ int domain_vgic_register(struct domain *d, unsigned int *mmio_count)
>       return 0;
>   }
>   
> +#ifdef CONFIG_GICV3_ESPI
> +/*
> + * The function behavior is the same as for regular SPIs (vgic_rank_offset),
> + * but it operates with extended SPI ranks.
> + */
> +struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v, unsigned int b,
> +                                           unsigned int n, unsigned int s)
> +{
> +    unsigned int rank = REG_RANK_NR(b, (n >> s));
> +
> +    return vgic_get_rank(v, rank + EXT_RANK_MIN);
> +}

So the difference between vgic_rank_offset() and vgic_ext_rank_offset() 
in using EXT_RANK_MIN in the latter. I am wondering, whether it makes 
sense to reuse the existing vgic_rank_offset() by moving #ifdef into it? 
Maybe it would be possible to optimize some code added in "[PATCH v4 
10/12] xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers".

e.g. __vgic_v3_distr_common_mmio_read:

Code for the following cases looks very similar to the regular counterparts.
case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):

The difference is in the said helper and the base calculated as
"reg - GICD_XXX", where GICD_XXX is the start of the range for the 
eSPI-based reg (e.g. GICD_IPRIORITYRnE) or regular SPI-based reg 
(e.g.GICD_IPRIORITYR). The GICD_XXX could probably be chosen
based on the "reg" value itself, for example:

rank = vgic_rank_offset(v, 8, reg >= GICD_IPRIORITYRnE ? reg - 
GICD_IPRIORITYRnE : reg - GICD_IPRIORITYR, DABT_WORD);

Please note, I am not requesting for any updates regarding to that, I am 
just thinking out loud.


> +
> +static unsigned int vgic_num_spi_lines(struct domain *d)
> +{
> +    return d->arch.vgic.nr_spis + d->arch.vgic.nr_espis;
> +}
> +
> +static int init_vgic_espi(struct domain *d)
> +{
> +    unsigned int i, idx;
> +
> +    if ( d->arch.vgic.nr_espis == 0 )
> +        return 0;
> +
> +    d->arch.vgic.ext_shared_irqs =
> +        xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_EXT_RANKS(d));
> +    if ( d->arch.vgic.ext_shared_irqs == NULL )
> +        return -ENOMEM;
> +
> +    for ( i = d->arch.vgic.nr_spis, idx = 0;
> +          i < vgic_num_spi_lines(d); i++, idx++ )
> +        vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i],
> +                              ESPI_IDX2INTID(idx));
> +
> +    for ( i = 0; i < DOMAIN_NR_EXT_RANKS(d); i++ )
> +        vgic_rank_init(&d->arch.vgic.ext_shared_irqs[i], i, 0);
> +
> +    return 0;
> +}
> +
> +struct pending_irq *espi_to_pending(struct domain *d, unsigned int irq)
> +{
> +    irq = ESPI_INTID2IDX(irq) + d->arch.vgic.nr_spis;
> +    return &d->arch.vgic.pending_irqs[irq];
> +}
> +#else
> +static unsigned int init_vgic_espi(struct domain *d
> +{
> +    return 0;
> +}
> +
> +static unsigned int vgic_num_spi_lines(struct domain *d)
> +{
> +    return d->arch.vgic.nr_spis;
> +}
> +
> +struct pending_irq *espi_to_pending(struct domain *d, unsigned int irq)
> +{
> +    return NULL;
> +}
> +#endif
> +
> +static unsigned int vgic_num_alloc_irqs(struct domain *d)
> +{
> +    return vgic_num_spi_lines(d) + NR_LOCAL_IRQS;
> +}
> +
>   int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>   {
>       int i;
> @@ -131,6 +274,38 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>        */
>       nr_spis = ROUNDUP(nr_spis, 32);
>   
> +#ifdef CONFIG_GICV3_ESPI
> +    /*
> +     * During domain creation, the dom0less DomUs code or toolstack specifies
> +     * the maximum INTID, which is defined in the domain config subtracted by
> +     * 32 to cover the local IRQs (please see the comment to VGIC_DEF_NR_SPIS).
> +     * To compute the actual number of eSPI that will be usable for,
> +     * add back 32.
> +     */
> +    if ( nr_spis + 32 > ESPI_IDX2INTID(NR_ESPI_IRQS) )
> +        return -EINVAL;
> +
> +    if ( nr_spis + 32 >= ESPI_BASE_INTID )
> +    {
> +        d->arch.vgic.nr_espis = min(nr_spis - ESPI_BASE_INTID + 32, 1024U);
> +        /* Verify if GIC HW can handle provided INTID */
> +        if ( d->arch.vgic.nr_espis > gic_number_espis() )
> +            return -EINVAL;
> +        /*
> +         * Set the maximum available number for regular
> +         * SPI to pass the next check
> +         */
> +        nr_spis = VGIC_DEF_NR_SPIS;
> +        d->arch.vgic.has_espi = true;
> +    }
> +    else
> +    {
> +        /* Domain will use the regular SPI range */
> +        d->arch.vgic.nr_espis = 0;
> +        d->arch.vgic.has_espi = false;
> +    }
> +#endif
> +
>       /* Limit the number of virtual SPIs supported to (1020 - 32) = 988  */
>       if ( nr_spis > (1020 - NR_LOCAL_IRQS) )
>           return -EINVAL;
> @@ -145,7 +320,7 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>           return -ENOMEM;
>   
>       d->arch.vgic.pending_irqs =
> -        xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis);
> +        xzalloc_array(struct pending_irq, vgic_num_spi_lines(d));
>       if ( d->arch.vgic.pending_irqs == NULL )
>           return -ENOMEM;
>   
> @@ -156,12 +331,16 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>       for ( i = 0; i < DOMAIN_NR_RANKS(d); i++ )
>           vgic_rank_init(&d->arch.vgic.shared_irqs[i], i + 1, 0);
>   
> +    ret = init_vgic_espi(d);
> +    if ( ret )
> +        return ret;
> +
>       ret = d->arch.vgic.handler->domain_init(d);
>       if ( ret )
>           return ret;
>   
>       d->arch.vgic.allocated_irqs =
> -        xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d)));
> +        xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_alloc_irqs(d)));
>       if ( !d->arch.vgic.allocated_irqs )
>           return -ENOMEM;
>   
> @@ -195,9 +374,27 @@ void domain_vgic_free(struct domain *d)
>           }
>       }
>   
> +#ifdef CONFIG_GICV3_ESPI
> +    for ( i = 0; i < d->arch.vgic.nr_espis; i++ )
> +    {
> +        struct pending_irq *p = spi_to_pending(d, ESPI_IDX2INTID(i));
> +
> +        if ( p->desc )
> +        {
> +            ret = release_guest_irq(d, p->irq);
> +            if ( ret )
> +                dprintk(XENLOG_G_WARNING, "d%u: Failed to release virq %u ret = %d\n",


NIT: you can use %pd for printing domain with its ID


> +                        d->domain_id, p->irq, ret);
> +        }
> +    }
> +#endif
> +
>       if ( d->arch.vgic.handler )
>           d->arch.vgic.handler->domain_free(d);
>       xfree(d->arch.vgic.shared_irqs);
> +#ifdef CONFIG_GICV3_ESPI
> +    xfree(d->arch.vgic.ext_shared_irqs);
> +#endif
>       xfree(d->arch.vgic.pending_irqs);
>       xfree(d->arch.vgic.allocated_irqs);
>   }
> @@ -331,6 +528,8 @@ void arch_move_irqs(struct vcpu *v)
>           if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) )
>               irq_set_affinity(p->desc, cpu_mask);
>       }
> +
> +    arch_move_espis(v);
>   }
>   
>   void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned int n)
> @@ -538,6 +737,8 @@ struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
>           n = &v->arch.vgic.pending_irqs[irq];
>       else if ( is_lpi(irq) )
>           n = v->domain->arch.vgic.handler->lpi_to_pending(v->domain, irq);
> +    else if ( is_espi(irq) )
> +        n = espi_to_pending(v->domain, irq);
>       else
>           n = &v->domain->arch.vgic.pending_irqs[irq - 32];
>       return n;
> @@ -547,6 +748,9 @@ struct pending_irq *spi_to_pending(struct domain *d, unsigned int irq)
>   {
>       ASSERT(irq >= NR_LOCAL_IRQS);
>   
> +    if ( is_espi(irq) )
> +        return espi_to_pending(d, irq);
> +
>       return &d->arch.vgic.pending_irqs[irq - 32];
>   }
>   
> @@ -668,6 +872,9 @@ bool vgic_reserve_virq(struct domain *d, unsigned int virq)
>       if ( !vgic_is_valid_line(d, virq) )
>           return false;
>   
> +    if ( is_espi(virq) )
> +        return vgic_reserve_espi_virq(d, virq);
> +
>       return !test_and_set_bit(virq, d->arch.vgic.allocated_irqs);
>   }
>   
> @@ -685,7 +892,7 @@ int vgic_allocate_virq(struct domain *d, bool spi)
>       else
>       {
>           first = 32;
> -        end = vgic_num_irqs(d);
> +        end = vgic_num_alloc_irqs(d);
>       }
>   
>       /*

Re: [PATCH v4 04/12] xen/arm/irq: add handling for IRQs in the eSPI range

[Oleksandr Tyshchenko]

On 27.08.25 21:24, Leonid Komarianskyi wrote:


Hello Leonid

> Currently, Xen does not support eSPI interrupts, leading
> to a data abort when such interrupts are defined in the DTS.
> 
> This patch introduces a separate array to initialize up to
> 1024 interrupt descriptors in the eSPI range and adds the
> necessary defines and helper function. These changes lay the
> groundwork for future implementation of full eSPI interrupt
> support. As this GICv3.1 feature is not required by all vendors,
> all changes are guarded by ifdefs, depending on the corresponding
> Kconfig option.
> 
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
> Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
> 
> ---
> Changes in V4:
> - removed redundant line with 'default n' in Kconfig, as it is disabled
>    by default, without explicit specification
> - added reviewed-by from Volodymyr Babchuk


only NITs below

Reviewed-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

> 
> Changes in V3:
> - introduced a new define NR_ESPI_IRQS to avoid confusion, like in the
>    case of using NR_IRQS for espi_desc array
> - implemented helper functions espi_to_desc and init_espi_data to make
>    it possible to add stubs with the same name, and as a result, reduce
>    the number of #ifdefs
> - disable CONFIG_GICV3_ESPI default value to n
> 
> Changes in V2:
> - use (ESPI_MAX_INTID + 1) instead of (ESPI_BASE_INTID + NR_IRQS)
> - remove unnecessary comment for nr_irqs initialization
> ---
>   xen/arch/arm/Kconfig           |  8 ++++++
>   xen/arch/arm/include/asm/irq.h | 26 +++++++++++++++++
>   xen/arch/arm/irq.c             | 52 +++++++++++++++++++++++++++++++++-
>   3 files changed, 85 insertions(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> index 17df147b25..43b05533b1 100644
> --- a/xen/arch/arm/Kconfig
> +++ b/xen/arch/arm/Kconfig
> @@ -135,6 +135,14 @@ config GICV3
>   	  Driver for the ARM Generic Interrupt Controller v3.
>   	  If unsure, use the default setting.
>   
> +config GICV3_ESPI
> +	bool "Extended SPI range support"
> +	depends on GICV3 && !NEW_VGIC
> +	help
> +	  Allow Xen and domains to use interrupt numbers from the extended SPI
> +	  range, from 4096 to 5119. This feature is introduced in GICv3.1
> +	  architecture.
> +
>   config HAS_ITS
>           bool "GICv3 ITS MSI controller support (UNSUPPORTED)" if UNSUPPORTED
>           depends on GICV3 && !NEW_VGIC && !ARM_32
> diff --git a/xen/arch/arm/include/asm/irq.h b/xen/arch/arm/include/asm/irq.h
> index 5bc6475eb4..221dbf23a2 100644
> --- a/xen/arch/arm/include/asm/irq.h
> +++ b/xen/arch/arm/include/asm/irq.h
> @@ -32,6 +32,15 @@ struct arch_irq_desc {
>   #define SPI_MAX_INTID   1019
>   #define LPI_OFFSET      8192
>   
> +#ifdef CONFIG_GICV3_ESPI
> +#define ESPI_BASE_INTID 4096
> +#define ESPI_MAX_INTID  5119
> +#define NR_ESPI_IRQS    1024
> +
> +#define ESPI_INTID2IDX(intid) ((intid) - ESPI_BASE_INTID)
> +#define ESPI_IDX2INTID(idx)   ((idx) + ESPI_BASE_INTID)
> +#endif
> +
>   /* LPIs are always numbered starting at 8192, so 0 is a good invalid case. */
>   #define INVALID_LPI     0
>   
> @@ -39,7 +48,15 @@ struct arch_irq_desc {
>   #define INVALID_IRQ     1023
>   
>   extern const unsigned int nr_irqs;
> +#ifdef CONFIG_GICV3_ESPI
> +/*
> + * This will also cover the eSPI range, as some critical devices
> + * for booting Xen (e.g., serial) may use this type of interrupts.
> + */
> +#define nr_static_irqs (ESPI_MAX_INTID + 1)
> +#else
>   #define nr_static_irqs NR_IRQS
> +#endif
>   
>   struct irq_desc;
>   struct irqaction;
> @@ -55,6 +72,15 @@ static inline bool is_lpi(unsigned int irq)
>       return irq >= LPI_OFFSET;
>   }
>   
> +static inline bool is_espi(unsigned int irq)
> +{
> +#ifdef CONFIG_GICV3_ESPI
> +    return (irq >= ESPI_BASE_INTID && irq <= ESPI_MAX_INTID);
> +#else
> +    return false;
> +#endif
> +}
> +
>   #define domain_pirq_to_irq(d, pirq) (pirq)
>   
>   bool is_assignable_irq(unsigned int irq);
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index b8eccfc924..adb5e49ea3 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -19,7 +19,11 @@
>   #include <asm/gic.h>
>   #include <asm/vgic.h>
>   
> +#ifdef CONFIG_GICV3_ESPI
> +const unsigned int nr_irqs = ESPI_MAX_INTID + 1;
> +#else
>   const unsigned int nr_irqs = NR_IRQS;
> +#endif
>   
>   static unsigned int local_irqs_type[NR_LOCAL_IRQS];
>   static DEFINE_SPINLOCK(local_irqs_type_lock);
> @@ -46,6 +50,49 @@ void irq_end_none(struct irq_desc *irq)
>   }
>   
>   static irq_desc_t irq_desc[NR_IRQS - NR_LOCAL_IRQS];
> +#ifdef CONFIG_GICV3_ESPI
> +static irq_desc_t espi_desc[NR_ESPI_IRQS];

NIT: I would write "TODO: Consider allocating an array dynamically if
there is a need to enable GICV3_ESPI by default". But it is up to you.

> +
> +static struct irq_desc *espi_to_desc(unsigned int irq)
> +{
> +    return &espi_desc[ESPI_INTID2IDX(irq)];
> +}
> +
> +static int __init init_espi_data(void)
> +{
> +    int irq;

NIT: please use unsigned int


> +
> +    for ( irq = ESPI_BASE_INTID; irq <= ESPI_MAX_INTID; irq++ )
> +    {
> +        struct irq_desc *desc = irq_to_desc(irq);
> +        int rc = init_one_irq_desc(desc);
> +
> +        if ( rc )
> +            return rc;
> +
> +        desc->irq = irq;
> +        desc->action  = NULL;
> +    }
> +
> +    return 0;
> +}

[snip]

Re: [PATCH v4 08/12] xen/arm: vgic: add resource management for extended SPIs

[Leonid Komarianskyi]

Hello Oleksandr,

Thank you for your comment and suggestions.

On 28.08.25 20:34, Oleksandr Tyshchenko wrote:
> 
> 
> On 27.08.25 21:24, Leonid Komarianskyi wrote:
> 
> Hello Leonid
> 
>> This change introduces resource management in the VGIC to handle
>> extended SPIs introduced in GICv3.1. The pending_irqs and
>> allocated_irqs arrays are resized to support the required
>> number of eSPIs, based on what is supported by the hardware and
>> requested by the guest. A new field, ext_shared_irqs, is added
>> to the VGIC structure to store information about eSPIs, similar
>> to how shared_irqs is used for regular SPIs.
>>
>> Since the eSPI range starts at INTID 4096 and INTIDs between 1025
>> and 4095 are reserved, helper macros are introduced to simplify the
>> transformation of indices and to enable easier access to eSPI-specific
>> resources. These changes prepare the VGIC for processing eSPIs as
>> required by future functionality.
>>
>> The initialization and deinitialization paths for vgic have been updated
>> to allocate and free these resources appropriately. Additionally,
>> updated handling of INTIDs greater than 1024, passed from the toolstack
>> during domain creation, and verification logic ensures only valid SPI or
>> eSPI INTIDs are used.
>>
>> The existing SPI behavior remains unaffected when guests do not request
>> eSPIs, GIC hardware does not support them, or the CONFIG_GICV3_ESPI
>> option is disabled.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V4:
>> - added has_espi field to simplify determining whether a domain is able
>>    to operate with eSPI
>> - fixed formatting issues and misspellings
>>
>> Changes in V3:
>> - fixed formatting for lines with more than 80 symbols
>> - introduced helper functions to be able to use stubs in case of
>>    CONFIG_GICV3_ESPI disabled, and as a result, reduce the number of
>>    #ifdefs
>> - fixed checks for nr_spis in domain_vgic_init
>> - updated comment about nr_spis adjustments with dom0less mention
>> - moved comment with additional explanations before checks
>> - used unsigned int for indexes since they cannot be negative
>> - removed unnecessary parentheses
>> - move vgic_ext_rank_offset to the below ifdef guard, to reduce the
>>    number of ifdefs
>>
>> Changes in V2:
>> - change is_espi_rank to is_valid_espi_rank to verify whether the array
>>    element ext_shared_irqs exists. The previous version, is_espi_rank,
>>    only checked if the rank index was less than the maximum possible eSPI
>>    rank index, but this could potentially result in accessing a
>>    non-existing array element. To address this, is_valid_espi_rank was
>>    introduced, which ensures that the required eSPI rank exists
>> - move gic_number_espis to
>>    xen/arm: gicv3: implement handling of GICv3.1 eSPI
>> - update vgic_is_valid_irq checks to allow operating with eSPIs
>> - remove redundant newline in vgic_allocate_virq
>> ---
>>   xen/arch/arm/include/asm/vgic.h |  20 +++
>>   xen/arch/arm/vgic.c             | 213 +++++++++++++++++++++++++++++++-
>>   2 files changed, 230 insertions(+), 3 deletions(-)
>>
>> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/ 
>> asm/vgic.h
>> index 3e7cbbb196..fb4cea73eb 100644
>> --- a/xen/arch/arm/include/asm/vgic.h
>> +++ b/xen/arch/arm/include/asm/vgic.h
>> @@ -146,6 +146,12 @@ struct vgic_dist {
>>       int nr_spis; /* Number of SPIs */
>>       unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
>>       struct vgic_irq_rank *shared_irqs;
>> +#ifdef CONFIG_GICV3_ESPI
>> +    struct vgic_irq_rank *ext_shared_irqs;
>> +    int nr_espis; /* Number of extended SPIs */
> 
> Can nr_espis be negative?
> 

No, it cannot, so I will change it to unsigned int in V5.

>> +    /* To simplify determining whether a domain is able to operate 
>> with eSPI */
>> +    bool has_espi;
> 
> I agree with the Volodymyr's comment provided in separate letter that we 
> could avoid adding an extra var.
> 

Sure, as I mentioned to Volodymyr, I will remove it in V5.

>> +#endif
>>       /*
>>        * SPIs are domain global, SGIs and PPIs are per-VCPU and stored in
>>        * struct arch_vcpu.
>> @@ -243,6 +249,14 @@ struct vgic_ops {
>>   /* Number of ranks of interrupt registers for a domain */
>>   #define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_spis+31)/32)
>> +#ifdef CONFIG_GICV3_ESPI
>> +#define DOMAIN_NR_EXT_RANKS(d) (((d)->arch.vgic.nr_espis+31)/32)
>> +#define EXT_RANK_MIN (ESPI_BASE_INTID/32)
>> +#define EXT_RANK_MAX ((ESPI_MAX_INTID+31)/32)
>> +#define EXT_RANK_NUM2IDX(num) ((num)-EXT_RANK_MIN)
>> +#define EXT_RANK_IDX2NUM(idx) ((idx)+EXT_RANK_MIN)
>> +#endif
>> +
>>   #define vgic_lock(v)   spin_lock_irq(&(v)->domain->arch.vgic.lock)
>>   #define vgic_unlock(v) spin_unlock_irq(&(v)->domain->arch.vgic.lock)
>> @@ -302,6 +316,12 @@ extern struct vgic_irq_rank 
>> *vgic_rank_offset(struct vcpu *v,
>>                                                 unsigned int b,
>>                                                 unsigned int n,
>>                                                 unsigned int s);
>> +#ifdef CONFIG_GICV3_ESPI
>> +extern struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v,
>> +                                                  unsigned int b,
>> +                                                  unsigned int n,
>> +                                                  unsigned int s);
>> +#endif
>>   extern struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned 
>> int irq);
>>   extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned 
>> int n);
>>   extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, unsigned 
>> int n);
>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>> index 2bbf4d99aa..f4b80cb05f 100644
>> --- a/xen/arch/arm/vgic.c
>> +++ b/xen/arch/arm/vgic.c
>> @@ -27,9 +27,82 @@
>>   bool vgic_is_valid_line(struct domain *d, unsigned int virq)
>>   {
>> +#ifdef CONFIG_GICV3_ESPI
>> +    if ( virq >= ESPI_BASE_INTID &&
>> +         virq < ESPI_IDX2INTID(d->arch.vgic.nr_espis) )
>> +        return true;
>> +#endif
>> +
>>       return virq < vgic_num_irqs(d);
>>   }
>> +#ifdef CONFIG_GICV3_ESPI
>> +/*
>> + * Since eSPI indexes start from 4096 and numbers from 1024 to
>> + * 4095 are forbidden, we need to check both lower and upper
>> + * limits for ranks.
>> + */
>> +static inline bool is_valid_espi_rank(struct domain *d, unsigned int 
>> rank)
>> +{
>> +    return rank >= EXT_RANK_MIN &&
>> +           EXT_RANK_NUM2IDX(rank) < DOMAIN_NR_EXT_RANKS(d);
>> +}
>> +
>> +static inline struct vgic_irq_rank *vgic_get_espi_rank(struct vcpu *v,
>> +                                                       unsigned int 
>> rank)
>> +{
>> +    return &v->domain- 
>> >arch.vgic.ext_shared_irqs[EXT_RANK_NUM2IDX(rank)];
>> +}
>> +
>> +static inline bool vgic_reserve_espi_virq(struct domain *d, unsigned 
>> int virq)
>> +{
>> +    return !test_and_set_bit(ESPI_INTID2IDX(virq) + vgic_num_irqs(d),
>> +                             d->arch.vgic.allocated_irqs);
>> +}
>> +
>> +static void arch_move_espis(struct vcpu *v)
>> +{
>> +    const cpumask_t *cpu_mask = cpumask_of(v->processor);
>> +    struct domain *d = v->domain;
>> +    struct pending_irq *p;
>> +    struct vcpu *v_target;
>> +    unsigned int i;
>> +
>> +    for ( i = ESPI_BASE_INTID;
>> +          i < EXT_RANK_IDX2NUM(d->arch.vgic.nr_espis); i++ )
>> +    {
>> +        v_target = vgic_get_target_vcpu(v, i);
>> +        p = irq_to_pending(v_target, i);
>> +
>> +        if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p- 
>> >status) )
>> +            irq_set_affinity(p->desc, cpu_mask);
>> +    }
>> +}
>> +#else
>> +static inline bool is_valid_espi_rank(struct domain *d, unsigned int 
>> rank)
>> +{
>> +    return false;
>> +}
>> +
>> +/*
>> + * This function is stub and will not be called if CONFIG_GICV3_ESPI=n,
>> + * because in this case, is_valid_espi_rank will always return false.
>> + */
>> +static inline struct vgic_irq_rank *vgic_get_espi_rank(struct vcpu *v,
>> +                                                       unsigned int 
>> rank)
>> +{
>> +    ASSERT_UNREACHABLE();
>> +    return NULL;
>> +}
>> +
>> +static inline bool vgic_reserve_espi_virq(struct domain *d, unsigned 
>> int virq)
>> +{
>> +    return false;
>> +}
>> +
>> +static void arch_move_espis(struct vcpu *v) { }
>> +#endif
>> +
>>   static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
>>                                                     unsigned int rank)
>>   {
>> @@ -37,6 +110,8 @@ static inline struct vgic_irq_rank 
>> *vgic_get_rank(struct vcpu *v,
>>           return v->arch.vgic.private_irqs;
>>       else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
>>           return &v->domain->arch.vgic.shared_irqs[rank - 1];
>> +    else if ( is_valid_espi_rank(v->domain, rank) )
>> +        return vgic_get_espi_rank(v, rank);
>>       else
>>           return NULL;
>>   }
>> @@ -117,6 +192,74 @@ int domain_vgic_register(struct domain *d, 
>> unsigned int *mmio_count)
>>       return 0;
>>   }
>> +#ifdef CONFIG_GICV3_ESPI
>> +/*
>> + * The function behavior is the same as for regular SPIs 
>> (vgic_rank_offset),
>> + * but it operates with extended SPI ranks.
>> + */
>> +struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v, unsigned 
>> int b,
>> +                                           unsigned int n, unsigned 
>> int s)
>> +{
>> +    unsigned int rank = REG_RANK_NR(b, (n >> s));
>> +
>> +    return vgic_get_rank(v, rank + EXT_RANK_MIN);
>> +}
> 
> So the difference between vgic_rank_offset() and vgic_ext_rank_offset() 
> in using EXT_RANK_MIN in the latter. I am wondering, whether it makes 
> sense to reuse the existing vgic_rank_offset() by moving #ifdef into it? 
> Maybe it would be possible to optimize some code added in "[PATCH v4 
> 10/12] xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers".
> 
> e.g. __vgic_v3_distr_common_mmio_read:
> 
> Code for the following cases looks very similar to the regular 
> counterparts.
> case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
> case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
> case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
> case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
> case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
> 
> The difference is in the said helper and the base calculated as
> "reg - GICD_XXX", where GICD_XXX is the start of the range for the eSPI- 
> based reg (e.g. GICD_IPRIORITYRnE) or regular SPI-based reg 
> (e.g.GICD_IPRIORITYR). The GICD_XXX could probably be chosen
> based on the "reg" value itself, for example:
> 
> rank = vgic_rank_offset(v, 8, reg >= GICD_IPRIORITYRnE ? reg - 
> GICD_IPRIORITYRnE : reg - GICD_IPRIORITYR, DABT_WORD);
> 
> Please note, I am not requesting for any updates regarding to that, I am 
> just thinking out loud.
> 

First of all, I have to say that I really tried to reuse the existing 
code for regular SPIs whenever possible, but I could not come up with 
something better than introducing a new function for eSPI ranks in this 
case. This is due to several reasons:
1) Currently, all eSPI-specific offsets are placed under #ifdef to avoid 
or at least reduce the introduction of dead code or defines that will 
not be used. This was requested by our teammates who are engaged in 
MISRA fixes. Therefore, this is the first reason why eSPI-specific 
elements are placed under the appropriate configuration. Without 
CONFIG_GICV3_ESPI, we cannot use these offsets.

2) The proposed solution:
 > rank = vgic_rank_offset(v, 8, reg >= GICD_IPRIORITYRnE ? reg -
 > GICD_IPRIORITYRnE : reg - GICD_IPRIORITYR, DABT_WORD);

will not work, unfortunately, because in this case, vgic_rank_offset 
will always return ranks for regular SPIs.

Long story short: we will encounter issues determining whether to use 
shared_irqs (for regular SPIs) or ext_shared_irqs (for eSPIs).

And here are the details about why this will not work:

In the case of receiving an offset for a regular SPI, e.g., 0x6200 
(GICD_IROUTER range [0x6100-0x7FD8]), we will get rank = 2 for regular SPIs:
vgic_rank_offset(v, 64, 0x6200-0x6000, DABT_DOUBLE_WORD) ->
DABT_DOUBLE_WORD = 3

Thus:
rank = REG_RANK_NR(64, (0x200 >> 3) ->
REG_RANK_NR will return in this case:
0x200 >> 3 >> 5 = 2

On the other hand, if we receive an offset for an eSPI, e.g., 0x8200 
(GICD_IROUTERnE range [0x8000-0x9FFC]), we will get the same rank = 2 
when using the proposed solution:
vgic_rank_offset(v, 64, 0x8200-0x8000, DABT_DOUBLE_WORD) ->
DABT_DOUBLE_WORD = 3

rank = REG_RANK_NR(64, (0x200 >> 3) ->
REG_RANK_NR will return also:
0x200 >> 3 >> 5 = 2

As a result, in both cases, vgic_get_rank(v, 2) will return the rank for 
regular SPIs:

/* This condition will apply because rank will be less than 32 */
else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
     return &v->domain->arch.vgic.shared_irqs[rank - 1];

Therefore, I decided to create a separate function to properly transform 
ranks into the correct range.

And, unfortunately, we cannot resolve the second issue with #ifdefs 
inside vgic_rank_offset, because this function will receive the same 
parameters for SPIs/eSPIs and we would not be able to determine whether 
we should return shared_irqs or ext_shared_irqs.

> 
>> +
>> +static unsigned int vgic_num_spi_lines(struct domain *d)
>> +{
>> +    return d->arch.vgic.nr_spis + d->arch.vgic.nr_espis;
>> +}
>> +
>> +static int init_vgic_espi(struct domain *d)
>> +{
>> +    unsigned int i, idx;
>> +
>> +    if ( d->arch.vgic.nr_espis == 0 )
>> +        return 0;
>> +
>> +    d->arch.vgic.ext_shared_irqs =
>> +        xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_EXT_RANKS(d));
>> +    if ( d->arch.vgic.ext_shared_irqs == NULL )
>> +        return -ENOMEM;
>> +
>> +    for ( i = d->arch.vgic.nr_spis, idx = 0;
>> +          i < vgic_num_spi_lines(d); i++, idx++ )
>> +        vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i],
>> +                              ESPI_IDX2INTID(idx));
>> +
>> +    for ( i = 0; i < DOMAIN_NR_EXT_RANKS(d); i++ )
>> +        vgic_rank_init(&d->arch.vgic.ext_shared_irqs[i], i, 0);
>> +
>> +    return 0;
>> +}
>> +
>> +struct pending_irq *espi_to_pending(struct domain *d, unsigned int irq)
>> +{
>> +    irq = ESPI_INTID2IDX(irq) + d->arch.vgic.nr_spis;
>> +    return &d->arch.vgic.pending_irqs[irq];
>> +}
>> +#else
>> +static unsigned int init_vgic_espi(struct domain *d
>> +{
>> +    return 0;
>> +}
>> +
>> +static unsigned int vgic_num_spi_lines(struct domain *d)
>> +{
>> +    return d->arch.vgic.nr_spis;
>> +}
>> +
>> +struct pending_irq *espi_to_pending(struct domain *d, unsigned int irq)
>> +{
>> +    return NULL;
>> +}
>> +#endif
>> +
>> +static unsigned int vgic_num_alloc_irqs(struct domain *d)
>> +{
>> +    return vgic_num_spi_lines(d) + NR_LOCAL_IRQS;
>> +}
>> +
>>   int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>>   {
>>       int i;
>> @@ -131,6 +274,38 @@ int domain_vgic_init(struct domain *d, unsigned 
>> int nr_spis)
>>        */
>>       nr_spis = ROUNDUP(nr_spis, 32);
>> +#ifdef CONFIG_GICV3_ESPI
>> +    /*
>> +     * During domain creation, the dom0less DomUs code or toolstack 
>> specifies
>> +     * the maximum INTID, which is defined in the domain config 
>> subtracted by
>> +     * 32 to cover the local IRQs (please see the comment to 
>> VGIC_DEF_NR_SPIS).
>> +     * To compute the actual number of eSPI that will be usable for,
>> +     * add back 32.
>> +     */
>> +    if ( nr_spis + 32 > ESPI_IDX2INTID(NR_ESPI_IRQS) )
>> +        return -EINVAL;
>> +
>> +    if ( nr_spis + 32 >= ESPI_BASE_INTID )
>> +    {
>> +        d->arch.vgic.nr_espis = min(nr_spis - ESPI_BASE_INTID + 32, 
>> 1024U);
>> +        /* Verify if GIC HW can handle provided INTID */
>> +        if ( d->arch.vgic.nr_espis > gic_number_espis() )
>> +            return -EINVAL;
>> +        /*
>> +         * Set the maximum available number for regular
>> +         * SPI to pass the next check
>> +         */
>> +        nr_spis = VGIC_DEF_NR_SPIS;
>> +        d->arch.vgic.has_espi = true;
>> +    }
>> +    else
>> +    {
>> +        /* Domain will use the regular SPI range */
>> +        d->arch.vgic.nr_espis = 0;
>> +        d->arch.vgic.has_espi = false;
>> +    }
>> +#endif
>> +
>>       /* Limit the number of virtual SPIs supported to (1020 - 32) = 
>> 988  */
>>       if ( nr_spis > (1020 - NR_LOCAL_IRQS) )
>>           return -EINVAL;
>> @@ -145,7 +320,7 @@ int domain_vgic_init(struct domain *d, unsigned 
>> int nr_spis)
>>           return -ENOMEM;
>>       d->arch.vgic.pending_irqs =
>> -        xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis);
>> +        xzalloc_array(struct pending_irq, vgic_num_spi_lines(d));
>>       if ( d->arch.vgic.pending_irqs == NULL )
>>           return -ENOMEM;
>> @@ -156,12 +331,16 @@ int domain_vgic_init(struct domain *d, unsigned 
>> int nr_spis)
>>       for ( i = 0; i < DOMAIN_NR_RANKS(d); i++ )
>>           vgic_rank_init(&d->arch.vgic.shared_irqs[i], i + 1, 0);
>> +    ret = init_vgic_espi(d);
>> +    if ( ret )
>> +        return ret;
>> +
>>       ret = d->arch.vgic.handler->domain_init(d);
>>       if ( ret )
>>           return ret;
>>       d->arch.vgic.allocated_irqs =
>> -        xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d)));
>> +        xzalloc_array(unsigned long, 
>> BITS_TO_LONGS(vgic_num_alloc_irqs(d)));
>>       if ( !d->arch.vgic.allocated_irqs )
>>           return -ENOMEM;
>> @@ -195,9 +374,27 @@ void domain_vgic_free(struct domain *d)
>>           }
>>       }
>> +#ifdef CONFIG_GICV3_ESPI
>> +    for ( i = 0; i < d->arch.vgic.nr_espis; i++ )
>> +    {
>> +        struct pending_irq *p = spi_to_pending(d, ESPI_IDX2INTID(i));
>> +
>> +        if ( p->desc )
>> +        {
>> +            ret = release_guest_irq(d, p->irq);
>> +            if ( ret )
>> +                dprintk(XENLOG_G_WARNING, "d%u: Failed to release 
>> virq %u ret = %d\n",
> 
> 
> NIT: you can use %pd for printing domain with its ID
> 
> 

I will fix it in V5.

>> +                        d->domain_id, p->irq, ret);
>> +        }
>> +    }
>> +#endif
>> +
>>       if ( d->arch.vgic.handler )
>>           d->arch.vgic.handler->domain_free(d);
>>       xfree(d->arch.vgic.shared_irqs);
>> +#ifdef CONFIG_GICV3_ESPI
>> +    xfree(d->arch.vgic.ext_shared_irqs);
>> +#endif
>>       xfree(d->arch.vgic.pending_irqs);
>>       xfree(d->arch.vgic.allocated_irqs);
>>   }
>> @@ -331,6 +528,8 @@ void arch_move_irqs(struct vcpu *v)
>>           if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p- 
>> >status) )
>>               irq_set_affinity(p->desc, cpu_mask);
>>       }
>> +
>> +    arch_move_espis(v);
>>   }
>>   void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned int n)
>> @@ -538,6 +737,8 @@ struct pending_irq *irq_to_pending(struct vcpu *v, 
>> unsigned int irq)
>>           n = &v->arch.vgic.pending_irqs[irq];
>>       else if ( is_lpi(irq) )
>>           n = v->domain->arch.vgic.handler->lpi_to_pending(v->domain, 
>> irq);
>> +    else if ( is_espi(irq) )
>> +        n = espi_to_pending(v->domain, irq);
>>       else
>>           n = &v->domain->arch.vgic.pending_irqs[irq - 32];
>>       return n;
>> @@ -547,6 +748,9 @@ struct pending_irq *spi_to_pending(struct domain 
>> *d, unsigned int irq)
>>   {
>>       ASSERT(irq >= NR_LOCAL_IRQS);
>> +    if ( is_espi(irq) )
>> +        return espi_to_pending(d, irq);
>> +
>>       return &d->arch.vgic.pending_irqs[irq - 32];
>>   }
>> @@ -668,6 +872,9 @@ bool vgic_reserve_virq(struct domain *d, unsigned 
>> int virq)
>>       if ( !vgic_is_valid_line(d, virq) )
>>           return false;
>> +    if ( is_espi(virq) )
>> +        return vgic_reserve_espi_virq(d, virq);
>> +
>>       return !test_and_set_bit(virq, d->arch.vgic.allocated_irqs);
>>   }
>> @@ -685,7 +892,7 @@ int vgic_allocate_virq(struct domain *d, bool spi)
>>       else
>>       {
>>           first = 32;
>> -        end = vgic_num_irqs(d);
>> +        end = vgic_num_alloc_irqs(d);
>>       }
>>       /*
> 

Best regards,
Leonid

Re: [PATCH v4 08/12] xen/arm: vgic: add resource management for extended SPIs

[Oleksandr Tyshchenko]

On 28.08.25 22:09, Leonid Komarianskyi wrote:
> Hello Oleksandr,

Hello Leonid


> 
> Thank you for your comment and suggestions.
> 
> On 28.08.25 20:34, Oleksandr Tyshchenko wrote:
>>
>>
>> On 27.08.25 21:24, Leonid Komarianskyi wrote:
>>
>> Hello Leonid
>>
>>> This change introduces resource management in the VGIC to handle
>>> extended SPIs introduced in GICv3.1. The pending_irqs and
>>> allocated_irqs arrays are resized to support the required
>>> number of eSPIs, based on what is supported by the hardware and
>>> requested by the guest. A new field, ext_shared_irqs, is added
>>> to the VGIC structure to store information about eSPIs, similar
>>> to how shared_irqs is used for regular SPIs.
>>>
>>> Since the eSPI range starts at INTID 4096 and INTIDs between 1025
>>> and 4095 are reserved, helper macros are introduced to simplify the
>>> transformation of indices and to enable easier access to eSPI-specific
>>> resources. These changes prepare the VGIC for processing eSPIs as
>>> required by future functionality.
>>>
>>> The initialization and deinitialization paths for vgic have been updated
>>> to allocate and free these resources appropriately. Additionally,
>>> updated handling of INTIDs greater than 1024, passed from the toolstack
>>> during domain creation, and verification logic ensures only valid SPI or
>>> eSPI INTIDs are used.
>>>
>>> The existing SPI behavior remains unaffected when guests do not request
>>> eSPIs, GIC hardware does not support them, or the CONFIG_GICV3_ESPI
>>> option is disabled.
>>>
>>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>>
>>> ---
>>> Changes in V4:
>>> - added has_espi field to simplify determining whether a domain is able
>>>     to operate with eSPI
>>> - fixed formatting issues and misspellings
>>>
>>> Changes in V3:
>>> - fixed formatting for lines with more than 80 symbols
>>> - introduced helper functions to be able to use stubs in case of
>>>     CONFIG_GICV3_ESPI disabled, and as a result, reduce the number of
>>>     #ifdefs
>>> - fixed checks for nr_spis in domain_vgic_init
>>> - updated comment about nr_spis adjustments with dom0less mention
>>> - moved comment with additional explanations before checks
>>> - used unsigned int for indexes since they cannot be negative
>>> - removed unnecessary parentheses
>>> - move vgic_ext_rank_offset to the below ifdef guard, to reduce the
>>>     number of ifdefs
>>>
>>> Changes in V2:
>>> - change is_espi_rank to is_valid_espi_rank to verify whether the array
>>>     element ext_shared_irqs exists. The previous version, is_espi_rank,
>>>     only checked if the rank index was less than the maximum possible eSPI
>>>     rank index, but this could potentially result in accessing a
>>>     non-existing array element. To address this, is_valid_espi_rank was
>>>     introduced, which ensures that the required eSPI rank exists
>>> - move gic_number_espis to
>>>     xen/arm: gicv3: implement handling of GICv3.1 eSPI
>>> - update vgic_is_valid_irq checks to allow operating with eSPIs
>>> - remove redundant newline in vgic_allocate_virq
>>> ---
>>>    xen/arch/arm/include/asm/vgic.h |  20 +++
>>>    xen/arch/arm/vgic.c             | 213 +++++++++++++++++++++++++++++++-
>>>    2 files changed, 230 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/
>>> asm/vgic.h
>>> index 3e7cbbb196..fb4cea73eb 100644
>>> --- a/xen/arch/arm/include/asm/vgic.h
>>> +++ b/xen/arch/arm/include/asm/vgic.h
>>> @@ -146,6 +146,12 @@ struct vgic_dist {
>>>        int nr_spis; /* Number of SPIs */
>>>        unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
>>>        struct vgic_irq_rank *shared_irqs;
>>> +#ifdef CONFIG_GICV3_ESPI
>>> +    struct vgic_irq_rank *ext_shared_irqs;
>>> +    int nr_espis; /* Number of extended SPIs */
>>
>> Can nr_espis be negative?
>>
> 
> No, it cannot, so I will change it to unsigned int in V5.
> 
>>> +    /* To simplify determining whether a domain is able to operate
>>> with eSPI */
>>> +    bool has_espi;
>>
>> I agree with the Volodymyr's comment provided in separate letter that we
>> could avoid adding an extra var.
>>
> 
> Sure, as I mentioned to Volodymyr, I will remove it in V5.
> 
>>> +#endif
>>>        /*
>>>         * SPIs are domain global, SGIs and PPIs are per-VCPU and stored in
>>>         * struct arch_vcpu.
>>> @@ -243,6 +249,14 @@ struct vgic_ops {
>>>    /* Number of ranks of interrupt registers for a domain */
>>>    #define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_spis+31)/32)
>>> +#ifdef CONFIG_GICV3_ESPI
>>> +#define DOMAIN_NR_EXT_RANKS(d) (((d)->arch.vgic.nr_espis+31)/32)
>>> +#define EXT_RANK_MIN (ESPI_BASE_INTID/32)
>>> +#define EXT_RANK_MAX ((ESPI_MAX_INTID+31)/32)
>>> +#define EXT_RANK_NUM2IDX(num) ((num)-EXT_RANK_MIN)
>>> +#define EXT_RANK_IDX2NUM(idx) ((idx)+EXT_RANK_MIN)
>>> +#endif
>>> +
>>>    #define vgic_lock(v)   spin_lock_irq(&(v)->domain->arch.vgic.lock)
>>>    #define vgic_unlock(v) spin_unlock_irq(&(v)->domain->arch.vgic.lock)
>>> @@ -302,6 +316,12 @@ extern struct vgic_irq_rank
>>> *vgic_rank_offset(struct vcpu *v,
>>>                                                  unsigned int b,
>>>                                                  unsigned int n,
>>>                                                  unsigned int s);
>>> +#ifdef CONFIG_GICV3_ESPI
>>> +extern struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v,
>>> +                                                  unsigned int b,
>>> +                                                  unsigned int n,
>>> +                                                  unsigned int s);
>>> +#endif
>>>    extern struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned
>>> int irq);
>>>    extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned
>>> int n);
>>>    extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, unsigned
>>> int n);
>>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>>> index 2bbf4d99aa..f4b80cb05f 100644
>>> --- a/xen/arch/arm/vgic.c
>>> +++ b/xen/arch/arm/vgic.c
>>> @@ -27,9 +27,82 @@
>>>    bool vgic_is_valid_line(struct domain *d, unsigned int virq)
>>>    {
>>> +#ifdef CONFIG_GICV3_ESPI
>>> +    if ( virq >= ESPI_BASE_INTID &&
>>> +         virq < ESPI_IDX2INTID(d->arch.vgic.nr_espis) )
>>> +        return true;
>>> +#endif
>>> +
>>>        return virq < vgic_num_irqs(d);
>>>    }
>>> +#ifdef CONFIG_GICV3_ESPI
>>> +/*
>>> + * Since eSPI indexes start from 4096 and numbers from 1024 to
>>> + * 4095 are forbidden, we need to check both lower and upper
>>> + * limits for ranks.
>>> + */
>>> +static inline bool is_valid_espi_rank(struct domain *d, unsigned int
>>> rank)
>>> +{
>>> +    return rank >= EXT_RANK_MIN &&
>>> +           EXT_RANK_NUM2IDX(rank) < DOMAIN_NR_EXT_RANKS(d);
>>> +}
>>> +
>>> +static inline struct vgic_irq_rank *vgic_get_espi_rank(struct vcpu *v,
>>> +                                                       unsigned int
>>> rank)
>>> +{
>>> +    return &v->domain-
>>>> arch.vgic.ext_shared_irqs[EXT_RANK_NUM2IDX(rank)];
>>> +}
>>> +
>>> +static inline bool vgic_reserve_espi_virq(struct domain *d, unsigned
>>> int virq)
>>> +{
>>> +    return !test_and_set_bit(ESPI_INTID2IDX(virq) + vgic_num_irqs(d),
>>> +                             d->arch.vgic.allocated_irqs);
>>> +}
>>> +
>>> +static void arch_move_espis(struct vcpu *v)
>>> +{
>>> +    const cpumask_t *cpu_mask = cpumask_of(v->processor);
>>> +    struct domain *d = v->domain;
>>> +    struct pending_irq *p;
>>> +    struct vcpu *v_target;
>>> +    unsigned int i;
>>> +
>>> +    for ( i = ESPI_BASE_INTID;
>>> +          i < EXT_RANK_IDX2NUM(d->arch.vgic.nr_espis); i++ )
>>> +    {
>>> +        v_target = vgic_get_target_vcpu(v, i);
>>> +        p = irq_to_pending(v_target, i);
>>> +
>>> +        if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p-
>>>> status) )
>>> +            irq_set_affinity(p->desc, cpu_mask);
>>> +    }
>>> +}
>>> +#else
>>> +static inline bool is_valid_espi_rank(struct domain *d, unsigned int
>>> rank)
>>> +{
>>> +    return false;
>>> +}
>>> +
>>> +/*
>>> + * This function is stub and will not be called if CONFIG_GICV3_ESPI=n,
>>> + * because in this case, is_valid_espi_rank will always return false.
>>> + */
>>> +static inline struct vgic_irq_rank *vgic_get_espi_rank(struct vcpu *v,
>>> +                                                       unsigned int
>>> rank)
>>> +{
>>> +    ASSERT_UNREACHABLE();
>>> +    return NULL;
>>> +}
>>> +
>>> +static inline bool vgic_reserve_espi_virq(struct domain *d, unsigned
>>> int virq)
>>> +{
>>> +    return false;
>>> +}
>>> +
>>> +static void arch_move_espis(struct vcpu *v) { }
>>> +#endif
>>> +
>>>    static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
>>>                                                      unsigned int rank)
>>>    {
>>> @@ -37,6 +110,8 @@ static inline struct vgic_irq_rank
>>> *vgic_get_rank(struct vcpu *v,
>>>            return v->arch.vgic.private_irqs;
>>>        else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
>>>            return &v->domain->arch.vgic.shared_irqs[rank - 1];
>>> +    else if ( is_valid_espi_rank(v->domain, rank) )
>>> +        return vgic_get_espi_rank(v, rank);
>>>        else
>>>            return NULL;
>>>    }
>>> @@ -117,6 +192,74 @@ int domain_vgic_register(struct domain *d,
>>> unsigned int *mmio_count)
>>>        return 0;
>>>    }
>>> +#ifdef CONFIG_GICV3_ESPI
>>> +/*
>>> + * The function behavior is the same as for regular SPIs
>>> (vgic_rank_offset),
>>> + * but it operates with extended SPI ranks.
>>> + */
>>> +struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v, unsigned
>>> int b,
>>> +                                           unsigned int n, unsigned
>>> int s)
>>> +{
>>> +    unsigned int rank = REG_RANK_NR(b, (n >> s));
>>> +
>>> +    return vgic_get_rank(v, rank + EXT_RANK_MIN);
>>> +}
>>
>> So the difference between vgic_rank_offset() and vgic_ext_rank_offset()
>> in using EXT_RANK_MIN in the latter. I am wondering, whether it makes
>> sense to reuse the existing vgic_rank_offset() by moving #ifdef into it?
>> Maybe it would be possible to optimize some code added in "[PATCH v4
>> 10/12] xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers".
>>
>> e.g. __vgic_v3_distr_common_mmio_read:
>>
>> Code for the following cases looks very similar to the regular
>> counterparts.
>> case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
>> case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
>> case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
>> case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
>> case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
>>
>> The difference is in the said helper and the base calculated as
>> "reg - GICD_XXX", where GICD_XXX is the start of the range for the eSPI-
>> based reg (e.g. GICD_IPRIORITYRnE) or regular SPI-based reg
>> (e.g.GICD_IPRIORITYR). The GICD_XXX could probably be chosen
>> based on the "reg" value itself, for example:
>>
>> rank = vgic_rank_offset(v, 8, reg >= GICD_IPRIORITYRnE ? reg -
>> GICD_IPRIORITYRnE : reg - GICD_IPRIORITYR, DABT_WORD);
>>
>> Please note, I am not requesting for any updates regarding to that, I am
>> just thinking out loud.
>>
> 
> First of all, I have to say that I really tried to reuse the existing
> code for regular SPIs whenever possible, but I could not come up with
> something better than introducing a new function for eSPI ranks in this
> case. This is due to several reasons:
> 1) Currently, all eSPI-specific offsets are placed under #ifdef to avoid
> or at least reduce the introduction of dead code or defines that will
> not be used. This was requested by our teammates who are engaged in
> MISRA fixes. Therefore, this is the first reason why eSPI-specific
> elements are placed under the appropriate configuration. Without
> CONFIG_GICV3_ESPI, we cannot use these offsets.
> 
> 2) The proposed solution:
>   > rank = vgic_rank_offset(v, 8, reg >= GICD_IPRIORITYRnE ? reg -
>   > GICD_IPRIORITYRnE : reg - GICD_IPRIORITYR, DABT_WORD);
> 
> will not work, unfortunately, because in this case, vgic_rank_offset
> will always return ranks for regular SPIs.
> 
> Long story short: we will encounter issues determining whether to use
> shared_irqs (for regular SPIs) or ext_shared_irqs (for eSPIs).
> 
> And here are the details about why this will not work:
> 
> In the case of receiving an offset for a regular SPI, e.g., 0x6200
> (GICD_IROUTER range [0x6100-0x7FD8]), we will get rank = 2 for regular SPIs:
> vgic_rank_offset(v, 64, 0x6200-0x6000, DABT_DOUBLE_WORD) ->
> DABT_DOUBLE_WORD = 3
> 
> Thus:
> rank = REG_RANK_NR(64, (0x200 >> 3) ->
> REG_RANK_NR will return in this case:
> 0x200 >> 3 >> 5 = 2
> 
> On the other hand, if we receive an offset for an eSPI, e.g., 0x8200
> (GICD_IROUTERnE range [0x8000-0x9FFC]), we will get the same rank = 2
> when using the proposed solution:
> vgic_rank_offset(v, 64, 0x8200-0x8000, DABT_DOUBLE_WORD) ->
> DABT_DOUBLE_WORD = 3
> 
> rank = REG_RANK_NR(64, (0x200 >> 3) ->
> REG_RANK_NR will return also:
> 0x200 >> 3 >> 5 = 2
> 
> As a result, in both cases, vgic_get_rank(v, 2) will return the rank for
> regular SPIs:
> 
> /* This condition will apply because rank will be less than 32 */
> else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
>       return &v->domain->arch.vgic.shared_irqs[rank - 1];
> 
> Therefore, I decided to create a separate function to properly transform
> ranks into the correct range.
> 
> And, unfortunately, we cannot resolve the second issue with #ifdefs
> inside vgic_rank_offset, because this function will receive the same
> parameters for SPIs/eSPIs and we would not be able to determine whether
> we should return shared_irqs or ext_shared_irqs.

Thanks for such a detailed analysis. I was thinking that somehow we could:
  - reduce the number of #ifdef-s introduced by the current series as a 
whole
  - minimize the changes (optimize code) added in "[PATCH v4 10/12] 
xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers"

Maybe there is no valid concern at all, or I missed somethig. But, what 
worries me a bit is that said patch adds the emulation code for eSPI 
regs that are close for emulation code for regular SPI regs (e.g. please 
refer to VRANGE64(GICD_IROUTERnE, GICD_IROUTERnEN) - 
VRANGE64(GICD_IROUTER32, GICD_IROUTER1019) or 
VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN) - 
VRANGE32(GICD_IPRIORITYR, GICD_IPRIORITYRN)), besides the code 
duplication, we would have to keep both regular and entended handlers
in sync when doing possible updates/fixing the bugs.

While I completely agree with the dead code for eSPI-specific 
helpers/functions if we move them out of #ifdef CONFIG_GICV3_ESPI, what 
might be wrong if we at least drop #ifdef CONFIG_GICV3_ESPI for simple 
#define-s (non functional code), such as:

  #ifdef CONFIG_GICV3_ESPI
/* Additional registers for GICv3.1 */
#define GICD_IGROUPRnE               (0x1000)
#define GICD_IGROUPRnEN              (0x107C)
...

or

#ifdef CONFIG_GICV3_ESPI
#define ESPI_BASE_INTID 4096
#define ESPI_MAX_INTID  5119
#define NR_ESPI_IRQS    1024

#define ESPI_INTID2IDX(intid) ((intid) - ESPI_BASE_INTID)
#define ESPI_IDX2INTID(idx)   ((idx) + ESPI_BASE_INTID)
#endif

> 
>>
>>> +
>>> +static unsigned int vgic_num_spi_lines(struct domain *d)
>>> +{
>>> +    return d->arch.vgic.nr_spis + d->arch.vgic.nr_espis;
>>> +}
>>> +
>>> +static int init_vgic_espi(struct domain *d)
>>> +{
>>> +    unsigned int i, idx;
>>> +
>>> +    if ( d->arch.vgic.nr_espis == 0 )
>>> +        return 0;
>>> +
>>> +    d->arch.vgic.ext_shared_irqs =
>>> +        xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_EXT_RANKS(d));
>>> +    if ( d->arch.vgic.ext_shared_irqs == NULL )
>>> +        return -ENOMEM;
>>> +
>>> +    for ( i = d->arch.vgic.nr_spis, idx = 0;
>>> +          i < vgic_num_spi_lines(d); i++, idx++ )
>>> +        vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i],
>>> +                              ESPI_IDX2INTID(idx));
>>> +
>>> +    for ( i = 0; i < DOMAIN_NR_EXT_RANKS(d); i++ )
>>> +        vgic_rank_init(&d->arch.vgic.ext_shared_irqs[i], i, 0);
>>> +
>>> +    return 0;
>>> +}
>>> +
>>> +struct pending_irq *espi_to_pending(struct domain *d, unsigned int irq)
>>> +{
>>> +    irq = ESPI_INTID2IDX(irq) + d->arch.vgic.nr_spis;
>>> +    return &d->arch.vgic.pending_irqs[irq];
>>> +}
>>> +#else
>>> +static unsigned int init_vgic_espi(struct domain *d
>>> +{
>>> +    return 0;
>>> +}
>>> +
>>> +static unsigned int vgic_num_spi_lines(struct domain *d)
>>> +{
>>> +    return d->arch.vgic.nr_spis;
>>> +}
>>> +
>>> +struct pending_irq *espi_to_pending(struct domain *d, unsigned int irq)
>>> +{
>>> +    return NULL;
>>> +}
>>> +#endif
>>> +
>>> +static unsigned int vgic_num_alloc_irqs(struct domain *d)
>>> +{
>>> +    return vgic_num_spi_lines(d) + NR_LOCAL_IRQS;
>>> +}
>>> +
>>>    int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>>>    {
>>>        int i;
>>> @@ -131,6 +274,38 @@ int domain_vgic_init(struct domain *d, unsigned
>>> int nr_spis)
>>>         */
>>>        nr_spis = ROUNDUP(nr_spis, 32);
>>> +#ifdef CONFIG_GICV3_ESPI
>>> +    /*
>>> +     * During domain creation, the dom0less DomUs code or toolstack
>>> specifies
>>> +     * the maximum INTID, which is defined in the domain config
>>> subtracted by
>>> +     * 32 to cover the local IRQs (please see the comment to
>>> VGIC_DEF_NR_SPIS).
>>> +     * To compute the actual number of eSPI that will be usable for,
>>> +     * add back 32.
>>> +     */
>>> +    if ( nr_spis + 32 > ESPI_IDX2INTID(NR_ESPI_IRQS) )
>>> +        return -EINVAL;
>>> +
>>> +    if ( nr_spis + 32 >= ESPI_BASE_INTID )
>>> +    {
>>> +        d->arch.vgic.nr_espis = min(nr_spis - ESPI_BASE_INTID + 32,
>>> 1024U);
>>> +        /* Verify if GIC HW can handle provided INTID */
>>> +        if ( d->arch.vgic.nr_espis > gic_number_espis() )
>>> +            return -EINVAL;
>>> +        /*
>>> +         * Set the maximum available number for regular
>>> +         * SPI to pass the next check
>>> +         */
>>> +        nr_spis = VGIC_DEF_NR_SPIS;
>>> +        d->arch.vgic.has_espi = true;
>>> +    }
>>> +    else
>>> +    {
>>> +        /* Domain will use the regular SPI range */
>>> +        d->arch.vgic.nr_espis = 0;
>>> +        d->arch.vgic.has_espi = false;
>>> +    }
>>> +#endif
>>> +
>>>        /* Limit the number of virtual SPIs supported to (1020 - 32) =
>>> 988  */
>>>        if ( nr_spis > (1020 - NR_LOCAL_IRQS) )
>>>            return -EINVAL;
>>> @@ -145,7 +320,7 @@ int domain_vgic_init(struct domain *d, unsigned
>>> int nr_spis)
>>>            return -ENOMEM;
>>>        d->arch.vgic.pending_irqs =
>>> -        xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis);
>>> +        xzalloc_array(struct pending_irq, vgic_num_spi_lines(d));
>>>        if ( d->arch.vgic.pending_irqs == NULL )
>>>            return -ENOMEM;
>>> @@ -156,12 +331,16 @@ int domain_vgic_init(struct domain *d, unsigned
>>> int nr_spis)
>>>        for ( i = 0; i < DOMAIN_NR_RANKS(d); i++ )
>>>            vgic_rank_init(&d->arch.vgic.shared_irqs[i], i + 1, 0);
>>> +    ret = init_vgic_espi(d);
>>> +    if ( ret )
>>> +        return ret;
>>> +
>>>        ret = d->arch.vgic.handler->domain_init(d);
>>>        if ( ret )
>>>            return ret;
>>>        d->arch.vgic.allocated_irqs =
>>> -        xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d)));
>>> +        xzalloc_array(unsigned long,
>>> BITS_TO_LONGS(vgic_num_alloc_irqs(d)));
>>>        if ( !d->arch.vgic.allocated_irqs )
>>>            return -ENOMEM;
>>> @@ -195,9 +374,27 @@ void domain_vgic_free(struct domain *d)
>>>            }
>>>        }
>>> +#ifdef CONFIG_GICV3_ESPI
>>> +    for ( i = 0; i < d->arch.vgic.nr_espis; i++ )
>>> +    {
>>> +        struct pending_irq *p = spi_to_pending(d, ESPI_IDX2INTID(i));
>>> +
>>> +        if ( p->desc )
>>> +        {
>>> +            ret = release_guest_irq(d, p->irq);
>>> +            if ( ret )
>>> +                dprintk(XENLOG_G_WARNING, "d%u: Failed to release
>>> virq %u ret = %d\n",
>>
>>
>> NIT: you can use %pd for printing domain with its ID
>>
>>
> 
> I will fix it in V5.
> 
>>> +                        d->domain_id, p->irq, ret);
>>> +        }
>>> +    }
>>> +#endif
>>> +
>>>        if ( d->arch.vgic.handler )
>>>            d->arch.vgic.handler->domain_free(d);
>>>        xfree(d->arch.vgic.shared_irqs);
>>> +#ifdef CONFIG_GICV3_ESPI
>>> +    xfree(d->arch.vgic.ext_shared_irqs);
>>> +#endif
>>>        xfree(d->arch.vgic.pending_irqs);
>>>        xfree(d->arch.vgic.allocated_irqs);
>>>    }
>>> @@ -331,6 +528,8 @@ void arch_move_irqs(struct vcpu *v)
>>>            if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p-
>>>> status) )
>>>                irq_set_affinity(p->desc, cpu_mask);
>>>        }
>>> +
>>> +    arch_move_espis(v);
>>>    }
>>>    void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned int n)
>>> @@ -538,6 +737,8 @@ struct pending_irq *irq_to_pending(struct vcpu *v,
>>> unsigned int irq)
>>>            n = &v->arch.vgic.pending_irqs[irq];
>>>        else if ( is_lpi(irq) )
>>>            n = v->domain->arch.vgic.handler->lpi_to_pending(v->domain,
>>> irq);
>>> +    else if ( is_espi(irq) )
>>> +        n = espi_to_pending(v->domain, irq);
>>>        else
>>>            n = &v->domain->arch.vgic.pending_irqs[irq - 32];
>>>        return n;
>>> @@ -547,6 +748,9 @@ struct pending_irq *spi_to_pending(struct domain
>>> *d, unsigned int irq)
>>>    {
>>>        ASSERT(irq >= NR_LOCAL_IRQS);
>>> +    if ( is_espi(irq) )
>>> +        return espi_to_pending(d, irq);
>>> +
>>>        return &d->arch.vgic.pending_irqs[irq - 32];
>>>    }
>>> @@ -668,6 +872,9 @@ bool vgic_reserve_virq(struct domain *d, unsigned
>>> int virq)
>>>        if ( !vgic_is_valid_line(d, virq) )
>>>            return false;
>>> +    if ( is_espi(virq) )
>>> +        return vgic_reserve_espi_virq(d, virq);
>>> +
>>>        return !test_and_set_bit(virq, d->arch.vgic.allocated_irqs);
>>>    }
>>> @@ -685,7 +892,7 @@ int vgic_allocate_virq(struct domain *d, bool spi)
>>>        else
>>>        {
>>>            first = 32;
>>> -        end = vgic_num_irqs(d);
>>> +        end = vgic_num_alloc_irqs(d);
>>>        }
>>>        /*
>>
> 
> Best regards,
> Leonid

Re: [PATCH v4 08/12] xen/arm: vgic: add resource management for extended SPIs

[Leonid Komarianskyi]

Hello Oleksandr,

Thank you for your comments and suggestions - they really help improve 
the code :)

On 29.08.25 12:48, Oleksandr Tyshchenko wrote:
> 
> 
> On 28.08.25 22:09, Leonid Komarianskyi wrote:
>> Hello Oleksandr,
> 
> Hello Leonid
> 
> 
>>
>> Thank you for your comment and suggestions.
>>
>> On 28.08.25 20:34, Oleksandr Tyshchenko wrote:
>>>
>>>
>>> On 27.08.25 21:24, Leonid Komarianskyi wrote:
>>>
>>> Hello Leonid
>>>
>>>> This change introduces resource management in the VGIC to handle
>>>> extended SPIs introduced in GICv3.1. The pending_irqs and
>>>> allocated_irqs arrays are resized to support the required
>>>> number of eSPIs, based on what is supported by the hardware and
>>>> requested by the guest. A new field, ext_shared_irqs, is added
>>>> to the VGIC structure to store information about eSPIs, similar
>>>> to how shared_irqs is used for regular SPIs.
>>>>
>>>> Since the eSPI range starts at INTID 4096 and INTIDs between 1025
>>>> and 4095 are reserved, helper macros are introduced to simplify the
>>>> transformation of indices and to enable easier access to eSPI-specific
>>>> resources. These changes prepare the VGIC for processing eSPIs as
>>>> required by future functionality.
>>>>
>>>> The initialization and deinitialization paths for vgic have been 
>>>> updated
>>>> to allocate and free these resources appropriately. Additionally,
>>>> updated handling of INTIDs greater than 1024, passed from the toolstack
>>>> during domain creation, and verification logic ensures only valid 
>>>> SPI or
>>>> eSPI INTIDs are used.
>>>>
>>>> The existing SPI behavior remains unaffected when guests do not request
>>>> eSPIs, GIC hardware does not support them, or the CONFIG_GICV3_ESPI
>>>> option is disabled.
>>>>
>>>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>>>
>>>> ---
>>>> Changes in V4:
>>>> - added has_espi field to simplify determining whether a domain is able
>>>>     to operate with eSPI
>>>> - fixed formatting issues and misspellings
>>>>
>>>> Changes in V3:
>>>> - fixed formatting for lines with more than 80 symbols
>>>> - introduced helper functions to be able to use stubs in case of
>>>>     CONFIG_GICV3_ESPI disabled, and as a result, reduce the number of
>>>>     #ifdefs
>>>> - fixed checks for nr_spis in domain_vgic_init
>>>> - updated comment about nr_spis adjustments with dom0less mention
>>>> - moved comment with additional explanations before checks
>>>> - used unsigned int for indexes since they cannot be negative
>>>> - removed unnecessary parentheses
>>>> - move vgic_ext_rank_offset to the below ifdef guard, to reduce the
>>>>     number of ifdefs
>>>>
>>>> Changes in V2:
>>>> - change is_espi_rank to is_valid_espi_rank to verify whether the array
>>>>     element ext_shared_irqs exists. The previous version, is_espi_rank,
>>>>     only checked if the rank index was less than the maximum 
>>>> possible eSPI
>>>>     rank index, but this could potentially result in accessing a
>>>>     non-existing array element. To address this, is_valid_espi_rank was
>>>>     introduced, which ensures that the required eSPI rank exists
>>>> - move gic_number_espis to
>>>>     xen/arm: gicv3: implement handling of GICv3.1 eSPI
>>>> - update vgic_is_valid_irq checks to allow operating with eSPIs
>>>> - remove redundant newline in vgic_allocate_virq
>>>> ---
>>>>    xen/arch/arm/include/asm/vgic.h |  20 +++
>>>>    xen/arch/arm/vgic.c             | 213 +++++++++++++++++++++++++++ 
>>>> ++++-
>>>>    2 files changed, 230 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/
>>>> asm/vgic.h
>>>> index 3e7cbbb196..fb4cea73eb 100644
>>>> --- a/xen/arch/arm/include/asm/vgic.h
>>>> +++ b/xen/arch/arm/include/asm/vgic.h
>>>> @@ -146,6 +146,12 @@ struct vgic_dist {
>>>>        int nr_spis; /* Number of SPIs */
>>>>        unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
>>>>        struct vgic_irq_rank *shared_irqs;
>>>> +#ifdef CONFIG_GICV3_ESPI
>>>> +    struct vgic_irq_rank *ext_shared_irqs;
>>>> +    int nr_espis; /* Number of extended SPIs */
>>>
>>> Can nr_espis be negative?
>>>
>>
>> No, it cannot, so I will change it to unsigned int in V5.
>>
>>>> +    /* To simplify determining whether a domain is able to operate
>>>> with eSPI */
>>>> +    bool has_espi;
>>>
>>> I agree with the Volodymyr's comment provided in separate letter that we
>>> could avoid adding an extra var.
>>>
>>
>> Sure, as I mentioned to Volodymyr, I will remove it in V5.
>>
>>>> +#endif
>>>>        /*
>>>>         * SPIs are domain global, SGIs and PPIs are per-VCPU and 
>>>> stored in
>>>>         * struct arch_vcpu.
>>>> @@ -243,6 +249,14 @@ struct vgic_ops {
>>>>    /* Number of ranks of interrupt registers for a domain */
>>>>    #define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_spis+31)/32)
>>>> +#ifdef CONFIG_GICV3_ESPI
>>>> +#define DOMAIN_NR_EXT_RANKS(d) (((d)->arch.vgic.nr_espis+31)/32)
>>>> +#define EXT_RANK_MIN (ESPI_BASE_INTID/32)
>>>> +#define EXT_RANK_MAX ((ESPI_MAX_INTID+31)/32)
>>>> +#define EXT_RANK_NUM2IDX(num) ((num)-EXT_RANK_MIN)
>>>> +#define EXT_RANK_IDX2NUM(idx) ((idx)+EXT_RANK_MIN)
>>>> +#endif
>>>> +
>>>>    #define vgic_lock(v)   spin_lock_irq(&(v)->domain->arch.vgic.lock)
>>>>    #define vgic_unlock(v) spin_unlock_irq(&(v)->domain->arch.vgic.lock)
>>>> @@ -302,6 +316,12 @@ extern struct vgic_irq_rank
>>>> *vgic_rank_offset(struct vcpu *v,
>>>>                                                  unsigned int b,
>>>>                                                  unsigned int n,
>>>>                                                  unsigned int s);
>>>> +#ifdef CONFIG_GICV3_ESPI
>>>> +extern struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v,
>>>> +                                                  unsigned int b,
>>>> +                                                  unsigned int n,
>>>> +                                                  unsigned int s);
>>>> +#endif
>>>>    extern struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned
>>>> int irq);
>>>>    extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned
>>>> int n);
>>>>    extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, unsigned
>>>> int n);
>>>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>>>> index 2bbf4d99aa..f4b80cb05f 100644
>>>> --- a/xen/arch/arm/vgic.c
>>>> +++ b/xen/arch/arm/vgic.c
>>>> @@ -27,9 +27,82 @@
>>>>    bool vgic_is_valid_line(struct domain *d, unsigned int virq)
>>>>    {
>>>> +#ifdef CONFIG_GICV3_ESPI
>>>> +    if ( virq >= ESPI_BASE_INTID &&
>>>> +         virq < ESPI_IDX2INTID(d->arch.vgic.nr_espis) )
>>>> +        return true;
>>>> +#endif
>>>> +
>>>>        return virq < vgic_num_irqs(d);
>>>>    }
>>>> +#ifdef CONFIG_GICV3_ESPI
>>>> +/*
>>>> + * Since eSPI indexes start from 4096 and numbers from 1024 to
>>>> + * 4095 are forbidden, we need to check both lower and upper
>>>> + * limits for ranks.
>>>> + */
>>>> +static inline bool is_valid_espi_rank(struct domain *d, unsigned int
>>>> rank)
>>>> +{
>>>> +    return rank >= EXT_RANK_MIN &&
>>>> +           EXT_RANK_NUM2IDX(rank) < DOMAIN_NR_EXT_RANKS(d);
>>>> +}
>>>> +
>>>> +static inline struct vgic_irq_rank *vgic_get_espi_rank(struct vcpu *v,
>>>> +                                                       unsigned int
>>>> rank)
>>>> +{
>>>> +    return &v->domain-
>>>>> arch.vgic.ext_shared_irqs[EXT_RANK_NUM2IDX(rank)];
>>>> +}
>>>> +
>>>> +static inline bool vgic_reserve_espi_virq(struct domain *d, unsigned
>>>> int virq)
>>>> +{
>>>> +    return !test_and_set_bit(ESPI_INTID2IDX(virq) + vgic_num_irqs(d),
>>>> +                             d->arch.vgic.allocated_irqs);
>>>> +}
>>>> +
>>>> +static void arch_move_espis(struct vcpu *v)
>>>> +{
>>>> +    const cpumask_t *cpu_mask = cpumask_of(v->processor);
>>>> +    struct domain *d = v->domain;
>>>> +    struct pending_irq *p;
>>>> +    struct vcpu *v_target;
>>>> +    unsigned int i;
>>>> +
>>>> +    for ( i = ESPI_BASE_INTID;
>>>> +          i < EXT_RANK_IDX2NUM(d->arch.vgic.nr_espis); i++ )
>>>> +    {
>>>> +        v_target = vgic_get_target_vcpu(v, i);
>>>> +        p = irq_to_pending(v_target, i);
>>>> +
>>>> +        if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p-
>>>>> status) )
>>>> +            irq_set_affinity(p->desc, cpu_mask);
>>>> +    }
>>>> +}
>>>> +#else
>>>> +static inline bool is_valid_espi_rank(struct domain *d, unsigned int
>>>> rank)
>>>> +{
>>>> +    return false;
>>>> +}
>>>> +
>>>> +/*
>>>> + * This function is stub and will not be called if 
>>>> CONFIG_GICV3_ESPI=n,
>>>> + * because in this case, is_valid_espi_rank will always return false.
>>>> + */
>>>> +static inline struct vgic_irq_rank *vgic_get_espi_rank(struct vcpu *v,
>>>> +                                                       unsigned int
>>>> rank)
>>>> +{
>>>> +    ASSERT_UNREACHABLE();
>>>> +    return NULL;
>>>> +}
>>>> +
>>>> +static inline bool vgic_reserve_espi_virq(struct domain *d, unsigned
>>>> int virq)
>>>> +{
>>>> +    return false;
>>>> +}
>>>> +
>>>> +static void arch_move_espis(struct vcpu *v) { }
>>>> +#endif
>>>> +
>>>>    static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
>>>>                                                      unsigned int rank)
>>>>    {
>>>> @@ -37,6 +110,8 @@ static inline struct vgic_irq_rank
>>>> *vgic_get_rank(struct vcpu *v,
>>>>            return v->arch.vgic.private_irqs;
>>>>        else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
>>>>            return &v->domain->arch.vgic.shared_irqs[rank - 1];
>>>> +    else if ( is_valid_espi_rank(v->domain, rank) )
>>>> +        return vgic_get_espi_rank(v, rank);
>>>>        else
>>>>            return NULL;
>>>>    }
>>>> @@ -117,6 +192,74 @@ int domain_vgic_register(struct domain *d,
>>>> unsigned int *mmio_count)
>>>>        return 0;
>>>>    }
>>>> +#ifdef CONFIG_GICV3_ESPI
>>>> +/*
>>>> + * The function behavior is the same as for regular SPIs
>>>> (vgic_rank_offset),
>>>> + * but it operates with extended SPI ranks.
>>>> + */
>>>> +struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v, unsigned
>>>> int b,
>>>> +                                           unsigned int n, unsigned
>>>> int s)
>>>> +{
>>>> +    unsigned int rank = REG_RANK_NR(b, (n >> s));
>>>> +
>>>> +    return vgic_get_rank(v, rank + EXT_RANK_MIN);
>>>> +}
>>>
>>> So the difference between vgic_rank_offset() and vgic_ext_rank_offset()
>>> in using EXT_RANK_MIN in the latter. I am wondering, whether it makes
>>> sense to reuse the existing vgic_rank_offset() by moving #ifdef into it?
>>> Maybe it would be possible to optimize some code added in "[PATCH v4
>>> 10/12] xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers".
>>>
>>> e.g. __vgic_v3_distr_common_mmio_read:
>>>
>>> Code for the following cases looks very similar to the regular
>>> counterparts.
>>> case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
>>> case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
>>> case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
>>> case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
>>> case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
>>>
>>> The difference is in the said helper and the base calculated as
>>> "reg - GICD_XXX", where GICD_XXX is the start of the range for the eSPI-
>>> based reg (e.g. GICD_IPRIORITYRnE) or regular SPI-based reg
>>> (e.g.GICD_IPRIORITYR). The GICD_XXX could probably be chosen
>>> based on the "reg" value itself, for example:
>>>
>>> rank = vgic_rank_offset(v, 8, reg >= GICD_IPRIORITYRnE ? reg -
>>> GICD_IPRIORITYRnE : reg - GICD_IPRIORITYR, DABT_WORD);
>>>
>>> Please note, I am not requesting for any updates regarding to that, I am
>>> just thinking out loud.
>>>
>>
>> First of all, I have to say that I really tried to reuse the existing
>> code for regular SPIs whenever possible, but I could not come up with
>> something better than introducing a new function for eSPI ranks in this
>> case. This is due to several reasons:
>> 1) Currently, all eSPI-specific offsets are placed under #ifdef to avoid
>> or at least reduce the introduction of dead code or defines that will
>> not be used. This was requested by our teammates who are engaged in
>> MISRA fixes. Therefore, this is the first reason why eSPI-specific
>> elements are placed under the appropriate configuration. Without
>> CONFIG_GICV3_ESPI, we cannot use these offsets.
>>
>> 2) The proposed solution:
>>   > rank = vgic_rank_offset(v, 8, reg >= GICD_IPRIORITYRnE ? reg -
>>   > GICD_IPRIORITYRnE : reg - GICD_IPRIORITYR, DABT_WORD);
>>
>> will not work, unfortunately, because in this case, vgic_rank_offset
>> will always return ranks for regular SPIs.
>>
>> Long story short: we will encounter issues determining whether to use
>> shared_irqs (for regular SPIs) or ext_shared_irqs (for eSPIs).
>>
>> And here are the details about why this will not work:
>>
>> In the case of receiving an offset for a regular SPI, e.g., 0x6200
>> (GICD_IROUTER range [0x6100-0x7FD8]), we will get rank = 2 for regular 
>> SPIs:
>> vgic_rank_offset(v, 64, 0x6200-0x6000, DABT_DOUBLE_WORD) ->
>> DABT_DOUBLE_WORD = 3
>>
>> Thus:
>> rank = REG_RANK_NR(64, (0x200 >> 3) ->
>> REG_RANK_NR will return in this case:
>> 0x200 >> 3 >> 5 = 2
>>
>> On the other hand, if we receive an offset for an eSPI, e.g., 0x8200
>> (GICD_IROUTERnE range [0x8000-0x9FFC]), we will get the same rank = 2
>> when using the proposed solution:
>> vgic_rank_offset(v, 64, 0x8200-0x8000, DABT_DOUBLE_WORD) ->
>> DABT_DOUBLE_WORD = 3
>>
>> rank = REG_RANK_NR(64, (0x200 >> 3) ->
>> REG_RANK_NR will return also:
>> 0x200 >> 3 >> 5 = 2
>>
>> As a result, in both cases, vgic_get_rank(v, 2) will return the rank for
>> regular SPIs:
>>
>> /* This condition will apply because rank will be less than 32 */
>> else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
>>       return &v->domain->arch.vgic.shared_irqs[rank - 1];
>>
>> Therefore, I decided to create a separate function to properly transform
>> ranks into the correct range.
>>
>> And, unfortunately, we cannot resolve the second issue with #ifdefs
>> inside vgic_rank_offset, because this function will receive the same
>> parameters for SPIs/eSPIs and we would not be able to determine whether
>> we should return shared_irqs or ext_shared_irqs.
> 
> Thanks for such a detailed analysis. I was thinking that somehow we could:
>   - reduce the number of #ifdef-s introduced by the current series as a 
> whole
>   - minimize the changes (optimize code) added in "[PATCH v4 10/12] xen/ 
> arm: vgic-v3: add emulation of GICv3.1 eSPI registers"
> 
> Maybe there is no valid concern at all, or I missed somethig. But, what 
> worries me a bit is that said patch adds the emulation code for eSPI 
> regs that are close for emulation code for regular SPI regs (e.g. please 
> refer to VRANGE64(GICD_IROUTERnE, GICD_IROUTERnEN) - 
> VRANGE64(GICD_IROUTER32, GICD_IROUTER1019) or 
> VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN) - 
> VRANGE32(GICD_IPRIORITYR, GICD_IPRIORITYRN)), besides the code 
> duplication, we would have to keep both regular and entended handlers
> in sync when doing possible updates/fixing the bugs.
> 
> While I completely agree with the dead code for eSPI-specific helpers/ 
> functions if we move them out of #ifdef CONFIG_GICV3_ESPI, what might be 
> wrong if we at least drop #ifdef CONFIG_GICV3_ESPI for simple #define-s 
> (non functional code), such as:
> 
>   #ifdef CONFIG_GICV3_ESPI
> /* Additional registers for GICv3.1 */
> #define GICD_IGROUPRnE               (0x1000)
> #define GICD_IGROUPRnEN              (0x107C)
> ...
> 
> or
> 
> #ifdef CONFIG_GICV3_ESPI
> #define ESPI_BASE_INTID 4096
> #define ESPI_MAX_INTID  5119
> #define NR_ESPI_IRQS    1024
> 
> #define ESPI_INTID2IDX(intid) ((intid) - ESPI_BASE_INTID)
> #define ESPI_IDX2INTID(idx)   ((idx) + ESPI_BASE_INTID)
> #endif
> 

Thank you again - after our discussion, I reviewed the entire patch 
series again and figured out how to significantly reduce the number of 
#ifdefs while also reusing the code implemented for regular SPIs. This 
allows the introduction of vGIC handlers and avoids dead code. I will 
send V5 soon and hope it will look much better.

>>
>>>
>>>> +
>>>> +static unsigned int vgic_num_spi_lines(struct domain *d)
>>>> +{
>>>> +    return d->arch.vgic.nr_spis + d->arch.vgic.nr_espis;
>>>> +}
>>>> +
>>>> +static int init_vgic_espi(struct domain *d)
>>>> +{
>>>> +    unsigned int i, idx;
>>>> +
>>>> +    if ( d->arch.vgic.nr_espis == 0 )
>>>> +        return 0;
>>>> +
>>>> +    d->arch.vgic.ext_shared_irqs =
>>>> +        xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_EXT_RANKS(d));
>>>> +    if ( d->arch.vgic.ext_shared_irqs == NULL )
>>>> +        return -ENOMEM;
>>>> +
>>>> +    for ( i = d->arch.vgic.nr_spis, idx = 0;
>>>> +          i < vgic_num_spi_lines(d); i++, idx++ )
>>>> +        vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i],
>>>> +                              ESPI_IDX2INTID(idx));
>>>> +
>>>> +    for ( i = 0; i < DOMAIN_NR_EXT_RANKS(d); i++ )
>>>> +        vgic_rank_init(&d->arch.vgic.ext_shared_irqs[i], i, 0);
>>>> +
>>>> +    return 0;
>>>> +}
>>>> +
>>>> +struct pending_irq *espi_to_pending(struct domain *d, unsigned int 
>>>> irq)
>>>> +{
>>>> +    irq = ESPI_INTID2IDX(irq) + d->arch.vgic.nr_spis;
>>>> +    return &d->arch.vgic.pending_irqs[irq];
>>>> +}
>>>> +#else
>>>> +static unsigned int init_vgic_espi(struct domain *d
>>>> +{
>>>> +    return 0;
>>>> +}
>>>> +
>>>> +static unsigned int vgic_num_spi_lines(struct domain *d)
>>>> +{
>>>> +    return d->arch.vgic.nr_spis;
>>>> +}
>>>> +
>>>> +struct pending_irq *espi_to_pending(struct domain *d, unsigned int 
>>>> irq)
>>>> +{
>>>> +    return NULL;
>>>> +}
>>>> +#endif
>>>> +
>>>> +static unsigned int vgic_num_alloc_irqs(struct domain *d)
>>>> +{
>>>> +    return vgic_num_spi_lines(d) + NR_LOCAL_IRQS;
>>>> +}
>>>> +
>>>>    int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>>>>    {
>>>>        int i;
>>>> @@ -131,6 +274,38 @@ int domain_vgic_init(struct domain *d, unsigned
>>>> int nr_spis)
>>>>         */
>>>>        nr_spis = ROUNDUP(nr_spis, 32);
>>>> +#ifdef CONFIG_GICV3_ESPI
>>>> +    /*
>>>> +     * During domain creation, the dom0less DomUs code or toolstack
>>>> specifies
>>>> +     * the maximum INTID, which is defined in the domain config
>>>> subtracted by
>>>> +     * 32 to cover the local IRQs (please see the comment to
>>>> VGIC_DEF_NR_SPIS).
>>>> +     * To compute the actual number of eSPI that will be usable for,
>>>> +     * add back 32.
>>>> +     */
>>>> +    if ( nr_spis + 32 > ESPI_IDX2INTID(NR_ESPI_IRQS) )
>>>> +        return -EINVAL;
>>>> +
>>>> +    if ( nr_spis + 32 >= ESPI_BASE_INTID )
>>>> +    {
>>>> +        d->arch.vgic.nr_espis = min(nr_spis - ESPI_BASE_INTID + 32,
>>>> 1024U);
>>>> +        /* Verify if GIC HW can handle provided INTID */
>>>> +        if ( d->arch.vgic.nr_espis > gic_number_espis() )
>>>> +            return -EINVAL;
>>>> +        /*
>>>> +         * Set the maximum available number for regular
>>>> +         * SPI to pass the next check
>>>> +         */
>>>> +        nr_spis = VGIC_DEF_NR_SPIS;
>>>> +        d->arch.vgic.has_espi = true;
>>>> +    }
>>>> +    else
>>>> +    {
>>>> +        /* Domain will use the regular SPI range */
>>>> +        d->arch.vgic.nr_espis = 0;
>>>> +        d->arch.vgic.has_espi = false;
>>>> +    }
>>>> +#endif
>>>> +
>>>>        /* Limit the number of virtual SPIs supported to (1020 - 32) =
>>>> 988  */
>>>>        if ( nr_spis > (1020 - NR_LOCAL_IRQS) )
>>>>            return -EINVAL;
>>>> @@ -145,7 +320,7 @@ int domain_vgic_init(struct domain *d, unsigned
>>>> int nr_spis)
>>>>            return -ENOMEM;
>>>>        d->arch.vgic.pending_irqs =
>>>> -        xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis);
>>>> +        xzalloc_array(struct pending_irq, vgic_num_spi_lines(d));
>>>>        if ( d->arch.vgic.pending_irqs == NULL )
>>>>            return -ENOMEM;
>>>> @@ -156,12 +331,16 @@ int domain_vgic_init(struct domain *d, unsigned
>>>> int nr_spis)
>>>>        for ( i = 0; i < DOMAIN_NR_RANKS(d); i++ )
>>>>            vgic_rank_init(&d->arch.vgic.shared_irqs[i], i + 1, 0);
>>>> +    ret = init_vgic_espi(d);
>>>> +    if ( ret )
>>>> +        return ret;
>>>> +
>>>>        ret = d->arch.vgic.handler->domain_init(d);
>>>>        if ( ret )
>>>>            return ret;
>>>>        d->arch.vgic.allocated_irqs =
>>>> -        xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d)));
>>>> +        xzalloc_array(unsigned long,
>>>> BITS_TO_LONGS(vgic_num_alloc_irqs(d)));
>>>>        if ( !d->arch.vgic.allocated_irqs )
>>>>            return -ENOMEM;
>>>> @@ -195,9 +374,27 @@ void domain_vgic_free(struct domain *d)
>>>>            }
>>>>        }
>>>> +#ifdef CONFIG_GICV3_ESPI
>>>> +    for ( i = 0; i < d->arch.vgic.nr_espis; i++ )
>>>> +    {
>>>> +        struct pending_irq *p = spi_to_pending(d, ESPI_IDX2INTID(i));
>>>> +
>>>> +        if ( p->desc )
>>>> +        {
>>>> +            ret = release_guest_irq(d, p->irq);
>>>> +            if ( ret )
>>>> +                dprintk(XENLOG_G_WARNING, "d%u: Failed to release
>>>> virq %u ret = %d\n",
>>>
>>>
>>> NIT: you can use %pd for printing domain with its ID
>>>
>>>
>>
>> I will fix it in V5.
>>
>>>> +                        d->domain_id, p->irq, ret);
>>>> +        }
>>>> +    }
>>>> +#endif
>>>> +
>>>>        if ( d->arch.vgic.handler )
>>>>            d->arch.vgic.handler->domain_free(d);
>>>>        xfree(d->arch.vgic.shared_irqs);
>>>> +#ifdef CONFIG_GICV3_ESPI
>>>> +    xfree(d->arch.vgic.ext_shared_irqs);
>>>> +#endif
>>>>        xfree(d->arch.vgic.pending_irqs);
>>>>        xfree(d->arch.vgic.allocated_irqs);
>>>>    }
>>>> @@ -331,6 +528,8 @@ void arch_move_irqs(struct vcpu *v)
>>>>            if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p-
>>>>> status) )
>>>>                irq_set_affinity(p->desc, cpu_mask);
>>>>        }
>>>> +
>>>> +    arch_move_espis(v);
>>>>    }
>>>>    void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned int n)
>>>> @@ -538,6 +737,8 @@ struct pending_irq *irq_to_pending(struct vcpu *v,
>>>> unsigned int irq)
>>>>            n = &v->arch.vgic.pending_irqs[irq];
>>>>        else if ( is_lpi(irq) )
>>>>            n = v->domain->arch.vgic.handler->lpi_to_pending(v->domain,
>>>> irq);
>>>> +    else if ( is_espi(irq) )
>>>> +        n = espi_to_pending(v->domain, irq);
>>>>        else
>>>>            n = &v->domain->arch.vgic.pending_irqs[irq - 32];
>>>>        return n;
>>>> @@ -547,6 +748,9 @@ struct pending_irq *spi_to_pending(struct domain
>>>> *d, unsigned int irq)
>>>>    {
>>>>        ASSERT(irq >= NR_LOCAL_IRQS);
>>>> +    if ( is_espi(irq) )
>>>> +        return espi_to_pending(d, irq);
>>>> +
>>>>        return &d->arch.vgic.pending_irqs[irq - 32];
>>>>    }
>>>> @@ -668,6 +872,9 @@ bool vgic_reserve_virq(struct domain *d, unsigned
>>>> int virq)
>>>>        if ( !vgic_is_valid_line(d, virq) )
>>>>            return false;
>>>> +    if ( is_espi(virq) )
>>>> +        return vgic_reserve_espi_virq(d, virq);
>>>> +
>>>>        return !test_and_set_bit(virq, d->arch.vgic.allocated_irqs);
>>>>    }
>>>> @@ -685,7 +892,7 @@ int vgic_allocate_virq(struct domain *d, bool spi)
>>>>        else
>>>>        {
>>>>            first = 32;
>>>> -        end = vgic_num_irqs(d);
>>>> +        end = vgic_num_alloc_irqs(d);
>>>>        }
>>>>        /*
>>>
>>
>> Best regards,
>> Leonid
> 

Best regards,
Leonid

Re: [PATCH v4 01/12] xen/arm: gicv3: refactor obtaining GIC addresses for common operations

[Julien Grall]

Hi Leonid,

On 28/08/2025 17:17, Leonid Komarianskyi wrote:
> On 28.08.25 15:00, Julien Grall wrote:
>> Hi Leonid,
>>
>> On 27/08/2025 19:24, Leonid Komarianskyi wrote:
>>> Currently, many common functions perform the same operations to calculate
>>> GIC register addresses. This patch consolidates the similar code into
>>> a separate helper function to improve maintainability and reduce
>>> duplication.
>>> This refactoring also simplifies the implementation of eSPI support in
>>> future
>>> changes.
>>>
>>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>> Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
>>>
>>> ---
>>> Changes in V4:
>>> - no changes
>>>
>>> Changes in V3:
>>> - changed panic() in get_addr_by_offset() to printing warning and
>>>     ASSERT_UNREACHABLE()
>>> - added verification of return pointer from get_addr_by_offset() in the
>>>     callers
>>> - moved invocation of get_addr_by_offset() from spinlock guards, since
>>>     it is not necessarry
>>> - added RB from Volodymyr Babchuk
>>
>> Procces remark, here you said the Reviewed-by from Volodymyr was added
>> in v3. However, given the changes you made this should have been
>> invalidated (reviewed-by means the person read the code and confirmed it
>> is correct).
>>
>> I see Volodymyr confirmed his reviewed-by on v3. So no issue, but this
>> should have been clarified in the changelog.
>>
> 
> Thank you for your explanation.
> Just to clarify: would it be okay to leave the RB tag (with appropriate
> text in the changelog) if I fix some minor nit from another reviewer in
> the next version, like in this patch?

It depends on the change. In general, typoes or coding style changes (I 
include s/uX/uint_X) are fine to keep the review by. Anything else may 
need a review again.

Acked-by are different because they don't carry a full review. So for 
slightly bigger change it would be fine to keep. But if the logic is 
fully rewritten, then they would need to be dropped.

Cheers,

-- 
Julien Grall