* [PATCH v2 03/10] xen/arm: vgic: implement helper functions for virq checks
2025-08-07 12:33 ` [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations Leonid Komarianskyi
@ 2025-08-07 12:33 ` Leonid Komarianskyi
2025-08-21 15:46 ` Volodymyr Babchuk
2025-08-23 12:29 ` Oleksandr Tyshchenko
2025-08-07 12:33 ` [PATCH v2 02/10] xen/arm: gic: implement helper functions for INTID checks Leonid Komarianskyi
` (9 subsequent siblings)
10 siblings, 2 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-07 12:33 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Leonid Komarianskyi, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk
Introduced two new helper functions for vGIC: vgic_is_valid_irq and
vgic_is_shared_irq. The functions are similar to the newly introduced
gic_is_valid_irq and gic_is_shared_irq, but they verify whether a vIRQ
is available for a specific domain, while GIC-specific functions
validate INTIDs for the real GIC hardware. For example, the GIC may
support all 992 SPI lines, but the domain may use only some part of them
(e.g., 640), depending on the highest IRQ number defined in the domain
configuration. Therefore, for vGIC-related code and checks, the
appropriate functions should be used. Also, updated the appropriate
checks to use these new helper functions.
The purpose of introducing new helper functions for vGIC is essentially
the same as for GIC: to avoid potential confusion with GIC-related
checks and to consolidate similar code into separate functions, which
can be more easily extended by additional conditions, e.g., when
implementing extended SPI interrupts.
Only the validation change in vgic_inject_irq may affect existing
functionality, as it currently checks whether the vIRQ is less than or
equal to vgic_num_irqs. Since IRQ indexes start from 0 (where 32 is the
first SPI), the check should behave consistently with similar logic in
other places and should check if the vIRQ number is less than
vgic_num_irqs. The remaining changes, which replace open-coded checks
with the use of these new helper functions, do not introduce any
functional changes, as the helper functions follow the current vIRQ
index verification logic.
Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
---
Changes in V2:
- introduced this patch
---
xen/arch/arm/gic.c | 3 +--
xen/arch/arm/include/asm/vgic.h | 7 +++++++
xen/arch/arm/irq.c | 4 ++--
xen/arch/arm/vgic.c | 10 ++++++++--
4 files changed, 18 insertions(+), 6 deletions(-)
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index eb0346a898..47fccf21d8 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -133,8 +133,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
ASSERT(spin_is_locked(&desc->lock));
/* Caller has already checked that the IRQ is an SPI */
- ASSERT(virq >= 32);
- ASSERT(virq < vgic_num_irqs(d));
+ ASSERT(vgic_is_shared_irq(d, virq));
ASSERT(!is_lpi(virq));
ret = vgic_connect_hw_irq(d, NULL, virq, desc, true);
diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
index 35c0c6a8b0..45201f4ca5 100644
--- a/xen/arch/arm/include/asm/vgic.h
+++ b/xen/arch/arm/include/asm/vgic.h
@@ -335,6 +335,13 @@ extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
/* Default number of vGIC SPIs. 32 are substracted to cover local IRQs. */
#define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
+extern bool vgic_is_valid_irq(struct domain *d, unsigned int virq);
+
+static inline bool vgic_is_shared_irq(struct domain *d, unsigned int virq)
+{
+ return (virq >= NR_LOCAL_IRQS && vgic_is_valid_irq(d, virq));
+}
+
/*
* Allocate a guest VIRQ
* - spi == 0 => allocate a PPI. It will be the same on every vCPU
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 12c70d02cc..50e57aaea7 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -442,7 +442,7 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
unsigned long flags;
int retval = 0;
- if ( virq >= vgic_num_irqs(d) )
+ if ( !vgic_is_valid_irq(d, virq) )
{
printk(XENLOG_G_ERR
"the vIRQ number %u is too high for domain %u (max = %u)\n",
@@ -560,7 +560,7 @@ int release_guest_irq(struct domain *d, unsigned int virq)
int ret;
/* Only SPIs are supported */
- if ( virq < NR_LOCAL_IRQS || virq >= vgic_num_irqs(d) )
+ if ( !vgic_is_shared_irq(d, virq) )
return -EINVAL;
desc = vgic_get_hw_irq_desc(d, NULL, virq);
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index c563ba93af..48fbaf56fb 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -24,6 +24,12 @@
#include <asm/gic.h>
#include <asm/vgic.h>
+
+bool vgic_is_valid_irq(struct domain *d, unsigned int virq)
+{
+ return virq < vgic_num_irqs(d);
+}
+
static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
unsigned int rank)
{
@@ -582,7 +588,7 @@ void vgic_inject_irq(struct domain *d, struct vcpu *v, unsigned int virq,
if ( !v )
{
/* The IRQ needs to be an SPI if no vCPU is specified. */
- ASSERT(virq >= 32 && virq <= vgic_num_irqs(d));
+ ASSERT(vgic_is_shared_irq(d, virq));
v = vgic_get_target_vcpu(d->vcpu[0], virq);
};
@@ -659,7 +665,7 @@ bool vgic_emulate(struct cpu_user_regs *regs, union hsr hsr)
bool vgic_reserve_virq(struct domain *d, unsigned int virq)
{
- if ( virq >= vgic_num_irqs(d) )
+ if ( !vgic_is_valid_irq(d, virq) )
return false;
return !test_and_set_bit(virq, d->arch.vgic.allocated_irqs);
--
2.34.1
^ permalink raw reply related [flat|nested] 55+ messages in thread* Re: [PATCH v2 03/10] xen/arm: vgic: implement helper functions for virq checks
2025-08-07 12:33 ` [PATCH v2 03/10] xen/arm: vgic: implement helper functions for virq checks Leonid Komarianskyi
@ 2025-08-21 15:46 ` Volodymyr Babchuk
2025-08-22 7:55 ` Leonid Komarianskyi
2025-08-23 12:29 ` Oleksandr Tyshchenko
1 sibling, 1 reply; 55+ messages in thread
From: Volodymyr Babchuk @ 2025-08-21 15:46 UTC (permalink / raw)
To: Leonid Komarianskyi
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
> Introduced two new helper functions for vGIC: vgic_is_valid_irq and
> vgic_is_shared_irq. The functions are similar to the newly introduced
> gic_is_valid_irq and gic_is_shared_irq, but they verify whether a vIRQ
> is available for a specific domain, while GIC-specific functions
> validate INTIDs for the real GIC hardware. For example, the GIC may
> support all 992 SPI lines, but the domain may use only some part of them
> (e.g., 640), depending on the highest IRQ number defined in the domain
> configuration. Therefore, for vGIC-related code and checks, the
> appropriate functions should be used. Also, updated the appropriate
> checks to use these new helper functions.
>
> The purpose of introducing new helper functions for vGIC is essentially
> the same as for GIC: to avoid potential confusion with GIC-related
> checks and to consolidate similar code into separate functions, which
> can be more easily extended by additional conditions, e.g., when
> implementing extended SPI interrupts.
>
> Only the validation change in vgic_inject_irq may affect existing
> functionality, as it currently checks whether the vIRQ is less than or
> equal to vgic_num_irqs. Since IRQ indexes start from 0 (where 32 is the
> first SPI), the check should behave consistently with similar logic in
> other places and should check if the vIRQ number is less than
> vgic_num_irqs. The remaining changes, which replace open-coded checks
> with the use of these new helper functions, do not introduce any
> functional changes, as the helper functions follow the current vIRQ
> index verification logic.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>
> ---
> Changes in V2:
> - introduced this patch
> ---
> xen/arch/arm/gic.c | 3 +--
> xen/arch/arm/include/asm/vgic.h | 7 +++++++
> xen/arch/arm/irq.c | 4 ++--
> xen/arch/arm/vgic.c | 10 ++++++++--
> 4 files changed, 18 insertions(+), 6 deletions(-)
>
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index eb0346a898..47fccf21d8 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -133,8 +133,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
>
> ASSERT(spin_is_locked(&desc->lock));
> /* Caller has already checked that the IRQ is an SPI */
> - ASSERT(virq >= 32);
> - ASSERT(virq < vgic_num_irqs(d));
> + ASSERT(vgic_is_shared_irq(d, virq));
> ASSERT(!is_lpi(virq));
>
> ret = vgic_connect_hw_irq(d, NULL, virq, desc, true);
> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
> index 35c0c6a8b0..45201f4ca5 100644
> --- a/xen/arch/arm/include/asm/vgic.h
> +++ b/xen/arch/arm/include/asm/vgic.h
> @@ -335,6 +335,13 @@ extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
> /* Default number of vGIC SPIs. 32 are substracted to cover local IRQs. */
> #define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
>
> +extern bool vgic_is_valid_irq(struct domain *d, unsigned int virq);
> +
> +static inline bool vgic_is_shared_irq(struct domain *d, unsigned int virq)
> +{
> + return (virq >= NR_LOCAL_IRQS && vgic_is_valid_irq(d, virq));
> +}
> +
> /*
> * Allocate a guest VIRQ
> * - spi == 0 => allocate a PPI. It will be the same on every vCPU
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 12c70d02cc..50e57aaea7 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -442,7 +442,7 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
> unsigned long flags;
> int retval = 0;
>
> - if ( virq >= vgic_num_irqs(d) )
> + if ( !vgic_is_valid_irq(d, virq) )
> {
> printk(XENLOG_G_ERR
> "the vIRQ number %u is too high for domain %u (max = %u)\n",
> @@ -560,7 +560,7 @@ int release_guest_irq(struct domain *d, unsigned int virq)
> int ret;
>
> /* Only SPIs are supported */
> - if ( virq < NR_LOCAL_IRQS || virq >= vgic_num_irqs(d) )
> + if ( !vgic_is_shared_irq(d, virq) )
> return -EINVAL;
>
> desc = vgic_get_hw_irq_desc(d, NULL, virq);
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index c563ba93af..48fbaf56fb 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -24,6 +24,12 @@
> #include <asm/gic.h>
> #include <asm/vgic.h>
>
> +
> +bool vgic_is_valid_irq(struct domain *d, unsigned int virq)
I have the same comment as for the previous patch. This function
completely ignores LPIs presence, while you can't argue that LPIs as
valid. Again, function callers are expecting this behavior, so this is
fine, but function name should better reflect its behavior.
[...]
--
WBR, Volodymyr
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 03/10] xen/arm: vgic: implement helper functions for virq checks
2025-08-21 15:46 ` Volodymyr Babchuk
@ 2025-08-22 7:55 ` Leonid Komarianskyi
2025-08-22 12:28 ` Volodymyr Babchuk
0 siblings, 1 reply; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-22 7:55 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Hi Volodymyr,
Thank you for you comment.
On 21.08.25 18:46, Volodymyr Babchuk wrote:
>
> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>
>> Introduced two new helper functions for vGIC: vgic_is_valid_irq and
>> vgic_is_shared_irq. The functions are similar to the newly introduced
>> gic_is_valid_irq and gic_is_shared_irq, but they verify whether a vIRQ
>> is available for a specific domain, while GIC-specific functions
>> validate INTIDs for the real GIC hardware. For example, the GIC may
>> support all 992 SPI lines, but the domain may use only some part of them
>> (e.g., 640), depending on the highest IRQ number defined in the domain
>> configuration. Therefore, for vGIC-related code and checks, the
>> appropriate functions should be used. Also, updated the appropriate
>> checks to use these new helper functions.
>>
>> The purpose of introducing new helper functions for vGIC is essentially
>> the same as for GIC: to avoid potential confusion with GIC-related
>> checks and to consolidate similar code into separate functions, which
>> can be more easily extended by additional conditions, e.g., when
>> implementing extended SPI interrupts.
>>
>> Only the validation change in vgic_inject_irq may affect existing
>> functionality, as it currently checks whether the vIRQ is less than or
>> equal to vgic_num_irqs. Since IRQ indexes start from 0 (where 32 is the
>> first SPI), the check should behave consistently with similar logic in
>> other places and should check if the vIRQ number is less than
>> vgic_num_irqs. The remaining changes, which replace open-coded checks
>> with the use of these new helper functions, do not introduce any
>> functional changes, as the helper functions follow the current vIRQ
>> index verification logic.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V2:
>> - introduced this patch
>> ---
>> xen/arch/arm/gic.c | 3 +--
>> xen/arch/arm/include/asm/vgic.h | 7 +++++++
>> xen/arch/arm/irq.c | 4 ++--
>> xen/arch/arm/vgic.c | 10 ++++++++--
>> 4 files changed, 18 insertions(+), 6 deletions(-)
>>
>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>> index eb0346a898..47fccf21d8 100644
>> --- a/xen/arch/arm/gic.c
>> +++ b/xen/arch/arm/gic.c
>> @@ -133,8 +133,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
>>
>> ASSERT(spin_is_locked(&desc->lock));
>> /* Caller has already checked that the IRQ is an SPI */
>> - ASSERT(virq >= 32);
>> - ASSERT(virq < vgic_num_irqs(d));
>> + ASSERT(vgic_is_shared_irq(d, virq));
>> ASSERT(!is_lpi(virq));
>>
>> ret = vgic_connect_hw_irq(d, NULL, virq, desc, true);
>> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
>> index 35c0c6a8b0..45201f4ca5 100644
>> --- a/xen/arch/arm/include/asm/vgic.h
>> +++ b/xen/arch/arm/include/asm/vgic.h
>> @@ -335,6 +335,13 @@ extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
>> /* Default number of vGIC SPIs. 32 are substracted to cover local IRQs. */
>> #define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
>>
>> +extern bool vgic_is_valid_irq(struct domain *d, unsigned int virq);
>> +
>> +static inline bool vgic_is_shared_irq(struct domain *d, unsigned int virq)
>> +{
>> + return (virq >= NR_LOCAL_IRQS && vgic_is_valid_irq(d, virq));
>> +}
>> +
>> /*
>> * Allocate a guest VIRQ
>> * - spi == 0 => allocate a PPI. It will be the same on every vCPU
>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>> index 12c70d02cc..50e57aaea7 100644
>> --- a/xen/arch/arm/irq.c
>> +++ b/xen/arch/arm/irq.c
>> @@ -442,7 +442,7 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
>> unsigned long flags;
>> int retval = 0;
>>
>> - if ( virq >= vgic_num_irqs(d) )
>> + if ( !vgic_is_valid_irq(d, virq) )
>> {
>> printk(XENLOG_G_ERR
>> "the vIRQ number %u is too high for domain %u (max = %u)\n",
>> @@ -560,7 +560,7 @@ int release_guest_irq(struct domain *d, unsigned int virq)
>> int ret;
>>
>> /* Only SPIs are supported */
>> - if ( virq < NR_LOCAL_IRQS || virq >= vgic_num_irqs(d) )
>> + if ( !vgic_is_shared_irq(d, virq) )
>> return -EINVAL;
>>
>> desc = vgic_get_hw_irq_desc(d, NULL, virq);
>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>> index c563ba93af..48fbaf56fb 100644
>> --- a/xen/arch/arm/vgic.c
>> +++ b/xen/arch/arm/vgic.c
>> @@ -24,6 +24,12 @@
>> #include <asm/gic.h>
>> #include <asm/vgic.h>
>>
>> +
>> +bool vgic_is_valid_irq(struct domain *d, unsigned int virq)
>
> I have the same comment as for the previous patch. This function
> completely ignores LPIs presence, while you can't argue that LPIs as
> valid. Again, function callers are expecting this behavior, so this is
> fine, but function name should better reflect its behavior.
>
> [...]
>
Would it be okay to rename these functions as proposed in the previous
patch discussion:
vgic_is_valid_irq -> vgic_is_valid_line
vgic_is_shared_irq -> vgic_is_spi?
Or, in the case of vgic, is it not a good idea to use the "line" suffix
because vgic does not have physical interrupt lines? Would it be better
to rename it to vgic_is_valid_non_lpi instead?
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 03/10] xen/arm: vgic: implement helper functions for virq checks
2025-08-22 7:55 ` Leonid Komarianskyi
@ 2025-08-22 12:28 ` Volodymyr Babchuk
2025-08-22 15:08 ` Leonid Komarianskyi
0 siblings, 1 reply; 55+ messages in thread
From: Volodymyr Babchuk @ 2025-08-22 12:28 UTC (permalink / raw)
To: Leonid Komarianskyi
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Leonid,
Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
> Hi Volodymyr,
>
> Thank you for you comment.
>
> On 21.08.25 18:46, Volodymyr Babchuk wrote:
>>
>> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>>
>>> Introduced two new helper functions for vGIC: vgic_is_valid_irq and
>>> vgic_is_shared_irq. The functions are similar to the newly introduced
>>> gic_is_valid_irq and gic_is_shared_irq, but they verify whether a vIRQ
>>> is available for a specific domain, while GIC-specific functions
>>> validate INTIDs for the real GIC hardware. For example, the GIC may
>>> support all 992 SPI lines, but the domain may use only some part of them
>>> (e.g., 640), depending on the highest IRQ number defined in the domain
>>> configuration. Therefore, for vGIC-related code and checks, the
>>> appropriate functions should be used. Also, updated the appropriate
>>> checks to use these new helper functions.
>>>
>>> The purpose of introducing new helper functions for vGIC is essentially
>>> the same as for GIC: to avoid potential confusion with GIC-related
>>> checks and to consolidate similar code into separate functions, which
>>> can be more easily extended by additional conditions, e.g., when
>>> implementing extended SPI interrupts.
>>>
>>> Only the validation change in vgic_inject_irq may affect existing
>>> functionality, as it currently checks whether the vIRQ is less than or
>>> equal to vgic_num_irqs. Since IRQ indexes start from 0 (where 32 is the
>>> first SPI), the check should behave consistently with similar logic in
>>> other places and should check if the vIRQ number is less than
>>> vgic_num_irqs. The remaining changes, which replace open-coded checks
>>> with the use of these new helper functions, do not introduce any
>>> functional changes, as the helper functions follow the current vIRQ
>>> index verification logic.
>>>
>>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>>
>>> ---
>>> Changes in V2:
>>> - introduced this patch
>>> ---
>>> xen/arch/arm/gic.c | 3 +--
>>> xen/arch/arm/include/asm/vgic.h | 7 +++++++
>>> xen/arch/arm/irq.c | 4 ++--
>>> xen/arch/arm/vgic.c | 10 ++++++++--
>>> 4 files changed, 18 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>>> index eb0346a898..47fccf21d8 100644
>>> --- a/xen/arch/arm/gic.c
>>> +++ b/xen/arch/arm/gic.c
>>> @@ -133,8 +133,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
>>>
>>> ASSERT(spin_is_locked(&desc->lock));
>>> /* Caller has already checked that the IRQ is an SPI */
>>> - ASSERT(virq >= 32);
>>> - ASSERT(virq < vgic_num_irqs(d));
>>> + ASSERT(vgic_is_shared_irq(d, virq));
>>> ASSERT(!is_lpi(virq));
>>>
>>> ret = vgic_connect_hw_irq(d, NULL, virq, desc, true);
>>> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
>>> index 35c0c6a8b0..45201f4ca5 100644
>>> --- a/xen/arch/arm/include/asm/vgic.h
>>> +++ b/xen/arch/arm/include/asm/vgic.h
>>> @@ -335,6 +335,13 @@ extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
>>> /* Default number of vGIC SPIs. 32 are substracted to cover local IRQs. */
>>> #define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
>>>
>>> +extern bool vgic_is_valid_irq(struct domain *d, unsigned int virq);
>>> +
>>> +static inline bool vgic_is_shared_irq(struct domain *d, unsigned int virq)
>>> +{
>>> + return (virq >= NR_LOCAL_IRQS && vgic_is_valid_irq(d, virq));
>>> +}
>>> +
>>> /*
>>> * Allocate a guest VIRQ
>>> * - spi == 0 => allocate a PPI. It will be the same on every vCPU
>>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>>> index 12c70d02cc..50e57aaea7 100644
>>> --- a/xen/arch/arm/irq.c
>>> +++ b/xen/arch/arm/irq.c
>>> @@ -442,7 +442,7 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
>>> unsigned long flags;
>>> int retval = 0;
>>>
>>> - if ( virq >= vgic_num_irqs(d) )
>>> + if ( !vgic_is_valid_irq(d, virq) )
>>> {
>>> printk(XENLOG_G_ERR
>>> "the vIRQ number %u is too high for domain %u (max = %u)\n",
>>> @@ -560,7 +560,7 @@ int release_guest_irq(struct domain *d, unsigned int virq)
>>> int ret;
>>>
>>> /* Only SPIs are supported */
>>> - if ( virq < NR_LOCAL_IRQS || virq >= vgic_num_irqs(d) )
>>> + if ( !vgic_is_shared_irq(d, virq) )
>>> return -EINVAL;
>>>
>>> desc = vgic_get_hw_irq_desc(d, NULL, virq);
>>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>>> index c563ba93af..48fbaf56fb 100644
>>> --- a/xen/arch/arm/vgic.c
>>> +++ b/xen/arch/arm/vgic.c
>>> @@ -24,6 +24,12 @@
>>> #include <asm/gic.h>
>>> #include <asm/vgic.h>
>>>
>>> +
>>> +bool vgic_is_valid_irq(struct domain *d, unsigned int virq)
>>
>> I have the same comment as for the previous patch. This function
>> completely ignores LPIs presence, while you can't argue that LPIs as
>> valid. Again, function callers are expecting this behavior, so this is
>> fine, but function name should better reflect its behavior.
>>
>> [...]
>>
>
> Would it be okay to rename these functions as proposed in the previous
> patch discussion:
> vgic_is_valid_irq -> vgic_is_valid_line
> vgic_is_shared_irq -> vgic_is_spi?
>
> Or, in the case of vgic, is it not a good idea to use the "line" suffix
> because vgic does not have physical interrupt lines? Would it be better
> to rename it to vgic_is_valid_non_lpi instead?
I think it is better to follow the pGIC naming convention. While there
is no physical IRQ lines in vGIC, it emulates real GIC anyways.
--
WBR, Volodymyr
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 03/10] xen/arm: vgic: implement helper functions for virq checks
2025-08-22 12:28 ` Volodymyr Babchuk
@ 2025-08-22 15:08 ` Leonid Komarianskyi
0 siblings, 0 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-22 15:08 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Hi Volodymyr,
On 22.08.25 15:28, Volodymyr Babchuk wrote:
>
> Leonid,
>
> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>
>> Hi Volodymyr,
>>
>> Thank you for you comment.
>>
>> On 21.08.25 18:46, Volodymyr Babchuk wrote:
>>>
>>> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>>>
>>>> Introduced two new helper functions for vGIC: vgic_is_valid_irq and
>>>> vgic_is_shared_irq. The functions are similar to the newly introduced
>>>> gic_is_valid_irq and gic_is_shared_irq, but they verify whether a vIRQ
>>>> is available for a specific domain, while GIC-specific functions
>>>> validate INTIDs for the real GIC hardware. For example, the GIC may
>>>> support all 992 SPI lines, but the domain may use only some part of them
>>>> (e.g., 640), depending on the highest IRQ number defined in the domain
>>>> configuration. Therefore, for vGIC-related code and checks, the
>>>> appropriate functions should be used. Also, updated the appropriate
>>>> checks to use these new helper functions.
>>>>
>>>> The purpose of introducing new helper functions for vGIC is essentially
>>>> the same as for GIC: to avoid potential confusion with GIC-related
>>>> checks and to consolidate similar code into separate functions, which
>>>> can be more easily extended by additional conditions, e.g., when
>>>> implementing extended SPI interrupts.
>>>>
>>>> Only the validation change in vgic_inject_irq may affect existing
>>>> functionality, as it currently checks whether the vIRQ is less than or
>>>> equal to vgic_num_irqs. Since IRQ indexes start from 0 (where 32 is the
>>>> first SPI), the check should behave consistently with similar logic in
>>>> other places and should check if the vIRQ number is less than
>>>> vgic_num_irqs. The remaining changes, which replace open-coded checks
>>>> with the use of these new helper functions, do not introduce any
>>>> functional changes, as the helper functions follow the current vIRQ
>>>> index verification logic.
>>>>
>>>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>>>
>>>> ---
>>>> Changes in V2:
>>>> - introduced this patch
>>>> ---
>>>> xen/arch/arm/gic.c | 3 +--
>>>> xen/arch/arm/include/asm/vgic.h | 7 +++++++
>>>> xen/arch/arm/irq.c | 4 ++--
>>>> xen/arch/arm/vgic.c | 10 ++++++++--
>>>> 4 files changed, 18 insertions(+), 6 deletions(-)
>>>>
>>>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>>>> index eb0346a898..47fccf21d8 100644
>>>> --- a/xen/arch/arm/gic.c
>>>> +++ b/xen/arch/arm/gic.c
>>>> @@ -133,8 +133,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
>>>>
>>>> ASSERT(spin_is_locked(&desc->lock));
>>>> /* Caller has already checked that the IRQ is an SPI */
>>>> - ASSERT(virq >= 32);
>>>> - ASSERT(virq < vgic_num_irqs(d));
>>>> + ASSERT(vgic_is_shared_irq(d, virq));
>>>> ASSERT(!is_lpi(virq));
>>>>
>>>> ret = vgic_connect_hw_irq(d, NULL, virq, desc, true);
>>>> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
>>>> index 35c0c6a8b0..45201f4ca5 100644
>>>> --- a/xen/arch/arm/include/asm/vgic.h
>>>> +++ b/xen/arch/arm/include/asm/vgic.h
>>>> @@ -335,6 +335,13 @@ extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
>>>> /* Default number of vGIC SPIs. 32 are substracted to cover local IRQs. */
>>>> #define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
>>>>
>>>> +extern bool vgic_is_valid_irq(struct domain *d, unsigned int virq);
>>>> +
>>>> +static inline bool vgic_is_shared_irq(struct domain *d, unsigned int virq)
>>>> +{
>>>> + return (virq >= NR_LOCAL_IRQS && vgic_is_valid_irq(d, virq));
>>>> +}
>>>> +
>>>> /*
>>>> * Allocate a guest VIRQ
>>>> * - spi == 0 => allocate a PPI. It will be the same on every vCPU
>>>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>>>> index 12c70d02cc..50e57aaea7 100644
>>>> --- a/xen/arch/arm/irq.c
>>>> +++ b/xen/arch/arm/irq.c
>>>> @@ -442,7 +442,7 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
>>>> unsigned long flags;
>>>> int retval = 0;
>>>>
>>>> - if ( virq >= vgic_num_irqs(d) )
>>>> + if ( !vgic_is_valid_irq(d, virq) )
>>>> {
>>>> printk(XENLOG_G_ERR
>>>> "the vIRQ number %u is too high for domain %u (max = %u)\n",
>>>> @@ -560,7 +560,7 @@ int release_guest_irq(struct domain *d, unsigned int virq)
>>>> int ret;
>>>>
>>>> /* Only SPIs are supported */
>>>> - if ( virq < NR_LOCAL_IRQS || virq >= vgic_num_irqs(d) )
>>>> + if ( !vgic_is_shared_irq(d, virq) )
>>>> return -EINVAL;
>>>>
>>>> desc = vgic_get_hw_irq_desc(d, NULL, virq);
>>>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>>>> index c563ba93af..48fbaf56fb 100644
>>>> --- a/xen/arch/arm/vgic.c
>>>> +++ b/xen/arch/arm/vgic.c
>>>> @@ -24,6 +24,12 @@
>>>> #include <asm/gic.h>
>>>> #include <asm/vgic.h>
>>>>
>>>> +
>>>> +bool vgic_is_valid_irq(struct domain *d, unsigned int virq)
>>>
>>> I have the same comment as for the previous patch. This function
>>> completely ignores LPIs presence, while you can't argue that LPIs as
>>> valid. Again, function callers are expecting this behavior, so this is
>>> fine, but function name should better reflect its behavior.
>>>
>>> [...]
>>>
>>
>> Would it be okay to rename these functions as proposed in the previous
>> patch discussion:
>> vgic_is_valid_irq -> vgic_is_valid_line
>> vgic_is_shared_irq -> vgic_is_spi?
>>
>> Or, in the case of vgic, is it not a good idea to use the "line" suffix
>> because vgic does not have physical interrupt lines? Would it be better
>> to rename it to vgic_is_valid_non_lpi instead?
>
> I think it is better to follow the pGIC naming convention. While there
> is no physical IRQ lines in vGIC, it emulates real GIC anyways.
>
Okay, good point. I will rename the vGIC-related functions in the pGIC
manner in V3.
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v2 03/10] xen/arm: vgic: implement helper functions for virq checks
2025-08-07 12:33 ` [PATCH v2 03/10] xen/arm: vgic: implement helper functions for virq checks Leonid Komarianskyi
2025-08-21 15:46 ` Volodymyr Babchuk
@ 2025-08-23 12:29 ` Oleksandr Tyshchenko
2025-08-24 18:08 ` Leonid Komarianskyi
1 sibling, 1 reply; 55+ messages in thread
From: Oleksandr Tyshchenko @ 2025-08-23 12:29 UTC (permalink / raw)
To: xen-devel, Leonid Komarianskyi
On 07.08.25 15:33, Leonid Komarianskyi wrote:
Hello Leonid
> Introduced two new helper functions for vGIC: vgic_is_valid_irq and
> vgic_is_shared_irq. The functions are similar to the newly introduced
> gic_is_valid_irq and gic_is_shared_irq, but they verify whether a vIRQ
> is available for a specific domain, while GIC-specific functions
> validate INTIDs for the real GIC hardware. For example, the GIC may
> support all 992 SPI lines, but the domain may use only some part of them
> (e.g., 640), depending on the highest IRQ number defined in the domain
> configuration. Therefore, for vGIC-related code and checks, the
> appropriate functions should be used. Also, updated the appropriate
> checks to use these new helper functions.
>
> The purpose of introducing new helper functions for vGIC is essentially
> the same as for GIC: to avoid potential confusion with GIC-related
> checks and to consolidate similar code into separate functions, which
> can be more easily extended by additional conditions, e.g., when
> implementing extended SPI interrupts.
>
> Only the validation change in vgic_inject_irq may affect existing
> functionality, as it currently checks whether the vIRQ is less than or
> equal to vgic_num_irqs. Since IRQ indexes start from 0 (where 32 is the
> first SPI), the check should behave consistently with similar logic in
> other places and should check if the vIRQ number is less than
> vgic_num_irqs. The remaining changes, which replace open-coded checks
> with the use of these new helper functions, do not introduce any
> functional changes, as the helper functions follow the current vIRQ
> index verification logic.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>
> ---
> Changes in V2:
> - introduced this patch
> ---
> xen/arch/arm/gic.c | 3 +--
> xen/arch/arm/include/asm/vgic.h | 7 +++++++
> xen/arch/arm/irq.c | 4 ++--
> xen/arch/arm/vgic.c | 10 ++++++++--
> 4 files changed, 18 insertions(+), 6 deletions(-)
>
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index eb0346a898..47fccf21d8 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -133,8 +133,7 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq,
>
> ASSERT(spin_is_locked(&desc->lock));
> /* Caller has already checked that the IRQ is an SPI */
> - ASSERT(virq >= 32);
> - ASSERT(virq < vgic_num_irqs(d));
> + ASSERT(vgic_is_shared_irq(d, virq));
> ASSERT(!is_lpi(virq));
>
> ret = vgic_connect_hw_irq(d, NULL, virq, desc, true);
> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
> index 35c0c6a8b0..45201f4ca5 100644
> --- a/xen/arch/arm/include/asm/vgic.h
> +++ b/xen/arch/arm/include/asm/vgic.h
> @@ -335,6 +335,13 @@ extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
> /* Default number of vGIC SPIs. 32 are substracted to cover local IRQs. */
> #define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
>
> +extern bool vgic_is_valid_irq(struct domain *d, unsigned int virq);
> +
> +static inline bool vgic_is_shared_irq(struct domain *d, unsigned int virq)
> +{
> + return (virq >= NR_LOCAL_IRQS && vgic_is_valid_irq(d, virq));
> +}
> +
> /*
> * Allocate a guest VIRQ
> * - spi == 0 => allocate a PPI. It will be the same on every vCPU
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 12c70d02cc..50e57aaea7 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -442,7 +442,7 @@ int route_irq_to_guest(struct domain *d, unsigned int virq,
> unsigned long flags;
> int retval = 0;
>
> - if ( virq >= vgic_num_irqs(d) )
> + if ( !vgic_is_valid_irq(d, virq) )
This file is common for all VGIC implementations, so
route_irq_to_guest() is used with CONFIG_NEW_VGIC=y as well.
If your series is built with CONFIG_NEW_VGIC=y (I know, that NEW_VGIC
does not support GICV3 HW) I have got the following error:
aarch64-poky-linux-ld: prelink.o: in function `route_irq_to_guest':
/usr/src/debug/xen/4.18.0+gitAUTOINC+ce58f56108-r0/git/xen/arch/arm/irq.c:445:
undefined reference to `vgic_is_valid_irq'
/usr/src/debug/xen/4.18.0+gitAUTOINC+ce58f56108-r0/git/xen/arch/arm/irq.c:445:(.text+0x5e2f8):
relocation truncated to fit: R_AARCH64_CALL26 against undefined symbol
`vgic_is_valid_irq'
...
From the quick look, vgic_is_valid_irq() needs a stub (or NEW_VGIC's
counterpart).
[snip]
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 03/10] xen/arm: vgic: implement helper functions for virq checks
2025-08-23 12:29 ` Oleksandr Tyshchenko
@ 2025-08-24 18:08 ` Leonid Komarianskyi
0 siblings, 0 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-24 18:08 UTC (permalink / raw)
To: Oleksandr Tyshchenko, xen-devel@lists.xenproject.org
Hello Oleksandr,
On 23.08.25 15:29, Oleksandr Tyshchenko wrote:
> [You don't often get email from olekstysh@gmail.com. Learn why this is
> important at https://aka.ms/LearnAboutSenderIdentification ]
>
> On 07.08.25 15:33, Leonid Komarianskyi wrote:
>
> Hello Leonid
>
>
>> Introduced two new helper functions for vGIC: vgic_is_valid_irq and
>> vgic_is_shared_irq. The functions are similar to the newly introduced
>> gic_is_valid_irq and gic_is_shared_irq, but they verify whether a vIRQ
>> is available for a specific domain, while GIC-specific functions
>> validate INTIDs for the real GIC hardware. For example, the GIC may
>> support all 992 SPI lines, but the domain may use only some part of them
>> (e.g., 640), depending on the highest IRQ number defined in the domain
>> configuration. Therefore, for vGIC-related code and checks, the
>> appropriate functions should be used. Also, updated the appropriate
>> checks to use these new helper functions.
>>
>> The purpose of introducing new helper functions for vGIC is essentially
>> the same as for GIC: to avoid potential confusion with GIC-related
>> checks and to consolidate similar code into separate functions, which
>> can be more easily extended by additional conditions, e.g., when
>> implementing extended SPI interrupts.
>>
>> Only the validation change in vgic_inject_irq may affect existing
>> functionality, as it currently checks whether the vIRQ is less than or
>> equal to vgic_num_irqs. Since IRQ indexes start from 0 (where 32 is the
>> first SPI), the check should behave consistently with similar logic in
>> other places and should check if the vIRQ number is less than
>> vgic_num_irqs. The remaining changes, which replace open-coded checks
>> with the use of these new helper functions, do not introduce any
>> functional changes, as the helper functions follow the current vIRQ
>> index verification logic.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V2:
>> - introduced this patch
>> ---
>> xen/arch/arm/gic.c | 3 +--
>> xen/arch/arm/include/asm/vgic.h | 7 +++++++
>> xen/arch/arm/irq.c | 4 ++--
>> xen/arch/arm/vgic.c | 10 ++++++++--
>> 4 files changed, 18 insertions(+), 6 deletions(-)
>>
>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>> index eb0346a898..47fccf21d8 100644
>> --- a/xen/arch/arm/gic.c
>> +++ b/xen/arch/arm/gic.c
>> @@ -133,8 +133,7 @@ int gic_route_irq_to_guest(struct domain *d,
>> unsigned int virq,
>>
>> ASSERT(spin_is_locked(&desc->lock));
>> /* Caller has already checked that the IRQ is an SPI */
>> - ASSERT(virq >= 32);
>> - ASSERT(virq < vgic_num_irqs(d));
>> + ASSERT(vgic_is_shared_irq(d, virq));
>> ASSERT(!is_lpi(virq));
>>
>> ret = vgic_connect_hw_irq(d, NULL, virq, desc, true);
>> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/
>> asm/vgic.h
>> index 35c0c6a8b0..45201f4ca5 100644
>> --- a/xen/arch/arm/include/asm/vgic.h
>> +++ b/xen/arch/arm/include/asm/vgic.h
>> @@ -335,6 +335,13 @@ extern void
>> vgic_check_inflight_irqs_pending(struct vcpu *v,
>> /* Default number of vGIC SPIs. 32 are substracted to cover local
>> IRQs. */
>> #define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
>>
>> +extern bool vgic_is_valid_irq(struct domain *d, unsigned int virq);
>> +
>> +static inline bool vgic_is_shared_irq(struct domain *d, unsigned int
>> virq)
>> +{
>> + return (virq >= NR_LOCAL_IRQS && vgic_is_valid_irq(d, virq));
>> +}
>> +
>> /*
>> * Allocate a guest VIRQ
>> * - spi == 0 => allocate a PPI. It will be the same on every vCPU
>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>> index 12c70d02cc..50e57aaea7 100644
>> --- a/xen/arch/arm/irq.c
>> +++ b/xen/arch/arm/irq.c
>> @@ -442,7 +442,7 @@ int route_irq_to_guest(struct domain *d, unsigned
>> int virq,
>> unsigned long flags;
>> int retval = 0;
>>
>> - if ( virq >= vgic_num_irqs(d) )
>> + if ( !vgic_is_valid_irq(d, virq) )
>
>
> This file is common for all VGIC implementations, so
> route_irq_to_guest() is used with CONFIG_NEW_VGIC=y as well.
>
> If your series is built with CONFIG_NEW_VGIC=y (I know, that NEW_VGIC
> does not support GICV3 HW) I have got the following error:
>
>
> aarch64-poky-linux-ld: prelink.o: in function `route_irq_to_guest':
> /usr/src/debug/xen/4.18.0+gitAUTOINC+ce58f56108-r0/git/xen/arch/arm/
> irq.c:445:
> undefined reference to `vgic_is_valid_irq'
> /usr/src/debug/xen/4.18.0+gitAUTOINC+ce58f56108-r0/git/xen/arch/arm/
> irq.c:445:(.text+0x5e2f8):
> relocation truncated to fit: R_AARCH64_CALL26 against undefined symbol
> `vgic_is_valid_irq'
> ...
>
> From the quick look, vgic_is_valid_irq() needs a stub (or NEW_VGIC's
> counterpart).
>
> [snip]
Thank you for your review and for testing the patch series with
different config options. I will add the vgic_is_valid_irq counterpart
(after discussion, we decided to rename the function to
vgic_is_valid_line) for NEW_VGIC and recheck the build with NEW_VGIC config.
Best regards,
Leonid.
^ permalink raw reply [flat|nested] 55+ messages in thread
* [PATCH v2 02/10] xen/arm: gic: implement helper functions for INTID checks
2025-08-07 12:33 ` [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations Leonid Komarianskyi
2025-08-07 12:33 ` [PATCH v2 03/10] xen/arm: vgic: implement helper functions for virq checks Leonid Komarianskyi
@ 2025-08-07 12:33 ` Leonid Komarianskyi
2025-08-21 15:39 ` Volodymyr Babchuk
2025-08-07 12:33 ` [PATCH v2 05/10] xen/arm: gicv3: implement handling of GICv3.1 eSPI Leonid Komarianskyi
` (8 subsequent siblings)
10 siblings, 1 reply; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-07 12:33 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Leonid Komarianskyi, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk
Introduced two new helper functions: gic_is_valid_irq and
gic_is_shared_irq. The first function helps determine whether an IRQ
number is less than the number of lines supported by hardware. The
second function additionally checks if the IRQ number falls within the
SPI range. Also, updated the appropriate checks to use these new helper
functions.
The current checks for the real GIC are very similar to those for the
vGIC but serve a different purpose. For GIC-related code, the interrupt
numbers should be validated based on whether the hardware can operate
with such interrupts. On the other hand, for the vGIC, the indexes must
also be verified to ensure they are available for a specific domain. The
first reason for introducing these helper functions is to avoid
potential confusion with vGIC-related checks. The second reason is to
consolidate similar code into separate functions, which can be more
easily extended by additional conditions, e.g., when implementing
extended SPI interrupts.
The changes, which replace open-coded checks with the use of the new
helper functions, do not introduce any functional changes, as the helper
functions follow the current IRQ index verification logic.
Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
---
Changes in V2:
- introduced this patch
---
xen/arch/arm/gic.c | 2 +-
xen/arch/arm/include/asm/gic.h | 9 +++++++++
xen/arch/arm/irq.c | 2 +-
3 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index e80fe0ca24..eb0346a898 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -111,7 +111,7 @@ static void gic_set_irq_priority(struct irq_desc *desc, unsigned int priority)
void gic_route_irq_to_xen(struct irq_desc *desc, unsigned int priority)
{
ASSERT(priority <= 0xff); /* Only 8 bits of priority */
- ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
+ ASSERT(gic_is_valid_irq(desc->irq));/* Can't route interrupts that don't exist */
ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
ASSERT(spin_is_locked(&desc->lock));
diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/asm/gic.h
index 541f0eeb80..ac0b7b783e 100644
--- a/xen/arch/arm/include/asm/gic.h
+++ b/xen/arch/arm/include/asm/gic.h
@@ -306,6 +306,15 @@ extern void gic_dump_vgic_info(struct vcpu *v);
/* Number of interrupt lines */
extern unsigned int gic_number_lines(void);
+static inline bool gic_is_valid_irq(unsigned int irq)
+{
+ return irq < gic_number_lines();
+}
+
+static inline bool gic_is_shared_irq(unsigned int irq)
+{
+ return (irq >= NR_LOCAL_IRQS && gic_is_valid_irq(irq));
+}
/* IRQ translation function for the device tree */
int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 03fbb90c6c..12c70d02cc 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -415,7 +415,7 @@ err:
bool is_assignable_irq(unsigned int irq)
{
/* For now, we can only route SPIs to the guest */
- return (irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines());
+ return gic_is_shared_irq(irq);
}
/*
--
2.34.1
^ permalink raw reply related [flat|nested] 55+ messages in thread* Re: [PATCH v2 02/10] xen/arm: gic: implement helper functions for INTID checks
2025-08-07 12:33 ` [PATCH v2 02/10] xen/arm: gic: implement helper functions for INTID checks Leonid Komarianskyi
@ 2025-08-21 15:39 ` Volodymyr Babchuk
2025-08-21 16:24 ` Julien Grall
0 siblings, 1 reply; 55+ messages in thread
From: Volodymyr Babchuk @ 2025-08-21 15:39 UTC (permalink / raw)
To: Leonid Komarianskyi
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Hi Leonid,
Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
> Introduced two new helper functions: gic_is_valid_irq and
> gic_is_shared_irq. The first function helps determine whether an IRQ
> number is less than the number of lines supported by hardware. The
> second function additionally checks if the IRQ number falls within the
> SPI range. Also, updated the appropriate checks to use these new helper
> functions.
>
> The current checks for the real GIC are very similar to those for the
> vGIC but serve a different purpose. For GIC-related code, the interrupt
> numbers should be validated based on whether the hardware can operate
> with such interrupts. On the other hand, for the vGIC, the indexes must
> also be verified to ensure they are available for a specific domain. The
> first reason for introducing these helper functions is to avoid
> potential confusion with vGIC-related checks. The second reason is to
> consolidate similar code into separate functions, which can be more
> easily extended by additional conditions, e.g., when implementing
> extended SPI interrupts.
>
> The changes, which replace open-coded checks with the use of the new
> helper functions, do not introduce any functional changes, as the helper
> functions follow the current IRQ index verification logic.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>
> ---
> Changes in V2:
> - introduced this patch
> ---
> xen/arch/arm/gic.c | 2 +-
> xen/arch/arm/include/asm/gic.h | 9 +++++++++
> xen/arch/arm/irq.c | 2 +-
> 3 files changed, 11 insertions(+), 2 deletions(-)
>
> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
> index e80fe0ca24..eb0346a898 100644
> --- a/xen/arch/arm/gic.c
> +++ b/xen/arch/arm/gic.c
> @@ -111,7 +111,7 @@ static void gic_set_irq_priority(struct irq_desc *desc, unsigned int priority)
> void gic_route_irq_to_xen(struct irq_desc *desc, unsigned int priority)
> {
> ASSERT(priority <= 0xff); /* Only 8 bits of priority */
> - ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
> + ASSERT(gic_is_valid_irq(desc->irq));/* Can't route interrupts that don't exist */
> ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
> ASSERT(spin_is_locked(&desc->lock));
>
> diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/asm/gic.h
> index 541f0eeb80..ac0b7b783e 100644
> --- a/xen/arch/arm/include/asm/gic.h
> +++ b/xen/arch/arm/include/asm/gic.h
> @@ -306,6 +306,15 @@ extern void gic_dump_vgic_info(struct vcpu *v);
>
> /* Number of interrupt lines */
> extern unsigned int gic_number_lines(void);
> +static inline bool gic_is_valid_irq(unsigned int irq)
We need to do something about naming, because this function completely
ignores presence of LPIs. What I mean, that it will return "false" for
any LPI, while you can't argue that LPI is a valid interrupt :)
I understand that this is expected behavior by current callers, but the
function name is misleading.
Name like "gic_is_valid_non_lpi()" seems to mouthful, but it is the best
I can come up with.
> +{
> + return irq < gic_number_lines();
> +}
> +
> +static inline bool gic_is_shared_irq(unsigned int irq)
> +{
> + return (irq >= NR_LOCAL_IRQS && gic_is_valid_irq(irq));
Again, because of misleading name of gic_is_valid_irq() it may seem that
this function will return "true" for LPIs as well...
> +}
>
> /* IRQ translation function for the device tree */
> int gic_irq_xlate(const u32 *intspec, unsigned int intsize,
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 03fbb90c6c..12c70d02cc 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -415,7 +415,7 @@ err:
> bool is_assignable_irq(unsigned int irq)
> {
> /* For now, we can only route SPIs to the guest */
> - return (irq >= NR_LOCAL_IRQS) && (irq < gic_number_lines());
> + return gic_is_shared_irq(irq);
> }
>
> /*
--
WBR, Volodymyr
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 02/10] xen/arm: gic: implement helper functions for INTID checks
2025-08-21 15:39 ` Volodymyr Babchuk
@ 2025-08-21 16:24 ` Julien Grall
2025-08-22 7:30 ` Leonid Komarianskyi
0 siblings, 1 reply; 55+ messages in thread
From: Julien Grall @ 2025-08-21 16:24 UTC (permalink / raw)
To: Volodymyr Babchuk, Leonid Komarianskyi
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini,
Bertrand Marquis, Michal Orzel
Hi,
On 21/08/2025 16:39, Volodymyr Babchuk wrote:
> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>
>> Introduced two new helper functions: gic_is_valid_irq and
>> gic_is_shared_irq. The first function helps determine whether an IRQ
>> number is less than the number of lines supported by hardware. The
>> second function additionally checks if the IRQ number falls within the
>> SPI range. Also, updated the appropriate checks to use these new helper
>> functions.
>>
>> The current checks for the real GIC are very similar to those for the
>> vGIC but serve a different purpose. For GIC-related code, the interrupt
>> numbers should be validated based on whether the hardware can operate
>> with such interrupts. On the other hand, for the vGIC, the indexes must
>> also be verified to ensure they are available for a specific domain. The
>> first reason for introducing these helper functions is to avoid
>> potential confusion with vGIC-related checks. The second reason is to
>> consolidate similar code into separate functions, which can be more
>> easily extended by additional conditions, e.g., when implementing
>> extended SPI interrupts.
>>
>> The changes, which replace open-coded checks with the use of the new
>> helper functions, do not introduce any functional changes, as the helper
>> functions follow the current IRQ index verification logic.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V2:
>> - introduced this patch
>> ---
>> xen/arch/arm/gic.c | 2 +-
>> xen/arch/arm/include/asm/gic.h | 9 +++++++++
>> xen/arch/arm/irq.c | 2 +-
>> 3 files changed, 11 insertions(+), 2 deletions(-)
>>
>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>> index e80fe0ca24..eb0346a898 100644
>> --- a/xen/arch/arm/gic.c
>> +++ b/xen/arch/arm/gic.c
>> @@ -111,7 +111,7 @@ static void gic_set_irq_priority(struct irq_desc *desc, unsigned int priority)
>> void gic_route_irq_to_xen(struct irq_desc *desc, unsigned int priority)
>> {
>> ASSERT(priority <= 0xff); /* Only 8 bits of priority */
>> - ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts that don't exist */
>> + ASSERT(gic_is_valid_irq(desc->irq));/* Can't route interrupts that don't exist */
>> ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
>> ASSERT(spin_is_locked(&desc->lock));
>>
>> diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/asm/gic.h
>> index 541f0eeb80..ac0b7b783e 100644
>> --- a/xen/arch/arm/include/asm/gic.h
>> +++ b/xen/arch/arm/include/asm/gic.h
>> @@ -306,6 +306,15 @@ extern void gic_dump_vgic_info(struct vcpu *v);
>>
>> /* Number of interrupt lines */
>> extern unsigned int gic_number_lines(void);
>> +static inline bool gic_is_valid_irq(unsigned int irq)
>
> We need to do something about naming, because this function completely
> ignores presence of LPIs. What I mean, that it will return "false" for
> any LPI, while you can't argue that LPI is a valid interrupt :)
> I understand that this is expected behavior by current callers, but the
> function name is misleading.
>
> Name like "gic_is_valid_non_lpi()" seems to mouthful, but it is the best
> I can come up with.
AFAIU, there is no interrupt lines for LPIs. So what about
gic_is_valid_line()?
>
>> +{
>> + return irq < gic_number_lines();
>> +}
>> +
>> +static inline bool gic_is_shared_irq(unsigned int irq)
>> +{
>> + return (irq >= NR_LOCAL_IRQS && gic_is_valid_irq(irq));
>
> Again, because of misleading name of gic_is_valid_irq() it may seem that
> this function will return "true" for LPIs as well...
Even if we rename gic_is_valid_irq(), the function name would be
misleading because LPIs are shared. I think it would be better named
gic_is_spi(...);
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 02/10] xen/arm: gic: implement helper functions for INTID checks
2025-08-21 16:24 ` Julien Grall
@ 2025-08-22 7:30 ` Leonid Komarianskyi
0 siblings, 0 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-22 7:30 UTC (permalink / raw)
To: Julien Grall, Volodymyr Babchuk
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini,
Bertrand Marquis, Michal Orzel
Hi Volodymyr and Julien,
Thank you for your comments and for your time.
On 21.08.25 19:24, Julien Grall wrote:
> Hi,
>
> On 21/08/2025 16:39, Volodymyr Babchuk wrote:
>> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>>
>>> Introduced two new helper functions: gic_is_valid_irq and
>>> gic_is_shared_irq. The first function helps determine whether an IRQ
>>> number is less than the number of lines supported by hardware. The
>>> second function additionally checks if the IRQ number falls within the
>>> SPI range. Also, updated the appropriate checks to use these new helper
>>> functions.
>>>
>>> The current checks for the real GIC are very similar to those for the
>>> vGIC but serve a different purpose. For GIC-related code, the interrupt
>>> numbers should be validated based on whether the hardware can operate
>>> with such interrupts. On the other hand, for the vGIC, the indexes must
>>> also be verified to ensure they are available for a specific domain. The
>>> first reason for introducing these helper functions is to avoid
>>> potential confusion with vGIC-related checks. The second reason is to
>>> consolidate similar code into separate functions, which can be more
>>> easily extended by additional conditions, e.g., when implementing
>>> extended SPI interrupts.
>>>
>>> The changes, which replace open-coded checks with the use of the new
>>> helper functions, do not introduce any functional changes, as the helper
>>> functions follow the current IRQ index verification logic.
>>>
>>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>>
>>> ---
>>> Changes in V2:
>>> - introduced this patch
>>> ---
>>> xen/arch/arm/gic.c | 2 +-
>>> xen/arch/arm/include/asm/gic.h | 9 +++++++++
>>> xen/arch/arm/irq.c | 2 +-
>>> 3 files changed, 11 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
>>> index e80fe0ca24..eb0346a898 100644
>>> --- a/xen/arch/arm/gic.c
>>> +++ b/xen/arch/arm/gic.c
>>> @@ -111,7 +111,7 @@ static void gic_set_irq_priority(struct irq_desc
>>> *desc, unsigned int priority)
>>> void gic_route_irq_to_xen(struct irq_desc *desc, unsigned int
>>> priority)
>>> {
>>> ASSERT(priority <= 0xff); /* Only 8 bits of priority */
>>> - ASSERT(desc->irq < gic_number_lines());/* Can't route interrupts
>>> that don't exist */
>>> + ASSERT(gic_is_valid_irq(desc->irq));/* Can't route interrupts
>>> that don't exist */
>>> ASSERT(test_bit(_IRQ_DISABLED, &desc->status));
>>> ASSERT(spin_is_locked(&desc->lock));
>>> diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/
>>> asm/gic.h
>>> index 541f0eeb80..ac0b7b783e 100644
>>> --- a/xen/arch/arm/include/asm/gic.h
>>> +++ b/xen/arch/arm/include/asm/gic.h
>>> @@ -306,6 +306,15 @@ extern void gic_dump_vgic_info(struct vcpu *v);
>>> /* Number of interrupt lines */
>>> extern unsigned int gic_number_lines(void);
>>> +static inline bool gic_is_valid_irq(unsigned int irq)
>>
>> We need to do something about naming, because this function completely
>> ignores presence of LPIs. What I mean, that it will return "false" for
>> any LPI, while you can't argue that LPI is a valid interrupt :)
>> I understand that this is expected behavior by current callers, but the
>> function name is misleading.
>>
>> Name like "gic_is_valid_non_lpi()" seems to mouthful, but it is the best
>> I can come up with.
>
> AFAIU, there is no interrupt lines for LPIs. So what about
> gic_is_valid_line()?
Oh, thank you. It would be much better to name the function
gic_is_valid_line(), so I will rename it in V3.
>>
>>> +{
>>> + return irq < gic_number_lines();
>>> +}
>>> +
>>> +static inline bool gic_is_shared_irq(unsigned int irq)
>>> +{
>>> + return (irq >= NR_LOCAL_IRQS && gic_is_valid_irq(irq));
>>
>> Again, because of misleading name of gic_is_valid_irq() it may seem that
>> this function will return "true" for LPIs as well...
>
> Even if we rename gic_is_valid_irq(), the function name would be
> misleading because LPIs are shared. I think it would be better named
> gic_is_spi(...);
>
> Cheers,
>
Okay, I will rename gic_is_shared_irq to gic_is_spi in V3.
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread
* [PATCH v2 05/10] xen/arm: gicv3: implement handling of GICv3.1 eSPI
2025-08-07 12:33 ` [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations Leonid Komarianskyi
2025-08-07 12:33 ` [PATCH v2 03/10] xen/arm: vgic: implement helper functions for virq checks Leonid Komarianskyi
2025-08-07 12:33 ` [PATCH v2 02/10] xen/arm: gic: implement helper functions for INTID checks Leonid Komarianskyi
@ 2025-08-07 12:33 ` Leonid Komarianskyi
2025-08-21 16:16 ` Volodymyr Babchuk
2025-08-23 14:23 ` Oleksandr Tyshchenko
2025-08-07 12:33 ` [PATCH v2 04/10] xen/arm/irq: add handling for IRQs in the eSPI range Leonid Komarianskyi
` (7 subsequent siblings)
10 siblings, 2 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-07 12:33 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Leonid Komarianskyi, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk
Introduced appropriate register definitions, helper macros,
and initialization of required GICv3.1 distributor registers
to support eSPI. This type of interrupt is handled in the
same way as regular SPI interrupts, with the following
differences:
1) eSPIs can have up to 1024 interrupts, starting from the
beginning of the range, whereas regular SPIs use INTIDs from
32 to 1019, totaling 988 interrupts;
2) eSPIs start at INTID 4096, necessitating additional interrupt
index conversion during register operations.
In case if appropriate config is disabled, or GIC HW doesn't
support eSPI, the existing functionality will remain the same.
Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
---
Changes in V2:
- move gic_number_espis function from
[PATCH 08/10] xen/arm: vgic: add resource management for extended SPIs
to use it in the newly introduced gic_is_valid_espi
- add gic_is_valid_espi which checks if IRQ number is in supported
by HW eSPI range
- update gic_is_valid_irq conditions to allow operations with eSPIs
---
xen/arch/arm/gic-v3.c | 73 ++++++++++++++++++++++++++
xen/arch/arm/include/asm/gic.h | 17 ++++++
xen/arch/arm/include/asm/gic_v3_defs.h | 33 ++++++++++++
3 files changed, 123 insertions(+)
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 8fd78aba44..a0e8ee1a1e 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -485,6 +485,36 @@ static void __iomem *get_addr_by_offset(struct irq_desc *irqd, u32 offset)
default:
break;
}
+#ifdef CONFIG_GICV3_ESPI
+ case ESPI_BASE_INTID ... ESPI_MAX_INTID:
+ {
+ u32 irq_index = ESPI_INTID2IDX(irqd->irq);
+
+ switch ( offset )
+ {
+ case GICD_ISENABLER:
+ return (GICD + GICD_ISENABLERnE + (irq_index / 32) * 4);
+ case GICD_ICENABLER:
+ return (GICD + GICD_ICENABLERnE + (irq_index / 32) * 4);
+ case GICD_ISPENDR:
+ return (GICD + GICD_ISPENDRnE + (irq_index / 32) * 4);
+ case GICD_ICPENDR:
+ return (GICD + GICD_ICPENDRnE + (irq_index / 32) * 4);
+ case GICD_ISACTIVER:
+ return (GICD + GICD_ISACTIVERnE + (irq_index / 32) * 4);
+ case GICD_ICACTIVER:
+ return (GICD + GICD_ICACTIVERnE + (irq_index / 32) * 4);
+ case GICD_ICFGR:
+ return (GICD + GICD_ICFGRnE + (irq_index / 16) * 4);
+ case GICD_IROUTER:
+ return (GICD + GICD_IROUTERnE + irq_index * 8);
+ case GICD_IPRIORITYR:
+ return (GICD + GICD_IPRIORITYRnE + irq_index);
+ default:
+ break;
+ }
+ }
+#endif
default:
break;
}
@@ -645,6 +675,40 @@ static void gicv3_set_irq_priority(struct irq_desc *desc,
spin_unlock(&gicv3.lock);
}
+#ifdef CONFIG_GICV3_ESPI
+unsigned int gic_number_espis(void)
+{
+ return gic_hw_ops->info->nr_espi;
+}
+
+static void gicv3_dist_espi_common_init(uint32_t type)
+{
+ unsigned int espi_nr;
+ int i;
+
+ espi_nr = min(1024U, GICD_TYPER_ESPIS_NUM(type));
+ gicv3_info.nr_espi = espi_nr;
+ /* The GIC HW doesn't support eSPI, so we can leave from here */
+ if ( gicv3_info.nr_espi == 0 )
+ return;
+
+ for ( i = 0; i < espi_nr; i += 16 )
+ writel_relaxed(0, GICD + GICD_ICFGRnE + (i / 16) * 4);
+
+ for ( i = 0; i < espi_nr; i += 4 )
+ writel_relaxed(GIC_PRI_IRQ_ALL, GICD + GICD_IPRIORITYRnE + (i / 4) * 4);
+
+ for ( i = 0; i < espi_nr; i += 32 )
+ {
+ writel_relaxed(0xffffffffU, GICD + GICD_ICENABLERnE + (i / 32) * 4);
+ writel_relaxed(0xffffffffU, GICD + GICD_ICACTIVERnE + (i / 32) * 4);
+ }
+
+ for ( i = 0; i < espi_nr; i += 32 )
+ writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPRnE + (i / 32) * 4);
+}
+#endif
+
static void __init gicv3_dist_init(void)
{
uint32_t type;
@@ -690,6 +754,10 @@ static void __init gicv3_dist_init(void)
for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i += 32 )
writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPR + (i / 32) * 4);
+#ifdef CONFIG_GICV3_ESPI
+ gicv3_dist_espi_common_init(type);
+#endif
+
gicv3_dist_wait_for_rwp();
/* Turn on the distributor */
@@ -703,6 +771,11 @@ static void __init gicv3_dist_init(void)
for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i++ )
writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTER + i * 8);
+
+#ifdef CONFIG_GICV3_ESPI
+ for ( i = 0; i < gicv3_info.nr_espi; i++ )
+ writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTERnE + i * 8);
+#endif
}
static int gicv3_enable_redist(void)
diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/asm/gic.h
index ac0b7b783e..2f570abf70 100644
--- a/xen/arch/arm/include/asm/gic.h
+++ b/xen/arch/arm/include/asm/gic.h
@@ -306,8 +306,21 @@ extern void gic_dump_vgic_info(struct vcpu *v);
/* Number of interrupt lines */
extern unsigned int gic_number_lines(void);
+#ifdef CONFIG_GICV3_ESPI
+extern unsigned int gic_number_espis(void);
+
+static inline bool gic_is_valid_espi(unsigned int irq)
+{
+ return (irq >= ESPI_BASE_INTID && irq < ESPI_IDX2INTID(gic_number_espis()));
+}
+#endif
+
static inline bool gic_is_valid_irq(unsigned int irq)
{
+#ifdef CONFIG_GICV3_ESPI
+ if ( gic_is_valid_espi(irq) )
+ return true;
+#endif
return irq < gic_number_lines();
}
@@ -325,6 +338,10 @@ struct gic_info {
enum gic_version hw_version;
/* Number of GIC lines supported */
unsigned int nr_lines;
+#ifdef CONFIG_GICV3_ESPI
+ /* Number of GIC eSPI supported */
+ unsigned int nr_espi;
+#endif
/* Number of LR registers */
uint8_t nr_lrs;
/* Maintenance irq number */
diff --git a/xen/arch/arm/include/asm/gic_v3_defs.h b/xen/arch/arm/include/asm/gic_v3_defs.h
index 2af093e774..7f769b38e3 100644
--- a/xen/arch/arm/include/asm/gic_v3_defs.h
+++ b/xen/arch/arm/include/asm/gic_v3_defs.h
@@ -37,6 +37,39 @@
#define GICD_IROUTER1019 (0x7FD8)
#define GICD_PIDR2 (0xFFE8)
+#ifdef CONFIG_GICV3_ESPI
+/* Additional registers for GICv3.1 */
+#define GICD_IGROUPRnE (0x1000)
+#define GICD_IGROUPRnEN (0x107C)
+#define GICD_ISENABLERnE (0x1200)
+#define GICD_ISENABLERnEN (0x127C)
+#define GICD_ICENABLERnE (0x1400)
+#define GICD_ICENABLERnEN (0x147C)
+#define GICD_ISPENDRnE (0x1600)
+#define GICD_ISPENDRnEN (0x167C)
+#define GICD_ICPENDRnE (0x1800)
+#define GICD_ICPENDRnEN (0x187C)
+#define GICD_ISACTIVERnE (0x1A00)
+#define GICD_ISACTIVERnEN (0x1A7C)
+#define GICD_ICACTIVERnE (0x1C00)
+#define GICD_ICACTIVERnEN (0x1C7C)
+#define GICD_IPRIORITYRnE (0x2000)
+#define GICD_IPRIORITYRnEN (0x23FC)
+#define GICD_ICFGRnE (0x3000)
+#define GICD_ICFGRnEN (0x30FC)
+#define GICD_IROUTERnE (0x8000)
+#define GICD_IROUTERnEN (0x9FFC)
+
+#define GICD_TYPER_ESPI_SHIFT 8
+#define GICD_TYPER_ESPI_RANGE_SHIFT 27
+#define GICD_TYPER_ESPI_RANGE_MASK (0x1F)
+#define GICD_TYPER_ESPI (1U << GICD_TYPER_ESPI_SHIFT)
+#define GICD_TYPER_ESPI_RANGE(typer) ((((typer) & GICD_TYPER_ESPI_RANGE_MASK) + 1) * 32)
+#define GICD_TYPER_ESPIS_NUM(typer) \
+ (((typer) & GICD_TYPER_ESPI) ? \
+ GICD_TYPER_ESPI_RANGE((typer) >> GICD_TYPER_ESPI_RANGE_SHIFT) : 0)
+#endif
+
/* Common between GICD_PIDR2 and GICR_PIDR2 */
#define GIC_PIDR2_ARCH_MASK (0xf0)
#define GIC_PIDR2_ARCH_GICv3 (0x30)
--
2.34.1
^ permalink raw reply related [flat|nested] 55+ messages in thread* Re: [PATCH v2 05/10] xen/arm: gicv3: implement handling of GICv3.1 eSPI
2025-08-07 12:33 ` [PATCH v2 05/10] xen/arm: gicv3: implement handling of GICv3.1 eSPI Leonid Komarianskyi
@ 2025-08-21 16:16 ` Volodymyr Babchuk
2025-08-22 14:39 ` Leonid Komarianskyi
2025-08-23 14:23 ` Oleksandr Tyshchenko
1 sibling, 1 reply; 55+ messages in thread
From: Volodymyr Babchuk @ 2025-08-21 16:16 UTC (permalink / raw)
To: Leonid Komarianskyi
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Hi,
Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
> Introduced appropriate register definitions, helper macros,
> and initialization of required GICv3.1 distributor registers
> to support eSPI. This type of interrupt is handled in the
> same way as regular SPI interrupts, with the following
> differences:
>
> 1) eSPIs can have up to 1024 interrupts, starting from the
> beginning of the range, whereas regular SPIs use INTIDs from
> 32 to 1019, totaling 988 interrupts;
> 2) eSPIs start at INTID 4096, necessitating additional interrupt
> index conversion during register operations.
>
> In case if appropriate config is disabled, or GIC HW doesn't
> support eSPI, the existing functionality will remain the same.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>
> ---
> Changes in V2:
> - move gic_number_espis function from
> [PATCH 08/10] xen/arm: vgic: add resource management for extended SPIs
> to use it in the newly introduced gic_is_valid_espi
> - add gic_is_valid_espi which checks if IRQ number is in supported
> by HW eSPI range
> - update gic_is_valid_irq conditions to allow operations with eSPIs
> ---
> xen/arch/arm/gic-v3.c | 73 ++++++++++++++++++++++++++
> xen/arch/arm/include/asm/gic.h | 17 ++++++
> xen/arch/arm/include/asm/gic_v3_defs.h | 33 ++++++++++++
> 3 files changed, 123 insertions(+)
>
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 8fd78aba44..a0e8ee1a1e 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -485,6 +485,36 @@ static void __iomem *get_addr_by_offset(struct irq_desc *irqd, u32 offset)
> default:
> break;
> }
> +#ifdef CONFIG_GICV3_ESPI
> + case ESPI_BASE_INTID ... ESPI_MAX_INTID:
> + {
> + u32 irq_index = ESPI_INTID2IDX(irqd->irq);
> +
> + switch ( offset )
> + {
> + case GICD_ISENABLER:
> + return (GICD + GICD_ISENABLERnE + (irq_index / 32) * 4);
> + case GICD_ICENABLER:
> + return (GICD + GICD_ICENABLERnE + (irq_index / 32) * 4);
> + case GICD_ISPENDR:
> + return (GICD + GICD_ISPENDRnE + (irq_index / 32) * 4);
> + case GICD_ICPENDR:
> + return (GICD + GICD_ICPENDRnE + (irq_index / 32) * 4);
> + case GICD_ISACTIVER:
> + return (GICD + GICD_ISACTIVERnE + (irq_index / 32) * 4);
> + case GICD_ICACTIVER:
> + return (GICD + GICD_ICACTIVERnE + (irq_index / 32) * 4);
> + case GICD_ICFGR:
> + return (GICD + GICD_ICFGRnE + (irq_index / 16) * 4);
> + case GICD_IROUTER:
> + return (GICD + GICD_IROUTERnE + irq_index * 8);
> + case GICD_IPRIORITYR:
> + return (GICD + GICD_IPRIORITYRnE + irq_index);
> + default:
> + break;
> + }
> + }
> +#endif
> default:
> break;
> }
> @@ -645,6 +675,40 @@ static void gicv3_set_irq_priority(struct irq_desc *desc,
> spin_unlock(&gicv3.lock);
> }
>
> +#ifdef CONFIG_GICV3_ESPI
> +unsigned int gic_number_espis(void)
> +{
> + return gic_hw_ops->info->nr_espi;
> +}
> +
> +static void gicv3_dist_espi_common_init(uint32_t type)
> +{
> + unsigned int espi_nr;
> + int i;
> +
> + espi_nr = min(1024U, GICD_TYPER_ESPIS_NUM(type));
> + gicv3_info.nr_espi = espi_nr;
> + /* The GIC HW doesn't support eSPI, so we can leave from here */
> + if ( gicv3_info.nr_espi == 0 )
> + return;
> +
> + for ( i = 0; i < espi_nr; i += 16 )
> + writel_relaxed(0, GICD + GICD_ICFGRnE + (i / 16) * 4);
> +
> + for ( i = 0; i < espi_nr; i += 4 )
> + writel_relaxed(GIC_PRI_IRQ_ALL, GICD + GICD_IPRIORITYRnE + (i / 4) * 4);
> +
> + for ( i = 0; i < espi_nr; i += 32 )
> + {
> + writel_relaxed(0xffffffffU, GICD + GICD_ICENABLERnE + (i / 32) * 4);
Is there are particular reason why you use GENMASK(31,0) below, but
open-coded 0xffffffff here?
> + writel_relaxed(0xffffffffU, GICD + GICD_ICACTIVERnE + (i / 32) * 4);
... and here?
> + }
> +
> + for ( i = 0; i < espi_nr; i += 32 )
> + writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPRnE + (i / 32) * 4);
> +}
> +#endif
> +
> static void __init gicv3_dist_init(void)
> {
> uint32_t type;
> @@ -690,6 +754,10 @@ static void __init gicv3_dist_init(void)
> for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i += 32 )
> writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPR + (i / 32) * 4);
>
> +#ifdef CONFIG_GICV3_ESPI
> + gicv3_dist_espi_common_init(type);
> +#endif
> +
> gicv3_dist_wait_for_rwp();
>
> /* Turn on the distributor */
> @@ -703,6 +771,11 @@ static void __init gicv3_dist_init(void)
>
> for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i++ )
> writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTER + i * 8);
> +
> +#ifdef CONFIG_GICV3_ESPI
> + for ( i = 0; i < gicv3_info.nr_espi; i++ )
> + writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTERnE + i * 8);
> +#endif
> }
>
> static int gicv3_enable_redist(void)
> diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/asm/gic.h
> index ac0b7b783e..2f570abf70 100644
> --- a/xen/arch/arm/include/asm/gic.h
> +++ b/xen/arch/arm/include/asm/gic.h
> @@ -306,8 +306,21 @@ extern void gic_dump_vgic_info(struct vcpu *v);
>
> /* Number of interrupt lines */
> extern unsigned int gic_number_lines(void);
> +#ifdef CONFIG_GICV3_ESPI
> +extern unsigned int gic_number_espis(void);
> +
> +static inline bool gic_is_valid_espi(unsigned int irq)
> +{
> + return (irq >= ESPI_BASE_INTID && irq < ESPI_IDX2INTID(gic_number_espis()));
> +}
> +#endif
> +
> static inline bool gic_is_valid_irq(unsigned int irq)
> {
> +#ifdef CONFIG_GICV3_ESPI
> + if ( gic_is_valid_espi(irq) )
> + return true;
> +#endif
> return irq < gic_number_lines();
> }
>
> @@ -325,6 +338,10 @@ struct gic_info {
> enum gic_version hw_version;
> /* Number of GIC lines supported */
> unsigned int nr_lines;
> +#ifdef CONFIG_GICV3_ESPI
> + /* Number of GIC eSPI supported */
> + unsigned int nr_espi;
> +#endif
> /* Number of LR registers */
> uint8_t nr_lrs;
> /* Maintenance irq number */
> diff --git a/xen/arch/arm/include/asm/gic_v3_defs.h b/xen/arch/arm/include/asm/gic_v3_defs.h
> index 2af093e774..7f769b38e3 100644
> --- a/xen/arch/arm/include/asm/gic_v3_defs.h
> +++ b/xen/arch/arm/include/asm/gic_v3_defs.h
> @@ -37,6 +37,39 @@
> #define GICD_IROUTER1019 (0x7FD8)
> #define GICD_PIDR2 (0xFFE8)
>
> +#ifdef CONFIG_GICV3_ESPI
> +/* Additional registers for GICv3.1 */
> +#define GICD_IGROUPRnE (0x1000)
> +#define GICD_IGROUPRnEN (0x107C)
> +#define GICD_ISENABLERnE (0x1200)
> +#define GICD_ISENABLERnEN (0x127C)
> +#define GICD_ICENABLERnE (0x1400)
> +#define GICD_ICENABLERnEN (0x147C)
> +#define GICD_ISPENDRnE (0x1600)
> +#define GICD_ISPENDRnEN (0x167C)
> +#define GICD_ICPENDRnE (0x1800)
> +#define GICD_ICPENDRnEN (0x187C)
> +#define GICD_ISACTIVERnE (0x1A00)
> +#define GICD_ISACTIVERnEN (0x1A7C)
> +#define GICD_ICACTIVERnE (0x1C00)
> +#define GICD_ICACTIVERnEN (0x1C7C)
> +#define GICD_IPRIORITYRnE (0x2000)
> +#define GICD_IPRIORITYRnEN (0x23FC)
> +#define GICD_ICFGRnE (0x3000)
> +#define GICD_ICFGRnEN (0x30FC)
> +#define GICD_IROUTERnE (0x8000)
> +#define GICD_IROUTERnEN (0x9FFC)
> +
> +#define GICD_TYPER_ESPI_SHIFT 8
> +#define GICD_TYPER_ESPI_RANGE_SHIFT 27
> +#define GICD_TYPER_ESPI_RANGE_MASK (0x1F)
> +#define GICD_TYPER_ESPI (1U << GICD_TYPER_ESPI_SHIFT)
> +#define GICD_TYPER_ESPI_RANGE(typer) ((((typer) & GICD_TYPER_ESPI_RANGE_MASK) + 1) * 32)
Isn't this line a bit long?
> +#define GICD_TYPER_ESPIS_NUM(typer) \
> + (((typer) & GICD_TYPER_ESPI) ? \
> + GICD_TYPER_ESPI_RANGE((typer) >> GICD_TYPER_ESPI_RANGE_SHIFT) : 0)
I am not sure that this is correct.
Probably you wanted to write
+ GICD_TYPER_ESPI_RANGE((typer >> GICD_TYPER_ESPI_RANGE_SHIFT)) : 0)
> +#endif
> +
> /* Common between GICD_PIDR2 and GICR_PIDR2 */
> #define GIC_PIDR2_ARCH_MASK (0xf0)
> #define GIC_PIDR2_ARCH_GICv3 (0x30)
--
WBR, Volodymyr
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 05/10] xen/arm: gicv3: implement handling of GICv3.1 eSPI
2025-08-21 16:16 ` Volodymyr Babchuk
@ 2025-08-22 14:39 ` Leonid Komarianskyi
0 siblings, 0 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-22 14:39 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Hi Volodymyr,
Thank you for your close review.
On 21.08.25 19:16, Volodymyr Babchuk wrote:
>
> Hi,
>
> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>
>> Introduced appropriate register definitions, helper macros,
>> and initialization of required GICv3.1 distributor registers
>> to support eSPI. This type of interrupt is handled in the
>> same way as regular SPI interrupts, with the following
>> differences:
>>
>> 1) eSPIs can have up to 1024 interrupts, starting from the
>> beginning of the range, whereas regular SPIs use INTIDs from
>> 32 to 1019, totaling 988 interrupts;
>> 2) eSPIs start at INTID 4096, necessitating additional interrupt
>> index conversion during register operations.
>>
>> In case if appropriate config is disabled, or GIC HW doesn't
>> support eSPI, the existing functionality will remain the same.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V2:
>> - move gic_number_espis function from
>> [PATCH 08/10] xen/arm: vgic: add resource management for extended SPIs
>> to use it in the newly introduced gic_is_valid_espi
>> - add gic_is_valid_espi which checks if IRQ number is in supported
>> by HW eSPI range
>> - update gic_is_valid_irq conditions to allow operations with eSPIs
>> ---
>> xen/arch/arm/gic-v3.c | 73 ++++++++++++++++++++++++++
>> xen/arch/arm/include/asm/gic.h | 17 ++++++
>> xen/arch/arm/include/asm/gic_v3_defs.h | 33 ++++++++++++
>> 3 files changed, 123 insertions(+)
>>
>> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
>> index 8fd78aba44..a0e8ee1a1e 100644
>> --- a/xen/arch/arm/gic-v3.c
>> +++ b/xen/arch/arm/gic-v3.c
>> @@ -485,6 +485,36 @@ static void __iomem *get_addr_by_offset(struct irq_desc *irqd, u32 offset)
>> default:
>> break;
>> }
>> +#ifdef CONFIG_GICV3_ESPI
>> + case ESPI_BASE_INTID ... ESPI_MAX_INTID:
>> + {
>> + u32 irq_index = ESPI_INTID2IDX(irqd->irq);
>> +
>> + switch ( offset )
>> + {
>> + case GICD_ISENABLER:
>> + return (GICD + GICD_ISENABLERnE + (irq_index / 32) * 4);
>> + case GICD_ICENABLER:
>> + return (GICD + GICD_ICENABLERnE + (irq_index / 32) * 4);
>> + case GICD_ISPENDR:
>> + return (GICD + GICD_ISPENDRnE + (irq_index / 32) * 4);
>> + case GICD_ICPENDR:
>> + return (GICD + GICD_ICPENDRnE + (irq_index / 32) * 4);
>> + case GICD_ISACTIVER:
>> + return (GICD + GICD_ISACTIVERnE + (irq_index / 32) * 4);
>> + case GICD_ICACTIVER:
>> + return (GICD + GICD_ICACTIVERnE + (irq_index / 32) * 4);
>> + case GICD_ICFGR:
>> + return (GICD + GICD_ICFGRnE + (irq_index / 16) * 4);
>> + case GICD_IROUTER:
>> + return (GICD + GICD_IROUTERnE + irq_index * 8);
>> + case GICD_IPRIORITYR:
>> + return (GICD + GICD_IPRIORITYRnE + irq_index);
>> + default:
>> + break;
>> + }
>> + }
>> +#endif
>> default:
>> break;
>> }
>> @@ -645,6 +675,40 @@ static void gicv3_set_irq_priority(struct irq_desc *desc,
>> spin_unlock(&gicv3.lock);
>> }
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> +unsigned int gic_number_espis(void)
>> +{
>> + return gic_hw_ops->info->nr_espi;
>> +}
>> +
>> +static void gicv3_dist_espi_common_init(uint32_t type)
>> +{
>> + unsigned int espi_nr;
>> + int i;
>> +
>> + espi_nr = min(1024U, GICD_TYPER_ESPIS_NUM(type));
>> + gicv3_info.nr_espi = espi_nr;
>> + /* The GIC HW doesn't support eSPI, so we can leave from here */
>> + if ( gicv3_info.nr_espi == 0 )
>> + return;
>> +
>> + for ( i = 0; i < espi_nr; i += 16 )
>> + writel_relaxed(0, GICD + GICD_ICFGRnE + (i / 16) * 4);
>> +
>> + for ( i = 0; i < espi_nr; i += 4 )
>> + writel_relaxed(GIC_PRI_IRQ_ALL, GICD + GICD_IPRIORITYRnE + (i / 4) * 4);
>> +
>> + for ( i = 0; i < espi_nr; i += 32 )
>> + {
>> + writel_relaxed(0xffffffffU, GICD + GICD_ICENABLERnE + (i / 32) * 4);
>
> Is there are particular reason why you use GENMASK(31,0) below, but
> open-coded 0xffffffff here?
>
>> + writel_relaxed(0xffffffffU, GICD + GICD_ICACTIVERnE + (i / 32) * 4);
>
> ... and here?
>
No, there is no particular reason to use open-code here. It is just a
copy-paste from the code for regular SPI initialization. Thus, I agree
it is much better to use GENMASK(31, 0). I will change this in V3.
>> + }
>> +
>> + for ( i = 0; i < espi_nr; i += 32 )
>> + writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPRnE + (i / 32) * 4);
>> +}
>> +#endif
>> +
>> static void __init gicv3_dist_init(void)
>> {
>> uint32_t type;
>> @@ -690,6 +754,10 @@ static void __init gicv3_dist_init(void)
>> for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i += 32 )
>> writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPR + (i / 32) * 4);
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> + gicv3_dist_espi_common_init(type);
>> +#endif
>> +
>> gicv3_dist_wait_for_rwp();
>>
>> /* Turn on the distributor */
>> @@ -703,6 +771,11 @@ static void __init gicv3_dist_init(void)
>>
>> for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i++ )
>> writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTER + i * 8);
>> +
>> +#ifdef CONFIG_GICV3_ESPI
>> + for ( i = 0; i < gicv3_info.nr_espi; i++ )
>> + writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTERnE + i * 8);
>> +#endif
>> }
>>
>> static int gicv3_enable_redist(void)
>> diff --git a/xen/arch/arm/include/asm/gic.h b/xen/arch/arm/include/asm/gic.h
>> index ac0b7b783e..2f570abf70 100644
>> --- a/xen/arch/arm/include/asm/gic.h
>> +++ b/xen/arch/arm/include/asm/gic.h
>> @@ -306,8 +306,21 @@ extern void gic_dump_vgic_info(struct vcpu *v);
>>
>> /* Number of interrupt lines */
>> extern unsigned int gic_number_lines(void);
>> +#ifdef CONFIG_GICV3_ESPI
>> +extern unsigned int gic_number_espis(void);
>> +
>> +static inline bool gic_is_valid_espi(unsigned int irq)
>> +{
>> + return (irq >= ESPI_BASE_INTID && irq < ESPI_IDX2INTID(gic_number_espis()));
>> +}
>> +#endif
>> +
>> static inline bool gic_is_valid_irq(unsigned int irq)
>> {
>> +#ifdef CONFIG_GICV3_ESPI
>> + if ( gic_is_valid_espi(irq) )
>> + return true;
>> +#endif
>> return irq < gic_number_lines();
>> }
>>
>> @@ -325,6 +338,10 @@ struct gic_info {
>> enum gic_version hw_version;
>> /* Number of GIC lines supported */
>> unsigned int nr_lines;
>> +#ifdef CONFIG_GICV3_ESPI
>> + /* Number of GIC eSPI supported */
>> + unsigned int nr_espi;
>> +#endif
>> /* Number of LR registers */
>> uint8_t nr_lrs;
>> /* Maintenance irq number */
>> diff --git a/xen/arch/arm/include/asm/gic_v3_defs.h b/xen/arch/arm/include/asm/gic_v3_defs.h
>> index 2af093e774..7f769b38e3 100644
>> --- a/xen/arch/arm/include/asm/gic_v3_defs.h
>> +++ b/xen/arch/arm/include/asm/gic_v3_defs.h
>> @@ -37,6 +37,39 @@
>> #define GICD_IROUTER1019 (0x7FD8)
>> #define GICD_PIDR2 (0xFFE8)
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> +/* Additional registers for GICv3.1 */
>> +#define GICD_IGROUPRnE (0x1000)
>> +#define GICD_IGROUPRnEN (0x107C)
>> +#define GICD_ISENABLERnE (0x1200)
>> +#define GICD_ISENABLERnEN (0x127C)
>> +#define GICD_ICENABLERnE (0x1400)
>> +#define GICD_ICENABLERnEN (0x147C)
>> +#define GICD_ISPENDRnE (0x1600)
>> +#define GICD_ISPENDRnEN (0x167C)
>> +#define GICD_ICPENDRnE (0x1800)
>> +#define GICD_ICPENDRnEN (0x187C)
>> +#define GICD_ISACTIVERnE (0x1A00)
>> +#define GICD_ISACTIVERnEN (0x1A7C)
>> +#define GICD_ICACTIVERnE (0x1C00)
>> +#define GICD_ICACTIVERnEN (0x1C7C)
>> +#define GICD_IPRIORITYRnE (0x2000)
>> +#define GICD_IPRIORITYRnEN (0x23FC)
>> +#define GICD_ICFGRnE (0x3000)
>> +#define GICD_ICFGRnEN (0x30FC)
>> +#define GICD_IROUTERnE (0x8000)
>> +#define GICD_IROUTERnEN (0x9FFC)
>> +
>> +#define GICD_TYPER_ESPI_SHIFT 8
>> +#define GICD_TYPER_ESPI_RANGE_SHIFT 27
>> +#define GICD_TYPER_ESPI_RANGE_MASK (0x1F)
>> +#define GICD_TYPER_ESPI (1U << GICD_TYPER_ESPI_SHIFT)
>> +#define GICD_TYPER_ESPI_RANGE(typer) ((((typer) & GICD_TYPER_ESPI_RANGE_MASK) + 1) * 32)
>
> Isn't this line a bit long?
>
Yes, I will revise all patches in the series with such long lines and
fix them in V3.
>> +#define GICD_TYPER_ESPIS_NUM(typer) \
>> + (((typer) & GICD_TYPER_ESPI) ? \
>> + GICD_TYPER_ESPI_RANGE((typer) >> GICD_TYPER_ESPI_RANGE_SHIFT) : 0)
>
> I am not sure that this is correct.
>
> Probably you wanted to write
> + GICD_TYPER_ESPI_RANGE((typer >> GICD_TYPER_ESPI_RANGE_SHIFT)) : 0)
>
>
I double checked - it seems like everything is correct - the typer
parameter should be surrounded by parentheses, and GICD_TYPER_ESPI_RANGE
should operate with the already shifted value. However, I wrote a
slightly confusing macro. At the very least, it would be better to
rename in V3 the typer parameter in GICD_TYPER_ESPI_RANGE, because it
takes the value of GICD_TYPER that is already shifted by 27, instead of
the raw register value. Would that be acceptable?
>> +#endif
>> +
>> /* Common between GICD_PIDR2 and GICR_PIDR2 */
>> #define GIC_PIDR2_ARCH_MASK (0xf0)
>> #define GIC_PIDR2_ARCH_GICv3 (0x30)
>
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v2 05/10] xen/arm: gicv3: implement handling of GICv3.1 eSPI
2025-08-07 12:33 ` [PATCH v2 05/10] xen/arm: gicv3: implement handling of GICv3.1 eSPI Leonid Komarianskyi
2025-08-21 16:16 ` Volodymyr Babchuk
@ 2025-08-23 14:23 ` Oleksandr Tyshchenko
2025-08-24 18:17 ` Leonid Komarianskyi
1 sibling, 1 reply; 55+ messages in thread
From: Oleksandr Tyshchenko @ 2025-08-23 14:23 UTC (permalink / raw)
To: xen-devel, Leonid Komarianskyi
On 07.08.25 15:33, Leonid Komarianskyi wrote:
Hello Leonid
> Introduced appropriate register definitions, helper macros,
> and initialization of required GICv3.1 distributor registers
> to support eSPI. This type of interrupt is handled in the
> same way as regular SPI interrupts, with the following
> differences:
>
> 1) eSPIs can have up to 1024 interrupts, starting from the
> beginning of the range, whereas regular SPIs use INTIDs from
> 32 to 1019, totaling 988 interrupts;
> 2) eSPIs start at INTID 4096, necessitating additional interrupt
> index conversion during register operations.
>
> In case if appropriate config is disabled, or GIC HW doesn't
> support eSPI, the existing functionality will remain the same.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>
> ---
> Changes in V2:
> - move gic_number_espis function from
> [PATCH 08/10] xen/arm: vgic: add resource management for extended SPIs
> to use it in the newly introduced gic_is_valid_espi
> - add gic_is_valid_espi which checks if IRQ number is in supported
> by HW eSPI range
> - update gic_is_valid_irq conditions to allow operations with eSPIs
> ---
> xen/arch/arm/gic-v3.c | 73 ++++++++++++++++++++++++++
> xen/arch/arm/include/asm/gic.h | 17 ++++++
> xen/arch/arm/include/asm/gic_v3_defs.h | 33 ++++++++++++
> 3 files changed, 123 insertions(+)
>
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index 8fd78aba44..a0e8ee1a1e 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -485,6 +485,36 @@ static void __iomem *get_addr_by_offset(struct irq_desc *irqd, u32 offset)
> default:
> break;
> }
> +#ifdef CONFIG_GICV3_ESPI
> + case ESPI_BASE_INTID ... ESPI_MAX_INTID:
> + {
> + u32 irq_index = ESPI_INTID2IDX(irqd->irq);
> +
> + switch ( offset )
> + {
> + case GICD_ISENABLER:
> + return (GICD + GICD_ISENABLERnE + (irq_index / 32) * 4);
> + case GICD_ICENABLER:
> + return (GICD + GICD_ICENABLERnE + (irq_index / 32) * 4);
> + case GICD_ISPENDR:
> + return (GICD + GICD_ISPENDRnE + (irq_index / 32) * 4);
> + case GICD_ICPENDR:
> + return (GICD + GICD_ICPENDRnE + (irq_index / 32) * 4);
> + case GICD_ISACTIVER:
> + return (GICD + GICD_ISACTIVERnE + (irq_index / 32) * 4);
> + case GICD_ICACTIVER:
> + return (GICD + GICD_ICACTIVERnE + (irq_index / 32) * 4);
> + case GICD_ICFGR:
> + return (GICD + GICD_ICFGRnE + (irq_index / 16) * 4);
> + case GICD_IROUTER:
> + return (GICD + GICD_IROUTERnE + irq_index * 8);
> + case GICD_IPRIORITYR:
> + return (GICD + GICD_IPRIORITYRnE + irq_index);
> + default:
> + break;
> + }
> + }
> +#endif
> default:
> break;
> }
> @@ -645,6 +675,40 @@ static void gicv3_set_irq_priority(struct irq_desc *desc,
> spin_unlock(&gicv3.lock);
> }
>
> +#ifdef CONFIG_GICV3_ESPI
> +unsigned int gic_number_espis(void)
> +{
> + return gic_hw_ops->info->nr_espi;
> +}
> +
> +static void gicv3_dist_espi_common_init(uint32_t type)
missing __init ?
> +{
> + unsigned int espi_nr;
> + int i;
please use unsigned int if "i" cannot be negative
> +
> + espi_nr = min(1024U, GICD_TYPER_ESPIS_NUM(type));
> + gicv3_info.nr_espi = espi_nr;
> + /* The GIC HW doesn't support eSPI, so we can leave from here */
> + if ( gicv3_info.nr_espi == 0 )
> + return;
> +
> + for ( i = 0; i < espi_nr; i += 16 )
> + writel_relaxed(0, GICD + GICD_ICFGRnE + (i / 16) * 4);
> +
> + for ( i = 0; i < espi_nr; i += 4 )
> + writel_relaxed(GIC_PRI_IRQ_ALL, GICD + GICD_IPRIORITYRnE + (i / 4) * 4);
> +
> + for ( i = 0; i < espi_nr; i += 32 )
> + {
> + writel_relaxed(0xffffffffU, GICD + GICD_ICENABLERnE + (i / 32) * 4);
> + writel_relaxed(0xffffffffU, GICD + GICD_ICACTIVERnE + (i / 32) * 4);
> + }
> +
> + for ( i = 0; i < espi_nr; i += 32 )
> + writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPRnE + (i / 32) * 4);
> +}
> +#endif
> +
> static void __init gicv3_dist_init(void)
> {
> uint32_t type;
> @@ -690,6 +754,10 @@ static void __init gicv3_dist_init(void)
> for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i += 32 )
> writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPR + (i / 32) * 4);
>
> +#ifdef CONFIG_GICV3_ESPI
> + gicv3_dist_espi_common_init(type);
> +#endif
> +
> gicv3_dist_wait_for_rwp();
>
> /* Turn on the distributor */
> @@ -703,6 +771,11 @@ static void __init gicv3_dist_init(void)
>
> for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i++ )
> writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTER + i * 8);
> +
> +#ifdef CONFIG_GICV3_ESPI
> + for ( i = 0; i < gicv3_info.nr_espi; i++ )
> + writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTERnE + i * 8);
> +#endif
> }
>
> static int gicv3_enable_redist(void)
[snip]
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 05/10] xen/arm: gicv3: implement handling of GICv3.1 eSPI
2025-08-23 14:23 ` Oleksandr Tyshchenko
@ 2025-08-24 18:17 ` Leonid Komarianskyi
0 siblings, 0 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-24 18:17 UTC (permalink / raw)
To: Oleksandr Tyshchenko, xen-devel@lists.xenproject.org
Hi Oleksandr,
Thank you for your review comments.
On 23.08.25 17:23, Oleksandr Tyshchenko wrote:
> [You don't often get email from olekstysh@gmail.com. Learn why this is
> important at https://aka.ms/LearnAboutSenderIdentification ]
>
> On 07.08.25 15:33, Leonid Komarianskyi wrote:
>
> Hello Leonid
>
>
>> Introduced appropriate register definitions, helper macros,
>> and initialization of required GICv3.1 distributor registers
>> to support eSPI. This type of interrupt is handled in the
>> same way as regular SPI interrupts, with the following
>> differences:
>>
>> 1) eSPIs can have up to 1024 interrupts, starting from the
>> beginning of the range, whereas regular SPIs use INTIDs from
>> 32 to 1019, totaling 988 interrupts;
>> 2) eSPIs start at INTID 4096, necessitating additional interrupt
>> index conversion during register operations.
>>
>> In case if appropriate config is disabled, or GIC HW doesn't
>> support eSPI, the existing functionality will remain the same.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V2:
>> - move gic_number_espis function from
>> [PATCH 08/10] xen/arm: vgic: add resource management for extended SPIs
>> to use it in the newly introduced gic_is_valid_espi
>> - add gic_is_valid_espi which checks if IRQ number is in supported
>> by HW eSPI range
>> - update gic_is_valid_irq conditions to allow operations with eSPIs
>> ---
>> xen/arch/arm/gic-v3.c | 73 ++++++++++++++++++++++++++
>> xen/arch/arm/include/asm/gic.h | 17 ++++++
>> xen/arch/arm/include/asm/gic_v3_defs.h | 33 ++++++++++++
>> 3 files changed, 123 insertions(+)
>>
>> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
>> index 8fd78aba44..a0e8ee1a1e 100644
>> --- a/xen/arch/arm/gic-v3.c
>> +++ b/xen/arch/arm/gic-v3.c
>> @@ -485,6 +485,36 @@ static void __iomem *get_addr_by_offset(struct
>> irq_desc *irqd, u32 offset)
>> default:
>> break;
>> }
>> +#ifdef CONFIG_GICV3_ESPI
>> + case ESPI_BASE_INTID ... ESPI_MAX_INTID:
>> + {
>> + u32 irq_index = ESPI_INTID2IDX(irqd->irq);
>> +
>> + switch ( offset )
>> + {
>> + case GICD_ISENABLER:
>> + return (GICD + GICD_ISENABLERnE + (irq_index / 32) * 4);
>> + case GICD_ICENABLER:
>> + return (GICD + GICD_ICENABLERnE + (irq_index / 32) * 4);
>> + case GICD_ISPENDR:
>> + return (GICD + GICD_ISPENDRnE + (irq_index / 32) * 4);
>> + case GICD_ICPENDR:
>> + return (GICD + GICD_ICPENDRnE + (irq_index / 32) * 4);
>> + case GICD_ISACTIVER:
>> + return (GICD + GICD_ISACTIVERnE + (irq_index / 32) * 4);
>> + case GICD_ICACTIVER:
>> + return (GICD + GICD_ICACTIVERnE + (irq_index / 32) * 4);
>> + case GICD_ICFGR:
>> + return (GICD + GICD_ICFGRnE + (irq_index / 16) * 4);
>> + case GICD_IROUTER:
>> + return (GICD + GICD_IROUTERnE + irq_index * 8);
>> + case GICD_IPRIORITYR:
>> + return (GICD + GICD_IPRIORITYRnE + irq_index);
>> + default:
>> + break;
>> + }
>> + }
>> +#endif
>> default:
>> break;
>> }
>> @@ -645,6 +675,40 @@ static void gicv3_set_irq_priority(struct
>> irq_desc *desc,
>> spin_unlock(&gicv3.lock);
>> }
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> +unsigned int gic_number_espis(void)
>> +{
>> + return gic_hw_ops->info->nr_espi;
>> +}
>> +
>> +static void gicv3_dist_espi_common_init(uint32_t type)
>
> missing __init ?
>
Yes, I missed this. I will add it in V3.
>> +{
>> + unsigned int espi_nr;
>> + int i;
>
> please use unsigned int if "i" cannot be negative
>
Sure, I will change int to unsigned int, because it really cannot be
negative.
>> +
>> + espi_nr = min(1024U, GICD_TYPER_ESPIS_NUM(type));
>> + gicv3_info.nr_espi = espi_nr;
>> + /* The GIC HW doesn't support eSPI, so we can leave from here */
>> + if ( gicv3_info.nr_espi == 0 )
>> + return;
>> +
>> + for ( i = 0; i < espi_nr; i += 16 )
>> + writel_relaxed(0, GICD + GICD_ICFGRnE + (i / 16) * 4);
>> +
>> + for ( i = 0; i < espi_nr; i += 4 )
>> + writel_relaxed(GIC_PRI_IRQ_ALL, GICD + GICD_IPRIORITYRnE +
>> (i / 4) * 4);
>> +
>> + for ( i = 0; i < espi_nr; i += 32 )
>> + {
>> + writel_relaxed(0xffffffffU, GICD + GICD_ICENABLERnE + (i /
>> 32) * 4);
>> + writel_relaxed(0xffffffffU, GICD + GICD_ICACTIVERnE + (i /
>> 32) * 4);
>> + }
>> +
>> + for ( i = 0; i < espi_nr; i += 32 )
>> + writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPRnE + (i /
>> 32) * 4);
>> +}
>> +#endif
>> +
>> static void __init gicv3_dist_init(void)
>> {
>> uint32_t type;
>> @@ -690,6 +754,10 @@ static void __init gicv3_dist_init(void)
>> for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i += 32 )
>> writel_relaxed(GENMASK(31, 0), GICD + GICD_IGROUPR + (i /
>> 32) * 4);
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> + gicv3_dist_espi_common_init(type);
>> +#endif
>> +
>> gicv3_dist_wait_for_rwp();
>>
>> /* Turn on the distributor */
>> @@ -703,6 +771,11 @@ static void __init gicv3_dist_init(void)
>>
>> for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i++ )
>> writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTER + i
>> * 8);
>> +
>> +#ifdef CONFIG_GICV3_ESPI
>> + for ( i = 0; i < gicv3_info.nr_espi; i++ )
>> + writeq_relaxed_non_atomic(affinity, GICD + GICD_IROUTERnE + i
>> * 8);
>> +#endif
>> }
>>
>> static int gicv3_enable_redist(void)
>
>
> [snip]
>
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread
* [PATCH v2 04/10] xen/arm/irq: add handling for IRQs in the eSPI range
2025-08-07 12:33 ` [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations Leonid Komarianskyi
` (2 preceding siblings ...)
2025-08-07 12:33 ` [PATCH v2 05/10] xen/arm: gicv3: implement handling of GICv3.1 eSPI Leonid Komarianskyi
@ 2025-08-07 12:33 ` Leonid Komarianskyi
2025-08-21 15:59 ` Volodymyr Babchuk
2025-08-07 12:33 ` [PATCH v2 07/10] xen/arm: gicv3: modify ICH_LR_PHYSICAL_MASK to allow eSPI processing Leonid Komarianskyi
` (6 subsequent siblings)
10 siblings, 1 reply; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-07 12:33 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Leonid Komarianskyi, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk
Currently, Xen does not support eSPI interrupts, leading
to a data abort when such interrupts are defined in the DTS.
This patch introduces a separate array to initialize up to
1024 interrupt descriptors in the eSPI range and adds the
necessary defines and helper function. These changes lay the
groundwork for future implementation of full eSPI interrupt
support. As this GICv3.1 feature is not required by all vendors,
all changes are guarded by ifdefs, depending on the corresponding
Kconfig option.
Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
---
Changes in V2:
- use (ESPI_MAX_INTID + 1) instead of (ESPI_BASE_INTID + NR_IRQS)
- remove unnecessary comment for nr_irqs initialization
---
xen/arch/arm/Kconfig | 9 +++++++++
xen/arch/arm/include/asm/irq.h | 25 +++++++++++++++++++++++++
xen/arch/arm/irq.c | 26 ++++++++++++++++++++++++++
3 files changed, 60 insertions(+)
diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 17df147b25..08073ece1f 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -135,6 +135,15 @@ config GICV3
Driver for the ARM Generic Interrupt Controller v3.
If unsure, use the default setting.
+config GICV3_ESPI
+ bool "Extended SPI range support"
+ depends on GICV3 && !NEW_VGIC
+ default y
+ help
+ Allow Xen and domains to use interrupt numbers from the extended SPI
+ range, from 4096 to 5119. This feature is introduced in GICv3.1
+ architecture.
+
config HAS_ITS
bool "GICv3 ITS MSI controller support (UNSUPPORTED)" if UNSUPPORTED
depends on GICV3 && !NEW_VGIC && !ARM_32
diff --git a/xen/arch/arm/include/asm/irq.h b/xen/arch/arm/include/asm/irq.h
index 5bc6475eb4..acebc3d42f 100644
--- a/xen/arch/arm/include/asm/irq.h
+++ b/xen/arch/arm/include/asm/irq.h
@@ -32,6 +32,14 @@ struct arch_irq_desc {
#define SPI_MAX_INTID 1019
#define LPI_OFFSET 8192
+#ifdef CONFIG_GICV3_ESPI
+#define ESPI_BASE_INTID 4096
+#define ESPI_MAX_INTID 5119
+
+#define ESPI_INTID2IDX(intid) ((intid) - ESPI_BASE_INTID)
+#define ESPI_IDX2INTID(idx) ((idx) + ESPI_BASE_INTID)
+#endif
+
/* LPIs are always numbered starting at 8192, so 0 is a good invalid case. */
#define INVALID_LPI 0
@@ -39,7 +47,15 @@ struct arch_irq_desc {
#define INVALID_IRQ 1023
extern const unsigned int nr_irqs;
+#ifdef CONFIG_GICV3_ESPI
+/*
+ * This will also cover the eSPI range, as some critical devices
+ * for booting Xen (e.g., serial) may use this type of interrupts.
+ */
+#define nr_static_irqs (ESPI_MAX_INTID + 1)
+#else
#define nr_static_irqs NR_IRQS
+#endif
struct irq_desc;
struct irqaction;
@@ -55,6 +71,15 @@ static inline bool is_lpi(unsigned int irq)
return irq >= LPI_OFFSET;
}
+static inline bool is_espi(unsigned int irq)
+{
+#ifdef CONFIG_GICV3_ESPI
+ return (irq >= ESPI_BASE_INTID && irq <= ESPI_MAX_INTID);
+#else
+ return false;
+#endif
+}
+
#define domain_pirq_to_irq(d, pirq) (pirq)
bool is_assignable_irq(unsigned int irq);
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 50e57aaea7..9bc72fbbc9 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -19,7 +19,11 @@
#include <asm/gic.h>
#include <asm/vgic.h>
+#ifdef CONFIG_GICV3_ESPI
+const unsigned int nr_irqs = ESPI_MAX_INTID + 1;
+#else
const unsigned int nr_irqs = NR_IRQS;
+#endif
static unsigned int local_irqs_type[NR_LOCAL_IRQS];
static DEFINE_SPINLOCK(local_irqs_type_lock);
@@ -46,6 +50,9 @@ void irq_end_none(struct irq_desc *irq)
}
static irq_desc_t irq_desc[NR_IRQS - NR_LOCAL_IRQS];
+#ifdef CONFIG_GICV3_ESPI
+static irq_desc_t espi_desc[NR_IRQS];
+#endif
static DEFINE_PER_CPU(irq_desc_t[NR_LOCAL_IRQS], local_irq_desc);
struct irq_desc *__irq_to_desc(unsigned int irq)
@@ -53,6 +60,11 @@ struct irq_desc *__irq_to_desc(unsigned int irq)
if ( irq < NR_LOCAL_IRQS )
return &this_cpu(local_irq_desc)[irq];
+#ifdef CONFIG_GICV3_ESPI
+ if ( is_espi(irq) )
+ return &espi_desc[ESPI_INTID2IDX(irq)];
+#endif
+
return &irq_desc[irq-NR_LOCAL_IRQS];
}
@@ -79,6 +91,20 @@ static int __init init_irq_data(void)
desc->action = NULL;
}
+#ifdef CONFIG_GICV3_ESPI
+ for ( irq = ESPI_BASE_INTID; irq <= ESPI_MAX_INTID; irq++ )
+ {
+ struct irq_desc *desc = irq_to_desc(irq);
+ int rc = init_one_irq_desc(desc);
+
+ if ( rc )
+ return rc;
+
+ desc->irq = irq;
+ desc->action = NULL;
+ }
+#endif
+
return 0;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 55+ messages in thread* Re: [PATCH v2 04/10] xen/arm/irq: add handling for IRQs in the eSPI range
2025-08-07 12:33 ` [PATCH v2 04/10] xen/arm/irq: add handling for IRQs in the eSPI range Leonid Komarianskyi
@ 2025-08-21 15:59 ` Volodymyr Babchuk
2025-08-21 16:46 ` Julien Grall
2025-08-22 12:59 ` Leonid Komarianskyi
0 siblings, 2 replies; 55+ messages in thread
From: Volodymyr Babchuk @ 2025-08-21 15:59 UTC (permalink / raw)
To: Leonid Komarianskyi
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
> Currently, Xen does not support eSPI interrupts, leading
> to a data abort when such interrupts are defined in the DTS.
>
> This patch introduces a separate array to initialize up to
> 1024 interrupt descriptors in the eSPI range and adds the
> necessary defines and helper function. These changes lay the
> groundwork for future implementation of full eSPI interrupt
> support. As this GICv3.1 feature is not required by all vendors,
> all changes are guarded by ifdefs, depending on the corresponding
> Kconfig option.
I don't think that it is a good idea to hide this feature under Kconfig
option, as this will increase number of different build variants.
I believe that runtime check for GICD_TYPER.ESPI should be sufficient,
but maintainers can correct me there.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>
> ---
> Changes in V2:
> - use (ESPI_MAX_INTID + 1) instead of (ESPI_BASE_INTID + NR_IRQS)
> - remove unnecessary comment for nr_irqs initialization
> ---
> xen/arch/arm/Kconfig | 9 +++++++++
> xen/arch/arm/include/asm/irq.h | 25 +++++++++++++++++++++++++
> xen/arch/arm/irq.c | 26 ++++++++++++++++++++++++++
> 3 files changed, 60 insertions(+)
>
> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> index 17df147b25..08073ece1f 100644
> --- a/xen/arch/arm/Kconfig
> +++ b/xen/arch/arm/Kconfig
> @@ -135,6 +135,15 @@ config GICV3
> Driver for the ARM Generic Interrupt Controller v3.
> If unsure, use the default setting.
>
> +config GICV3_ESPI
> + bool "Extended SPI range support"
> + depends on GICV3 && !NEW_VGIC
> + default y
> + help
> + Allow Xen and domains to use interrupt numbers from the extended SPI
> + range, from 4096 to 5119. This feature is introduced in GICv3.1
> + architecture.
> +
> config HAS_ITS
> bool "GICv3 ITS MSI controller support (UNSUPPORTED)" if UNSUPPORTED
> depends on GICV3 && !NEW_VGIC && !ARM_32
> diff --git a/xen/arch/arm/include/asm/irq.h b/xen/arch/arm/include/asm/irq.h
> index 5bc6475eb4..acebc3d42f 100644
> --- a/xen/arch/arm/include/asm/irq.h
> +++ b/xen/arch/arm/include/asm/irq.h
> @@ -32,6 +32,14 @@ struct arch_irq_desc {
> #define SPI_MAX_INTID 1019
> #define LPI_OFFSET 8192
>
> +#ifdef CONFIG_GICV3_ESPI
> +#define ESPI_BASE_INTID 4096
> +#define ESPI_MAX_INTID 5119
> +
> +#define ESPI_INTID2IDX(intid) ((intid) - ESPI_BASE_INTID)
> +#define ESPI_IDX2INTID(idx) ((idx) + ESPI_BASE_INTID)
> +#endif
> +
> /* LPIs are always numbered starting at 8192, so 0 is a good invalid case. */
> #define INVALID_LPI 0
>
> @@ -39,7 +47,15 @@ struct arch_irq_desc {
> #define INVALID_IRQ 1023
>
> extern const unsigned int nr_irqs;
> +#ifdef CONFIG_GICV3_ESPI
> +/*
> + * This will also cover the eSPI range, as some critical devices
> + * for booting Xen (e.g., serial) may use this type of interrupts.
> + */
> +#define nr_static_irqs (ESPI_MAX_INTID + 1)
> +#else
> #define nr_static_irqs NR_IRQS
> +#endif
Don't introduce defines that look like variables. I am sure that MISRA
team will be unhappy about that. But what you can really do is to
introduce variable nr_static_irqs, which value will depend on
GICD_TYPER.ESPI and GICD_TYPER.ESPI_range
>
> struct irq_desc;
> struct irqaction;
> @@ -55,6 +71,15 @@ static inline bool is_lpi(unsigned int irq)
> return irq >= LPI_OFFSET;
> }
>
> +static inline bool is_espi(unsigned int irq)
> +{
> +#ifdef CONFIG_GICV3_ESPI
> + return (irq >= ESPI_BASE_INTID && irq <= ESPI_MAX_INTID);
> +#else
> + return false;
> +#endif
> +}
> +
> #define domain_pirq_to_irq(d, pirq) (pirq)
>
> bool is_assignable_irq(unsigned int irq);
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 50e57aaea7..9bc72fbbc9 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -19,7 +19,11 @@
> #include <asm/gic.h>
> #include <asm/vgic.h>
>
> +#ifdef CONFIG_GICV3_ESPI
> +const unsigned int nr_irqs = ESPI_MAX_INTID + 1;
> +#else
> const unsigned int nr_irqs = NR_IRQS;
> +#endif
>
> static unsigned int local_irqs_type[NR_LOCAL_IRQS];
> static DEFINE_SPINLOCK(local_irqs_type_lock);
> @@ -46,6 +50,9 @@ void irq_end_none(struct irq_desc *irq)
> }
>
> static irq_desc_t irq_desc[NR_IRQS - NR_LOCAL_IRQS];
> +#ifdef CONFIG_GICV3_ESPI
> +static irq_desc_t espi_desc[NR_IRQS];
This is really confusing. Should it be something like espi_desc[NR_ESPI_IRQS]?
> +#endif
> static DEFINE_PER_CPU(irq_desc_t[NR_LOCAL_IRQS], local_irq_desc);
>
> struct irq_desc *__irq_to_desc(unsigned int irq)
> @@ -53,6 +60,11 @@ struct irq_desc *__irq_to_desc(unsigned int irq)
> if ( irq < NR_LOCAL_IRQS )
> return &this_cpu(local_irq_desc)[irq];
>
> +#ifdef CONFIG_GICV3_ESPI
> + if ( is_espi(irq) )
> + return &espi_desc[ESPI_INTID2IDX(irq)];
> +#endif
> +
> return &irq_desc[irq-NR_LOCAL_IRQS];
> }
>
> @@ -79,6 +91,20 @@ static int __init init_irq_data(void)
> desc->action = NULL;
> }
>
> +#ifdef CONFIG_GICV3_ESPI
> + for ( irq = ESPI_BASE_INTID; irq <= ESPI_MAX_INTID; irq++ )
> + {
> + struct irq_desc *desc = irq_to_desc(irq);
> + int rc = init_one_irq_desc(desc);
> +
> + if ( rc )
> + return rc;
> +
> + desc->irq = irq;
> + desc->action = NULL;
> + }
> +#endif
> +
> return 0;
> }
--
WBR, Volodymyr
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 04/10] xen/arm/irq: add handling for IRQs in the eSPI range
2025-08-21 15:59 ` Volodymyr Babchuk
@ 2025-08-21 16:46 ` Julien Grall
2025-08-21 16:59 ` Volodymyr Babchuk
2025-08-22 12:59 ` Leonid Komarianskyi
1 sibling, 1 reply; 55+ messages in thread
From: Julien Grall @ 2025-08-21 16:46 UTC (permalink / raw)
To: Volodymyr Babchuk, Leonid Komarianskyi
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini,
Bertrand Marquis, Michal Orzel
Hi,
On 21/08/2025 16:59, Volodymyr Babchuk wrote:
> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>
>> Currently, Xen does not support eSPI interrupts, leading
>> to a data abort when such interrupts are defined in the DTS.
>>
>> This patch introduces a separate array to initialize up to
>> 1024 interrupt descriptors in the eSPI range and adds the
>> necessary defines and helper function. These changes lay the
>> groundwork for future implementation of full eSPI interrupt
>> support. As this GICv3.1 feature is not required by all vendors,
>> all changes are guarded by ifdefs, depending on the corresponding
>> Kconfig option.
>
> I don't think that it is a good idea to hide this feature under Kconfig
> option, as this will increase number of different build variants.
> I believe that runtime check for GICD_TYPER.ESPI should be
sufficient,> but maintainers can correct me there.
I haven't seen many board with ESPI available. So I think it would be
better if this is under a Kconfig because not everyone may want to have
the code.
[...]
>> struct irq_desc;
>> struct irqaction;
>> @@ -55,6 +71,15 @@ static inline bool is_lpi(unsigned int irq)
>> return irq >= LPI_OFFSET;
>> }
>>
>> +static inline bool is_espi(unsigned int irq)
>> +{
>> +#ifdef CONFIG_GICV3_ESPI
>> + return (irq >= ESPI_BASE_INTID && irq <= ESPI_MAX_INTID);
>> +#else
>> + return false;
>> +#endif
>> +}
>> +
>> #define domain_pirq_to_irq(d, pirq) (pirq)
>>
>> bool is_assignable_irq(unsigned int irq);
>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>> index 50e57aaea7..9bc72fbbc9 100644
>> --- a/xen/arch/arm/irq.c
>> +++ b/xen/arch/arm/irq.c
>> @@ -19,7 +19,11 @@
>> #include <asm/gic.h>
>> #include <asm/vgic.h>
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> +const unsigned int nr_irqs = ESPI_MAX_INTID + 1;
>> +#else
>> const unsigned int nr_irqs = NR_IRQS;
>> +#endif
>>
>> static unsigned int local_irqs_type[NR_LOCAL_IRQS];
>> static DEFINE_SPINLOCK(local_irqs_type_lock);
>> @@ -46,6 +50,9 @@ void irq_end_none(struct irq_desc *irq)
>> }
>>
>> static irq_desc_t irq_desc[NR_IRQS - NR_LOCAL_IRQS];
>> +#ifdef CONFIG_GICV3_ESPI
>> +static irq_desc_t espi_desc[NR_IRQS];
By how much will this increase the Xen binary?
>
> This is really confusing. Should it be something like espi_desc[NR_ESPI_IRQS]?
+1.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 04/10] xen/arm/irq: add handling for IRQs in the eSPI range
2025-08-21 16:46 ` Julien Grall
@ 2025-08-21 16:59 ` Volodymyr Babchuk
2025-08-21 17:13 ` Julien Grall
0 siblings, 1 reply; 55+ messages in thread
From: Volodymyr Babchuk @ 2025-08-21 16:59 UTC (permalink / raw)
To: Julien Grall
Cc: Leonid Komarianskyi, xen-devel@lists.xenproject.org,
Stefano Stabellini, Bertrand Marquis, Michal Orzel
Hi Julien,
Julien Grall <julien@xen.org> writes:
> Hi,
>
> On 21/08/2025 16:59, Volodymyr Babchuk wrote:
>> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>>
>>> Currently, Xen does not support eSPI interrupts, leading
>>> to a data abort when such interrupts are defined in the DTS.
>>>
>>> This patch introduces a separate array to initialize up to
>>> 1024 interrupt descriptors in the eSPI range and adds the
>>> necessary defines and helper function. These changes lay the
>>> groundwork for future implementation of full eSPI interrupt
>>> support. As this GICv3.1 feature is not required by all vendors,
>>> all changes are guarded by ifdefs, depending on the corresponding
>>> Kconfig option.
>> I don't think that it is a good idea to hide this feature under
>> Kconfig
>> option, as this will increase number of different build variants.
>> I believe that runtime check for GICD_TYPER.ESPI should be
> sufficient,> but maintainers can correct me there.
>
> I haven't seen many board with ESPI available. So I think it would be
> better if this is under a Kconfig because not everyone may want to
> have the code.
Probably, we can expect more in the future... Anyways, after reviewing
all patches in the series, I can see that code will be littered with
#ifdef CONFIG_GICV3_ESPI, which, probably, not a good thing.
>
> [...]
>
>>> struct irq_desc;
>>> struct irqaction;
>>> @@ -55,6 +71,15 @@ static inline bool is_lpi(unsigned int irq)
>>> return irq >= LPI_OFFSET;
>>> }
>>> +static inline bool is_espi(unsigned int irq)
>>> +{
>>> +#ifdef CONFIG_GICV3_ESPI
>>> + return (irq >= ESPI_BASE_INTID && irq <= ESPI_MAX_INTID);
>>> +#else
>>> + return false;
>>> +#endif
>>> +}
>>> +
>>> #define domain_pirq_to_irq(d, pirq) (pirq)
>>> bool is_assignable_irq(unsigned int irq);
>>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>>> index 50e57aaea7..9bc72fbbc9 100644
>>> --- a/xen/arch/arm/irq.c
>>> +++ b/xen/arch/arm/irq.c
>>> @@ -19,7 +19,11 @@
>>> #include <asm/gic.h>
>>> #include <asm/vgic.h>
>>> +#ifdef CONFIG_GICV3_ESPI
>>> +const unsigned int nr_irqs = ESPI_MAX_INTID + 1;
>>> +#else
>>> const unsigned int nr_irqs = NR_IRQS;
>>> +#endif
>>> static unsigned int local_irqs_type[NR_LOCAL_IRQS];
>>> static DEFINE_SPINLOCK(local_irqs_type_lock);
>>> @@ -46,6 +50,9 @@ void irq_end_none(struct irq_desc *irq)
>>> }
>>> static irq_desc_t irq_desc[NR_IRQS - NR_LOCAL_IRQS];
>>> +#ifdef CONFIG_GICV3_ESPI
>>> +static irq_desc_t espi_desc[NR_IRQS];
>
> By how much will this increase the Xen binary?
I am wondering this also. The struct is cache aligned, so it will take
at least 128 bytes. The whole array will take at least 128kb. Not
great, not terrible. As this should go to .bss, it should not increase
the binary itself.
Maybe it is better to allocate this dynamically? I do understand that we
want to get rid of as many dynamic allocs as possible, but maybe in this
case it will be okay. As a bonus point, we can't leave this pointer
present even if CONFIG_GICV3_ESPI=n, which will simplify some code in
latter patches.
[...]
--
WBR, Volodymyr
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 04/10] xen/arm/irq: add handling for IRQs in the eSPI range
2025-08-21 16:59 ` Volodymyr Babchuk
@ 2025-08-21 17:13 ` Julien Grall
2025-08-21 17:38 ` Volodymyr Babchuk
2025-08-22 12:41 ` Leonid Komarianskyi
0 siblings, 2 replies; 55+ messages in thread
From: Julien Grall @ 2025-08-21 17:13 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: Leonid Komarianskyi, xen-devel@lists.xenproject.org,
Stefano Stabellini, Bertrand Marquis, Michal Orzel
On 21/08/2025 17:59, Volodymyr Babchuk wrote:
> Julien Grall <julien@xen.org> writes:
>
>> Hi,
>>
>> On 21/08/2025 16:59, Volodymyr Babchuk wrote:
>>> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>>>
>>>> Currently, Xen does not support eSPI interrupts, leading
>>>> to a data abort when such interrupts are defined in the DTS.
>>>>
>>>> This patch introduces a separate array to initialize up to
>>>> 1024 interrupt descriptors in the eSPI range and adds the
>>>> necessary defines and helper function. These changes lay the
>>>> groundwork for future implementation of full eSPI interrupt
>>>> support. As this GICv3.1 feature is not required by all vendors,
>>>> all changes are guarded by ifdefs, depending on the corresponding
>>>> Kconfig option.
>>> I don't think that it is a good idea to hide this feature under
>>> Kconfig
>>> option, as this will increase number of different build variants.
>>> I believe that runtime check for GICD_TYPER.ESPI should be
>> sufficient,> but maintainers can correct me there.
>>
>> I haven't seen many board with ESPI available. So I think it would be
>> better if this is under a Kconfig because not everyone may want to
>> have the code.
>
> Probably, we can expect more in the future...
Well yes. But I was under the impression this the preferred approach.
See the discussion about disabling 32-bit support on 64-bit:
20250723075835.3993182-1-grygorii_strashko@epam.com
Anyways, after reviewing
> all patches in the series, I can see that code will be littered with
> #ifdef CONFIG_GICV3_ESPI, which, probably, not a good thing.
The solution is to provide wrappers to reduce the number of #ifdef. I
haven't checked all the places.
>
>>
>> [...]
>>
>>>> struct irq_desc;
>>>> struct irqaction;
>>>> @@ -55,6 +71,15 @@ static inline bool is_lpi(unsigned int irq)
>>>> return irq >= LPI_OFFSET;
>>>> }
>>>> +static inline bool is_espi(unsigned int irq)
>>>> +{
>>>> +#ifdef CONFIG_GICV3_ESPI
>>>> + return (irq >= ESPI_BASE_INTID && irq <= ESPI_MAX_INTID);
>>>> +#else
>>>> + return false;
>>>> +#endif
>>>> +}
>>>> +
>>>> #define domain_pirq_to_irq(d, pirq) (pirq)
>>>> bool is_assignable_irq(unsigned int irq);
>>>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>>>> index 50e57aaea7..9bc72fbbc9 100644
>>>> --- a/xen/arch/arm/irq.c
>>>> +++ b/xen/arch/arm/irq.c
>>>> @@ -19,7 +19,11 @@
>>>> #include <asm/gic.h>
>>>> #include <asm/vgic.h>
>>>> +#ifdef CONFIG_GICV3_ESPI
>>>> +const unsigned int nr_irqs = ESPI_MAX_INTID + 1;
>>>> +#else
>>>> const unsigned int nr_irqs = NR_IRQS;
>>>> +#endif
>>>> static unsigned int local_irqs_type[NR_LOCAL_IRQS];
>>>> static DEFINE_SPINLOCK(local_irqs_type_lock);
>>>> @@ -46,6 +50,9 @@ void irq_end_none(struct irq_desc *irq)
>>>> }
>>>> static irq_desc_t irq_desc[NR_IRQS - NR_LOCAL_IRQS];
>>>> +#ifdef CONFIG_GICV3_ESPI
>>>> +static irq_desc_t espi_desc[NR_IRQS];
>>
>> By how much will this increase the Xen binary?
>
> I am wondering this also. The struct is cache aligned, so it will take
> at least 128 bytes. The whole array will take at least 128kb. Not
> great, not terrible. As this should go to .bss, it should not increase
> the binary itself.
I guess "binary" was the wrong word. I was referring to the size of the
Xen in memory. On my setup Xen is 1448kb. Here you would increase ~9% of
resident size. This seems quite steep for a feature that is not often used.
>
> Maybe it is better to allocate this dynamically? I do understand that we
> want to get rid of as many dynamic allocs as possible, but maybe in this
> case it will be okay.
This is up to Leonid. I don't think this is strictly necessary in order
to get the eSPI support. However, until this is solved CONFIG_GICV3_EPSI
*must not* be on by default as this is done in this patch.
> As a bonus point, we can't leave this pointer
> present even if CONFIG_GICV3_ESPI=n, which will simplify some code in
> latter patches.
Did you intend to say "We can leave" rather than "we can't leave"?
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 04/10] xen/arm/irq: add handling for IRQs in the eSPI range
2025-08-21 17:13 ` Julien Grall
@ 2025-08-21 17:38 ` Volodymyr Babchuk
2025-08-22 12:41 ` Leonid Komarianskyi
1 sibling, 0 replies; 55+ messages in thread
From: Volodymyr Babchuk @ 2025-08-21 17:38 UTC (permalink / raw)
To: Julien Grall
Cc: Leonid Komarianskyi, xen-devel@lists.xenproject.org,
Stefano Stabellini, Bertrand Marquis, Michal Orzel
Julien Grall <julien@xen.org> writes:
> On 21/08/2025 17:59, Volodymyr Babchuk wrote:
>> Julien Grall <julien@xen.org> writes:
>>
>>> Hi,
>>>
>>> On 21/08/2025 16:59, Volodymyr Babchuk wrote:
>>>> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>>>>
>>>>> Currently, Xen does not support eSPI interrupts, leading
>>>>> to a data abort when such interrupts are defined in the DTS.
>>>>>
>>>>> This patch introduces a separate array to initialize up to
>>>>> 1024 interrupt descriptors in the eSPI range and adds the
>>>>> necessary defines and helper function. These changes lay the
>>>>> groundwork for future implementation of full eSPI interrupt
>>>>> support. As this GICv3.1 feature is not required by all vendors,
>>>>> all changes are guarded by ifdefs, depending on the corresponding
>>>>> Kconfig option.
>>>> I don't think that it is a good idea to hide this feature under
>>>> Kconfig
>>>> option, as this will increase number of different build variants.
>>>> I believe that runtime check for GICD_TYPER.ESPI should be
>>> sufficient,> but maintainers can correct me there.
>>>
>>> I haven't seen many board with ESPI available. So I think it would be
>>> better if this is under a Kconfig because not everyone may want to
>>> have the code.
>> Probably, we can expect more in the future...
>
> Well yes. But I was under the impression this the preferred
> approach. See the discussion about disabling 32-bit support on 64-bit:
>
> 20250723075835.3993182-1-grygorii_strashko@epam.com
Ah yes, safety certification. Welp, can't argue with that.
>
>> Anyways, after reviewing
>> all patches in the series, I can see that code will be littered with
>> #ifdef CONFIG_GICV3_ESPI, which, probably, not a good thing.
>
> The solution is to provide wrappers to reduce the number of #ifdef. I
> haven't checked all the places.
Yes, I was also thinking about that, but I got an impression, that in
many cases it will be hard to provide such wrappers. Anyways, something
is needs to be done here.
[...]
>
>> As a bonus point, we can't leave this pointer
>> present even if CONFIG_GICV3_ESPI=n, which will simplify some code in
>> latter patches.
>
> Did you intend to say "We can leave" rather than "we can't leave"?
Correct
--
WBR, Volodymyr
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v2 04/10] xen/arm/irq: add handling for IRQs in the eSPI range
2025-08-21 17:13 ` Julien Grall
2025-08-21 17:38 ` Volodymyr Babchuk
@ 2025-08-22 12:41 ` Leonid Komarianskyi
1 sibling, 0 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-22 12:41 UTC (permalink / raw)
To: Julien Grall, Volodymyr Babchuk
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini,
Bertrand Marquis, Michal Orzel
Hi Volodymyr and Julien,
Thank you for your review comments and for raising an important
discussion regarding memory usage and wrappers.
On 21.08.25 20:13, Julien Grall wrote:
>
>
> On 21/08/2025 17:59, Volodymyr Babchuk wrote:
>> Julien Grall <julien@xen.org> writes:
>>
>>> Hi,
>>>
>>> On 21/08/2025 16:59, Volodymyr Babchuk wrote:
>>>> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>>>>
>>>>> Currently, Xen does not support eSPI interrupts, leading
>>>>> to a data abort when such interrupts are defined in the DTS.
>>>>>
>>>>> This patch introduces a separate array to initialize up to
>>>>> 1024 interrupt descriptors in the eSPI range and adds the
>>>>> necessary defines and helper function. These changes lay the
>>>>> groundwork for future implementation of full eSPI interrupt
>>>>> support. As this GICv3.1 feature is not required by all vendors,
>>>>> all changes are guarded by ifdefs, depending on the corresponding
>>>>> Kconfig option.
>>>> I don't think that it is a good idea to hide this feature under
>>>> Kconfig
>>>> option, as this will increase number of different build variants.
>>>> I believe that runtime check for GICD_TYPER.ESPI should be
>>> sufficient,> but maintainers can correct me there.
>>>
>>> I haven't seen many board with ESPI available. So I think it would be
>>> better if this is under a Kconfig because not everyone may want to
>>> have the code.
>>
>> Probably, we can expect more in the future...
>
> Well yes. But I was under the impression this the preferred approach.
> See the discussion about disabling 32-bit support on 64-bit:
>
> 20250723075835.3993182-1-grygorii_strashko@epam.com
>
> Anyways, after reviewing
>> all patches in the series, I can see that code will be littered with
>> #ifdef CONFIG_GICV3_ESPI, which, probably, not a good thing.
>
> The solution is to provide wrappers to reduce the number of #ifdef. I
> haven't checked all the places.
I agree with you, it's a good point to use wrappers where possible. I
will revise all patches in the series and try to use wrappers where
possible to reduce #ifdefs.
>>
>>>
>>> [...]
>>>
>>>>> struct irq_desc;
>>>>> struct irqaction;
>>>>> @@ -55,6 +71,15 @@ static inline bool is_lpi(unsigned int irq)
>>>>> return irq >= LPI_OFFSET;
>>>>> }
>>>>> +static inline bool is_espi(unsigned int irq)
>>>>> +{
>>>>> +#ifdef CONFIG_GICV3_ESPI
>>>>> + return (irq >= ESPI_BASE_INTID && irq <= ESPI_MAX_INTID);
>>>>> +#else
>>>>> + return false;
>>>>> +#endif
>>>>> +}
>>>>> +
>>>>> #define domain_pirq_to_irq(d, pirq) (pirq)
>>>>> bool is_assignable_irq(unsigned int irq);
>>>>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>>>>> index 50e57aaea7..9bc72fbbc9 100644
>>>>> --- a/xen/arch/arm/irq.c
>>>>> +++ b/xen/arch/arm/irq.c
>>>>> @@ -19,7 +19,11 @@
>>>>> #include <asm/gic.h>
>>>>> #include <asm/vgic.h>
>>>>> +#ifdef CONFIG_GICV3_ESPI
>>>>> +const unsigned int nr_irqs = ESPI_MAX_INTID + 1;
>>>>> +#else
>>>>> const unsigned int nr_irqs = NR_IRQS;
>>>>> +#endif
>>>>> static unsigned int local_irqs_type[NR_LOCAL_IRQS];
>>>>> static DEFINE_SPINLOCK(local_irqs_type_lock);
>>>>> @@ -46,6 +50,9 @@ void irq_end_none(struct irq_desc *irq)
>>>>> }
>>>>> static irq_desc_t irq_desc[NR_IRQS - NR_LOCAL_IRQS];
>>>>> +#ifdef CONFIG_GICV3_ESPI
>>>>> +static irq_desc_t espi_desc[NR_IRQS];
>>>
>>> By how much will this increase the Xen binary?
>>
>> I am wondering this also. The struct is cache aligned, so it will take
>> at least 128 bytes. The whole array will take at least 128kb. Not
>> great, not terrible. As this should go to .bss, it should not increase
>> the binary itself.
>
> I guess "binary" was the wrong word. I was referring to the size of the
> Xen in memory. On my setup Xen is 1448kb. Here you would increase ~9% of
> resident size. This seems quite steep for a feature that is not often used.
>
>>
>> Maybe it is better to allocate this dynamically? I do understand that we
>> want to get rid of as many dynamic allocs as possible, but maybe in this
>> case it will be okay.
>
> This is up to Leonid. I don't think this is strictly necessary in order
> to get the eSPI support. However, until this is solved CONFIG_GICV3_EPSI
> *must not* be on by default as this is done in this patch.
>
I will check how much time and effort are required to switch to dynamic
allocation. If it does not take much time and does not require many
changes, I will prepare an additional preparation patch in V3.
Otherwise, I will disable the config by default in V3.
>> As a bonus point, we can't leave this pointer
>> present even if CONFIG_GICV3_ESPI=n, which will simplify some code in
>> latter patches.
>
> Did you intend to say "We can leave" rather than "we can't leave"?
>
> Cheers,
>
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v2 04/10] xen/arm/irq: add handling for IRQs in the eSPI range
2025-08-21 15:59 ` Volodymyr Babchuk
2025-08-21 16:46 ` Julien Grall
@ 2025-08-22 12:59 ` Leonid Komarianskyi
1 sibling, 0 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-22 12:59 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Hi Volodymyr,
Thank you for your review.
On 21.08.25 18:59, Volodymyr Babchuk wrote:
> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>
>> Currently, Xen does not support eSPI interrupts, leading
>> to a data abort when such interrupts are defined in the DTS.
>>
>> This patch introduces a separate array to initialize up to
>> 1024 interrupt descriptors in the eSPI range and adds the
>> necessary defines and helper function. These changes lay the
>> groundwork for future implementation of full eSPI interrupt
>> support. As this GICv3.1 feature is not required by all vendors,
>> all changes are guarded by ifdefs, depending on the corresponding
>> Kconfig option.
>
> I don't think that it is a good idea to hide this feature under Kconfig
> option, as this will increase number of different build variants.
> I believe that runtime check for GICD_TYPER.ESPI should be sufficient,
> but maintainers can correct me there.
>
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V2:
>> - use (ESPI_MAX_INTID + 1) instead of (ESPI_BASE_INTID + NR_IRQS)
>> - remove unnecessary comment for nr_irqs initialization
>> ---
>> xen/arch/arm/Kconfig | 9 +++++++++
>> xen/arch/arm/include/asm/irq.h | 25 +++++++++++++++++++++++++
>> xen/arch/arm/irq.c | 26 ++++++++++++++++++++++++++
>> 3 files changed, 60 insertions(+)
>>
>> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
>> index 17df147b25..08073ece1f 100644
>> --- a/xen/arch/arm/Kconfig
>> +++ b/xen/arch/arm/Kconfig
>> @@ -135,6 +135,15 @@ config GICV3
>> Driver for the ARM Generic Interrupt Controller v3.
>> If unsure, use the default setting.
>>
>> +config GICV3_ESPI
>> + bool "Extended SPI range support"
>> + depends on GICV3 && !NEW_VGIC
>> + default y
>> + help
>> + Allow Xen and domains to use interrupt numbers from the extended SPI
>> + range, from 4096 to 5119. This feature is introduced in GICv3.1
>> + architecture.
>> +
>> config HAS_ITS
>> bool "GICv3 ITS MSI controller support (UNSUPPORTED)" if UNSUPPORTED
>> depends on GICV3 && !NEW_VGIC && !ARM_32
>> diff --git a/xen/arch/arm/include/asm/irq.h b/xen/arch/arm/include/asm/irq.h
>> index 5bc6475eb4..acebc3d42f 100644
>> --- a/xen/arch/arm/include/asm/irq.h
>> +++ b/xen/arch/arm/include/asm/irq.h
>> @@ -32,6 +32,14 @@ struct arch_irq_desc {
>> #define SPI_MAX_INTID 1019
>> #define LPI_OFFSET 8192
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> +#define ESPI_BASE_INTID 4096
>> +#define ESPI_MAX_INTID 5119
>> +
>> +#define ESPI_INTID2IDX(intid) ((intid) - ESPI_BASE_INTID)
>> +#define ESPI_IDX2INTID(idx) ((idx) + ESPI_BASE_INTID)
>> +#endif
>> +
>> /* LPIs are always numbered starting at 8192, so 0 is a good invalid case. */
>> #define INVALID_LPI 0
>>
>> @@ -39,7 +47,15 @@ struct arch_irq_desc {
>> #define INVALID_IRQ 1023
>>
>> extern const unsigned int nr_irqs;
>> +#ifdef CONFIG_GICV3_ESPI
>> +/*
>> + * This will also cover the eSPI range, as some critical devices
>> + * for booting Xen (e.g., serial) may use this type of interrupts.
>> + */
>> +#define nr_static_irqs (ESPI_MAX_INTID + 1)
>> +#else
>> #define nr_static_irqs NR_IRQS
>> +#endif
>
> Don't introduce defines that look like variables. I am sure that MISRA
> team will be unhappy about that. But what you can really do is to
> introduce variable nr_static_irqs, which value will depend on
> GICD_TYPER.ESPI and GICD_TYPER.ESPI_range
>
This is a generic define, which is present in all arch-specific irq
headers (x86, PPC, RISC-V, ARM). I think it is not a good idea to make
this a const variable only when CONFIG_GICV3_ESPI is enabled, while in
other places it is a define. Moreover, I clarified with one of our
teammates, who is engaged in MISRA fixes for Xen, and he confirmed that
there is no violation of the MISRA rule here. To summarize all of that,
would it be okay to leave this place unchanged?
>>
>> struct irq_desc;
>> struct irqaction;
>> @@ -55,6 +71,15 @@ static inline bool is_lpi(unsigned int irq)
>> return irq >= LPI_OFFSET;
>> }
>>
>> +static inline bool is_espi(unsigned int irq)
>> +{
>> +#ifdef CONFIG_GICV3_ESPI
>> + return (irq >= ESPI_BASE_INTID && irq <= ESPI_MAX_INTID);
>> +#else
>> + return false;
>> +#endif
>> +}
>> +
>> #define domain_pirq_to_irq(d, pirq) (pirq)
>>
>> bool is_assignable_irq(unsigned int irq);
>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>> index 50e57aaea7..9bc72fbbc9 100644
>> --- a/xen/arch/arm/irq.c
>> +++ b/xen/arch/arm/irq.c
>> @@ -19,7 +19,11 @@
>> #include <asm/gic.h>
>> #include <asm/vgic.h>
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> +const unsigned int nr_irqs = ESPI_MAX_INTID + 1;
>> +#else
>> const unsigned int nr_irqs = NR_IRQS;
>> +#endif
>>
>> static unsigned int local_irqs_type[NR_LOCAL_IRQS];
>> static DEFINE_SPINLOCK(local_irqs_type_lock);
>> @@ -46,6 +50,9 @@ void irq_end_none(struct irq_desc *irq)
>> }
>>
>> static irq_desc_t irq_desc[NR_IRQS - NR_LOCAL_IRQS];
>> +#ifdef CONFIG_GICV3_ESPI
>> +static irq_desc_t espi_desc[NR_IRQS];
>
> This is really confusing. Should it be something like espi_desc[NR_ESPI_IRQS]?
>
Sure, that would be better. So, I will add the NR_ESPI_IRQS define in
V3. Of course, if I am not able to make a dynamic allocation, which is
being discussed in the sub-thread for this patch. :)
>> +#endif
>
>> static DEFINE_PER_CPU(irq_desc_t[NR_LOCAL_IRQS], local_irq_desc);
>>
>> struct irq_desc *__irq_to_desc(unsigned int irq)
>> @@ -53,6 +60,11 @@ struct irq_desc *__irq_to_desc(unsigned int irq)
>> if ( irq < NR_LOCAL_IRQS )
>> return &this_cpu(local_irq_desc)[irq];
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> + if ( is_espi(irq) )
>> + return &espi_desc[ESPI_INTID2IDX(irq)];
>> +#endif
>> +
>> return &irq_desc[irq-NR_LOCAL_IRQS];
>> }
>>
>> @@ -79,6 +91,20 @@ static int __init init_irq_data(void)
>> desc->action = NULL;
>> }
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> + for ( irq = ESPI_BASE_INTID; irq <= ESPI_MAX_INTID; irq++ )
>> + {
>> + struct irq_desc *desc = irq_to_desc(irq);
>> + int rc = init_one_irq_desc(desc);
>> +
>> + if ( rc )
>> + return rc;
>> +
>> + desc->irq = irq;
>> + desc->action = NULL;
>> + }
>> +#endif
>> +
>> return 0;
>> }
>
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread
* [PATCH v2 07/10] xen/arm: gicv3: modify ICH_LR_PHYSICAL_MASK to allow eSPI processing
2025-08-07 12:33 ` [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations Leonid Komarianskyi
` (3 preceding siblings ...)
2025-08-07 12:33 ` [PATCH v2 04/10] xen/arm/irq: add handling for IRQs in the eSPI range Leonid Komarianskyi
@ 2025-08-07 12:33 ` Leonid Komarianskyi
2025-08-21 16:27 ` Volodymyr Babchuk
2025-08-07 12:33 ` [PATCH v2 06/10] xen/arm/irq: allow eSPI processing in the do_IRQ function Leonid Komarianskyi
` (5 subsequent siblings)
10 siblings, 1 reply; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-07 12:33 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Leonid Komarianskyi, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk
To properly deactivate guest interrupts and allow them to be retriggered
after the initial trigger, the LR needs to be updated. The current
implementation ignores interrupts outside the range specified by the mask
0x3FF, which only covers IRQ numbers up to 1023. To enable processing of
eSPI interrupts, this patch updates the mask to 0x13FF.
Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
---
Changes in V2:
- remove unnecessary CONFIG_GICV3_ESPI ifdef guard
---
xen/arch/arm/include/asm/gic_v3_defs.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/xen/arch/arm/include/asm/gic_v3_defs.h b/xen/arch/arm/include/asm/gic_v3_defs.h
index 7f769b38e3..4f27708a70 100644
--- a/xen/arch/arm/include/asm/gic_v3_defs.h
+++ b/xen/arch/arm/include/asm/gic_v3_defs.h
@@ -206,7 +206,7 @@
#define ICH_LR_VIRTUAL_SHIFT 0
#define ICH_LR_CPUID_MASK 0x7
#define ICH_LR_CPUID_SHIFT 10
-#define ICH_LR_PHYSICAL_MASK 0x3ff
+#define ICH_LR_PHYSICAL_MASK 0x13ff
#define ICH_LR_PHYSICAL_SHIFT 32
#define ICH_LR_STATE_MASK 0x3
#define ICH_LR_STATE_SHIFT 62
--
2.34.1
^ permalink raw reply related [flat|nested] 55+ messages in thread* Re: [PATCH v2 07/10] xen/arm: gicv3: modify ICH_LR_PHYSICAL_MASK to allow eSPI processing
2025-08-07 12:33 ` [PATCH v2 07/10] xen/arm: gicv3: modify ICH_LR_PHYSICAL_MASK to allow eSPI processing Leonid Komarianskyi
@ 2025-08-21 16:27 ` Volodymyr Babchuk
2025-08-22 6:56 ` Leonid Komarianskyi
0 siblings, 1 reply; 55+ messages in thread
From: Volodymyr Babchuk @ 2025-08-21 16:27 UTC (permalink / raw)
To: Leonid Komarianskyi
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Hi Leonid,
Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
> To properly deactivate guest interrupts and allow them to be retriggered
> after the initial trigger, the LR needs to be updated. The current
> implementation ignores interrupts outside the range specified by the mask
> 0x3FF, which only covers IRQ numbers up to 1023. To enable processing of
> eSPI interrupts, this patch updates the mask to 0x13FF.
I am not sure how this supposed to work. According to the arch
specification, pINTID field in ICH_LR<n>_EL2 is only 12 bits wide.
--
WBR, Volodymyr
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v2 07/10] xen/arm: gicv3: modify ICH_LR_PHYSICAL_MASK to allow eSPI processing
2025-08-21 16:27 ` Volodymyr Babchuk
@ 2025-08-22 6:56 ` Leonid Komarianskyi
0 siblings, 0 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-22 6:56 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Hi Volodymyr,
On 21.08.25 19:27, Volodymyr Babchuk wrote:
> Hi Leonid,
>
>
> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>
>> To properly deactivate guest interrupts and allow them to be retriggered
>> after the initial trigger, the LR needs to be updated. The current
>> implementation ignores interrupts outside the range specified by the mask
>> 0x3FF, which only covers IRQ numbers up to 1023. To enable processing of
>> eSPI interrupts, this patch updates the mask to 0x13FF.
>
> I am not sure how this supposed to work. According to the arch
> specification, pINTID field in ICH_LR<n>_EL2 is only 12 bits wide.
>
>
Thank you for your review.
According to the specification, the pINTID field in ICH_LR<n>_EL2 is 13
bits wide ([44:32] bits):
> A hardware physical identifier is only required in List Registers for
> interrupts that require deactivation. This means only 13 bits of
> Physical INTID are required, regardless of the number specified by
> ICC_CTLR_EL1.IDbits.
0x13FF is a 13-bit number that fits into the corresponding register
field and covers the maximum eSPI INTID value - 5119.
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread
* [PATCH v2 06/10] xen/arm/irq: allow eSPI processing in the do_IRQ function
2025-08-07 12:33 ` [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations Leonid Komarianskyi
` (4 preceding siblings ...)
2025-08-07 12:33 ` [PATCH v2 07/10] xen/arm: gicv3: modify ICH_LR_PHYSICAL_MASK to allow eSPI processing Leonid Komarianskyi
@ 2025-08-07 12:33 ` Leonid Komarianskyi
2025-08-07 12:33 ` [PATCH v2 08/10] xen/arm: vgic: add resource management for extended SPIs Leonid Komarianskyi
` (4 subsequent siblings)
10 siblings, 0 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-07 12:33 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Leonid Komarianskyi, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk
The do_IRQ() function is the main handler for processing IRQs.
Currently, due to restrictive checks, it does not process interrupt
numbers greater than 1024. This patch updates the condition to allow
the handling of interrupts from the eSPI range.
Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
---
Changes in V2:
- no changes
---
xen/arch/arm/gic.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 47fccf21d8..7186c3d40a 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -341,7 +341,7 @@ void gic_interrupt(struct cpu_user_regs *regs, int is_fiq)
/* Reading IRQ will ACK it */
irq = gic_hw_ops->read_irq();
- if ( likely(irq >= GIC_SGI_STATIC_MAX && irq < 1020) )
+ if ( likely(irq >= GIC_SGI_STATIC_MAX && irq < 1020) || is_espi(irq) )
{
isb();
do_IRQ(regs, irq, is_fiq);
--
2.34.1
^ permalink raw reply related [flat|nested] 55+ messages in thread* [PATCH v2 08/10] xen/arm: vgic: add resource management for extended SPIs
2025-08-07 12:33 ` [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations Leonid Komarianskyi
` (5 preceding siblings ...)
2025-08-07 12:33 ` [PATCH v2 06/10] xen/arm/irq: allow eSPI processing in the do_IRQ function Leonid Komarianskyi
@ 2025-08-07 12:33 ` Leonid Komarianskyi
2025-08-21 16:43 ` Volodymyr Babchuk
2025-08-23 14:39 ` Oleksandr Tyshchenko
2025-08-07 12:33 ` [PATCH v2 10/10] xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers Leonid Komarianskyi
` (3 subsequent siblings)
10 siblings, 2 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-07 12:33 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Leonid Komarianskyi, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk
This change introduces resource management in the VGIC to handle
extended SPIs introduced in GICv3.1. The pending_irqs and
allocated_irqs arrays are resized to support the required
number of eSPIs, based on what is supported by the hardware and
requested by the guest. A new field, ext_shared_irqs, is added
to the VGIC structure to store information about eSPIs, similar
to how shared_irqs is used for regular SPIs.
Since the eSPI range starts at INTID 4096 and INTIDs between 1025
and 4095 are reserved, helper macros are introduced to simplify the
transformation of indices and to enable easier access to eSPI-specific
resources. These changes prepare the VGIC for processing eSPIs as
required by future functionality.
The initialization and deinitialization paths for vgic have been updated
to allocate and free these resources appropriately. Additionally,
updated handling of INTIDs greater than 1024, passed from the toolstack
during domain creation, and verification logic ensures only valid SPI or
eSPI INTIDs are used.
The existing SPI behavior remains unaffected when guests do not request
eSPIs, GIC hardware does not support them, or the CONFIG_GICV3_ESPI
option is disabled.
Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
---
Changes in V2:
- change is_espi_rank to is_valid_espi_rank to verify whether the array
element ext_shared_irqs exists. The previous version, is_espi_rank,
only checked if the rank index was less than the maximum possible eSPI
rank index, but this could potentially result in accessing a
non-existing array element. To address this, is_valid_espi_rank was
introduced, which ensures that the required eSPI rank exists
- move gic_number_espis to
xen/arm: gicv3: implement handling of GICv3.1 eSPI
- update vgic_is_valid_irq checks to allow operating with eSPIs
- remove redundant newline in vgic_allocate_virq
---
xen/arch/arm/include/asm/vgic.h | 18 ++++
xen/arch/arm/vgic.c | 145 ++++++++++++++++++++++++++++++++
2 files changed, 163 insertions(+)
diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
index 45201f4ca5..9fa4523018 100644
--- a/xen/arch/arm/include/asm/vgic.h
+++ b/xen/arch/arm/include/asm/vgic.h
@@ -146,6 +146,10 @@ struct vgic_dist {
int nr_spis; /* Number of SPIs */
unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
struct vgic_irq_rank *shared_irqs;
+#ifdef CONFIG_GICV3_ESPI
+ struct vgic_irq_rank *ext_shared_irqs;
+ int nr_espis; /* Number of extended SPIs */
+#endif
/*
* SPIs are domain global, SGIs and PPIs are per-VCPU and stored in
* struct arch_vcpu.
@@ -243,6 +247,14 @@ struct vgic_ops {
/* Number of ranks of interrupt registers for a domain */
#define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_spis+31)/32)
+#ifdef CONFIG_GICV3_ESPI
+#define DOMAIN_NR_EXT_RANKS(d) (((d)->arch.vgic.nr_espis+31)/32)
+#define EXT_RANK_MIN (ESPI_BASE_INTID/32)
+#define EXT_RANK_MAX ((ESPI_MAX_INTID+31)/32)
+#define EXT_RANK_NUM2IDX(num) ((num)-EXT_RANK_MIN)
+#define EXT_RANK_IDX2NUM(idx) ((idx)+EXT_RANK_MIN)
+#endif
+
#define vgic_lock(v) spin_lock_irq(&(v)->domain->arch.vgic.lock)
#define vgic_unlock(v) spin_unlock_irq(&(v)->domain->arch.vgic.lock)
@@ -302,6 +314,12 @@ extern struct vgic_irq_rank *vgic_rank_offset(struct vcpu *v,
unsigned int b,
unsigned int n,
unsigned int s);
+#ifdef CONFIG_GICV3_ESPI
+extern struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v,
+ unsigned int b,
+ unsigned int n,
+ unsigned int s);
+#endif
extern struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq);
extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned int n);
extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, unsigned int n);
diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
index 48fbaf56fb..1a6c765af9 100644
--- a/xen/arch/arm/vgic.c
+++ b/xen/arch/arm/vgic.c
@@ -27,9 +27,26 @@
bool vgic_is_valid_irq(struct domain *d, unsigned int virq)
{
+#ifdef CONFIG_GICV3_ESPI
+ if ( virq >= ESPI_BASE_INTID && virq < ESPI_IDX2INTID(d->arch.vgic.nr_espis) )
+ return true;
+#endif
+
return virq < vgic_num_irqs(d);
}
+#ifdef CONFIG_GICV3_ESPI
+/*
+ * Since eSPI indexes start from 4096 and numbers from 1024 to
+ * 4095 are forbidden, we need to check both lower and upper
+ * limits for ranks.
+ */
+static inline bool is_valid_espi_rank(struct domain *d, unsigned int rank)
+{
+ return ( rank >= EXT_RANK_MIN && EXT_RANK_NUM2IDX(rank) < DOMAIN_NR_EXT_RANKS(d) );
+}
+#endif
+
static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
unsigned int rank)
{
@@ -37,6 +54,10 @@ static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
return v->arch.vgic.private_irqs;
else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
return &v->domain->arch.vgic.shared_irqs[rank - 1];
+#ifdef CONFIG_GICV3_ESPI
+ else if ( is_valid_espi_rank(v->domain, rank) )
+ return &v->domain->arch.vgic.ext_shared_irqs[EXT_RANK_NUM2IDX(rank)];
+#endif
else
return NULL;
}
@@ -53,6 +74,16 @@ struct vgic_irq_rank *vgic_rank_offset(struct vcpu *v, unsigned int b,
return vgic_get_rank(v, rank);
}
+#ifdef CONFIG_GICV3_ESPI
+struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v, unsigned int b,
+ unsigned int n, unsigned int s)
+{
+ unsigned int rank = REG_RANK_NR(b, (n >> s));
+
+ return vgic_get_rank(v, rank + EXT_RANK_MIN);
+}
+#endif
+
struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq)
{
unsigned int rank = irq / 32;
@@ -117,6 +148,29 @@ int domain_vgic_register(struct domain *d, unsigned int *mmio_count)
return 0;
}
+#ifdef CONFIG_GICV3_ESPI
+static int init_vgic_espi(struct domain *d)
+{
+ int i;
+
+ if ( d->arch.vgic.nr_espis == 0 )
+ return 0;
+
+ d->arch.vgic.ext_shared_irqs =
+ xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_EXT_RANKS(d));
+ if ( d->arch.vgic.ext_shared_irqs == NULL )
+ return -ENOMEM;
+
+ for ( i = 0; i < d->arch.vgic.nr_espis; i++ )
+ vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i + d->arch.vgic.nr_spis], ESPI_IDX2INTID(i));
+
+ for ( i = 0; i < DOMAIN_NR_EXT_RANKS(d); i++ )
+ vgic_rank_init(&d->arch.vgic.ext_shared_irqs[i], i, 0);
+
+ return 0;
+}
+#endif
+
int domain_vgic_init(struct domain *d, unsigned int nr_spis)
{
int i;
@@ -131,6 +185,30 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
*/
nr_spis = ROUNDUP(nr_spis, 32);
+#ifdef CONFIG_GICV3_ESPI
+ if ( nr_spis > ESPI_MAX_INTID )
+ return -EINVAL;
+
+ if ( is_espi(nr_spis) )
+ {
+ /*
+ * During domain creation, the toolstack specifies the maximum INTID,
+ * which is defined in the domain config subtracted by 32. To compute the
+ * actual number of eSPI that will be usable for, add back 32.
+ */
+ d->arch.vgic.nr_espis = min(nr_spis - ESPI_BASE_INTID + 32, 1024U);
+ /* Verify if GIC HW can handle provided INTID */
+ if ( d->arch.vgic.nr_espis > gic_number_espis() )
+ return -EINVAL;
+ /* Set the maximum available number for defult SPI to pass the next check */
+ nr_spis = VGIC_DEF_NR_SPIS;
+ } else
+ {
+ /* Domain will use the regular SPI range */
+ d->arch.vgic.nr_espis = 0;
+ }
+#endif
+
/* Limit the number of virtual SPIs supported to (1020 - 32) = 988 */
if ( nr_spis > (1020 - NR_LOCAL_IRQS) )
return -EINVAL;
@@ -145,7 +223,12 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
return -ENOMEM;
d->arch.vgic.pending_irqs =
+#ifdef CONFIG_GICV3_ESPI
+ xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis +
+ d->arch.vgic.nr_espis);
+#else
xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis);
+#endif
if ( d->arch.vgic.pending_irqs == NULL )
return -ENOMEM;
@@ -156,12 +239,23 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
for ( i = 0; i < DOMAIN_NR_RANKS(d); i++ )
vgic_rank_init(&d->arch.vgic.shared_irqs[i], i + 1, 0);
+#ifdef CONFIG_GICV3_ESPI
+ ret = init_vgic_espi(d);
+ if ( ret )
+ return ret;
+#endif
+
ret = d->arch.vgic.handler->domain_init(d);
if ( ret )
return ret;
d->arch.vgic.allocated_irqs =
+#ifdef CONFIG_GICV3_ESPI
+ xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d) +
+ d->arch.vgic.nr_espis));
+#else
xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d)));
+#endif
if ( !d->arch.vgic.allocated_irqs )
return -ENOMEM;
@@ -195,9 +289,27 @@ void domain_vgic_free(struct domain *d)
}
}
+#ifdef CONFIG_GICV3_ESPI
+ for ( i = 0; i < (d->arch.vgic.nr_espis); i++ )
+ {
+ struct pending_irq *p = spi_to_pending(d, ESPI_IDX2INTID(i));
+
+ if ( p->desc )
+ {
+ ret = release_guest_irq(d, p->irq);
+ if ( ret )
+ dprintk(XENLOG_G_WARNING, "d%u: Failed to release virq %u ret = %d\n",
+ d->domain_id, p->irq, ret);
+ }
+ }
+#endif
+
if ( d->arch.vgic.handler )
d->arch.vgic.handler->domain_free(d);
xfree(d->arch.vgic.shared_irqs);
+#ifdef CONFIG_GICV3_ESPI
+ xfree(d->arch.vgic.ext_shared_irqs);
+#endif
xfree(d->arch.vgic.pending_irqs);
xfree(d->arch.vgic.allocated_irqs);
}
@@ -331,6 +443,17 @@ void arch_move_irqs(struct vcpu *v)
if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) )
irq_set_affinity(p->desc, cpu_mask);
}
+
+#ifdef CONFIG_GICV3_ESPI
+ for ( i = ESPI_BASE_INTID; i < (d)->arch.vgic.nr_espis; i++ )
+ {
+ v_target = vgic_get_target_vcpu(v, i);
+ p = irq_to_pending(v_target, i);
+
+ if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) )
+ irq_set_affinity(p->desc, cpu_mask);
+ }
+#endif
}
void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned int n)
@@ -538,6 +661,10 @@ struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
n = &v->arch.vgic.pending_irqs[irq];
else if ( is_lpi(irq) )
n = v->domain->arch.vgic.handler->lpi_to_pending(v->domain, irq);
+#ifdef CONFIG_GICV3_ESPI
+ else if ( is_espi(irq) )
+ n = &v->domain->arch.vgic.pending_irqs[ESPI_INTID2IDX(irq) + v->domain->arch.vgic.nr_spis];
+#endif
else
n = &v->domain->arch.vgic.pending_irqs[irq - 32];
return n;
@@ -547,6 +674,14 @@ struct pending_irq *spi_to_pending(struct domain *d, unsigned int irq)
{
ASSERT(irq >= NR_LOCAL_IRQS);
+#ifdef CONFIG_GICV3_ESPI
+ if ( is_espi(irq) )
+ {
+ irq = ESPI_INTID2IDX(irq) + d->arch.vgic.nr_spis;
+ return &d->arch.vgic.pending_irqs[irq];
+ }
+#endif
+
return &d->arch.vgic.pending_irqs[irq - 32];
}
@@ -668,6 +803,11 @@ bool vgic_reserve_virq(struct domain *d, unsigned int virq)
if ( !vgic_is_valid_irq(d, virq) )
return false;
+#ifdef CONFIG_GICV3_ESPI
+ if ( is_espi(virq) )
+ return !test_and_set_bit(ESPI_INTID2IDX(virq) + vgic_num_irqs(d), d->arch.vgic.allocated_irqs);
+#endif
+
return !test_and_set_bit(virq, d->arch.vgic.allocated_irqs);
}
@@ -686,6 +826,11 @@ int vgic_allocate_virq(struct domain *d, bool spi)
{
first = 32;
end = vgic_num_irqs(d);
+#ifdef CONFIG_GICV3_ESPI
+ /* Take into account extended SPI range */
+ end += d->arch.vgic.nr_espis;
+#endif
+
}
/*
--
2.34.1
^ permalink raw reply related [flat|nested] 55+ messages in thread* Re: [PATCH v2 08/10] xen/arm: vgic: add resource management for extended SPIs
2025-08-07 12:33 ` [PATCH v2 08/10] xen/arm: vgic: add resource management for extended SPIs Leonid Komarianskyi
@ 2025-08-21 16:43 ` Volodymyr Babchuk
2025-08-22 8:27 ` Leonid Komarianskyi
2025-08-23 14:39 ` Oleksandr Tyshchenko
1 sibling, 1 reply; 55+ messages in thread
From: Volodymyr Babchuk @ 2025-08-21 16:43 UTC (permalink / raw)
To: Leonid Komarianskyi
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
> This change introduces resource management in the VGIC to handle
> extended SPIs introduced in GICv3.1. The pending_irqs and
> allocated_irqs arrays are resized to support the required
> number of eSPIs, based on what is supported by the hardware and
> requested by the guest. A new field, ext_shared_irqs, is added
> to the VGIC structure to store information about eSPIs, similar
> to how shared_irqs is used for regular SPIs.
>
> Since the eSPI range starts at INTID 4096 and INTIDs between 1025
> and 4095 are reserved, helper macros are introduced to simplify the
> transformation of indices and to enable easier access to eSPI-specific
> resources. These changes prepare the VGIC for processing eSPIs as
> required by future functionality.
>
> The initialization and deinitialization paths for vgic have been updated
> to allocate and free these resources appropriately. Additionally,
> updated handling of INTIDs greater than 1024, passed from the toolstack
> during domain creation, and verification logic ensures only valid SPI or
> eSPI INTIDs are used.
>
> The existing SPI behavior remains unaffected when guests do not request
> eSPIs, GIC hardware does not support them, or the CONFIG_GICV3_ESPI
> option is disabled.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>
> ---
> Changes in V2:
> - change is_espi_rank to is_valid_espi_rank to verify whether the array
> element ext_shared_irqs exists. The previous version, is_espi_rank,
> only checked if the rank index was less than the maximum possible eSPI
> rank index, but this could potentially result in accessing a
> non-existing array element. To address this, is_valid_espi_rank was
> introduced, which ensures that the required eSPI rank exists
> - move gic_number_espis to
> xen/arm: gicv3: implement handling of GICv3.1 eSPI
> - update vgic_is_valid_irq checks to allow operating with eSPIs
> - remove redundant newline in vgic_allocate_virq
> ---
> xen/arch/arm/include/asm/vgic.h | 18 ++++
> xen/arch/arm/vgic.c | 145 ++++++++++++++++++++++++++++++++
> 2 files changed, 163 insertions(+)
>
> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
> index 45201f4ca5..9fa4523018 100644
> --- a/xen/arch/arm/include/asm/vgic.h
> +++ b/xen/arch/arm/include/asm/vgic.h
> @@ -146,6 +146,10 @@ struct vgic_dist {
> int nr_spis; /* Number of SPIs */
> unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
> struct vgic_irq_rank *shared_irqs;
> +#ifdef CONFIG_GICV3_ESPI
> + struct vgic_irq_rank *ext_shared_irqs;
> + int nr_espis; /* Number of extended SPIs */
> +#endif
> /*
> * SPIs are domain global, SGIs and PPIs are per-VCPU and stored in
> * struct arch_vcpu.
> @@ -243,6 +247,14 @@ struct vgic_ops {
> /* Number of ranks of interrupt registers for a domain */
> #define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_spis+31)/32)
>
> +#ifdef CONFIG_GICV3_ESPI
> +#define DOMAIN_NR_EXT_RANKS(d) (((d)->arch.vgic.nr_espis+31)/32)
> +#define EXT_RANK_MIN (ESPI_BASE_INTID/32)
> +#define EXT_RANK_MAX ((ESPI_MAX_INTID+31)/32)
> +#define EXT_RANK_NUM2IDX(num) ((num)-EXT_RANK_MIN)
> +#define EXT_RANK_IDX2NUM(idx) ((idx)+EXT_RANK_MIN)
> +#endif
> +
> #define vgic_lock(v) spin_lock_irq(&(v)->domain->arch.vgic.lock)
> #define vgic_unlock(v) spin_unlock_irq(&(v)->domain->arch.vgic.lock)
>
> @@ -302,6 +314,12 @@ extern struct vgic_irq_rank *vgic_rank_offset(struct vcpu *v,
> unsigned int b,
> unsigned int n,
> unsigned int s);
> +#ifdef CONFIG_GICV3_ESPI
> +extern struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v,
> + unsigned int b,
> + unsigned int n,
> + unsigned int s);
> +#endif
> extern struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq);
> extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned int n);
> extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, unsigned int n);
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 48fbaf56fb..1a6c765af9 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -27,9 +27,26 @@
>
> bool vgic_is_valid_irq(struct domain *d, unsigned int virq)
> {
> +#ifdef CONFIG_GICV3_ESPI
> + if ( virq >= ESPI_BASE_INTID && virq < ESPI_IDX2INTID(d->arch.vgic.nr_espis) )
> + return true;
> +#endif
> +
> return virq < vgic_num_irqs(d);
> }
>
> +#ifdef CONFIG_GICV3_ESPI
> +/*
> + * Since eSPI indexes start from 4096 and numbers from 1024 to
> + * 4095 are forbidden, we need to check both lower and upper
> + * limits for ranks.
> + */
> +static inline bool is_valid_espi_rank(struct domain *d, unsigned int rank)
> +{
> + return ( rank >= EXT_RANK_MIN && EXT_RANK_NUM2IDX(rank) < DOMAIN_NR_EXT_RANKS(d) );
I am pretty sure that max line length in 80 symbols, and here you have
more. As well in couple of other places below.
> +}
> +#endif
> +
> static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
> unsigned int rank)
> {
> @@ -37,6 +54,10 @@ static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
> return v->arch.vgic.private_irqs;
> else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
> return &v->domain->arch.vgic.shared_irqs[rank - 1];
> +#ifdef CONFIG_GICV3_ESPI
> + else if ( is_valid_espi_rank(v->domain, rank) )
> + return &v->domain->arch.vgic.ext_shared_irqs[EXT_RANK_NUM2IDX(rank)];
> +#endif
> else
> return NULL;
> }
> @@ -53,6 +74,16 @@ struct vgic_irq_rank *vgic_rank_offset(struct vcpu *v, unsigned int b,
> return vgic_get_rank(v, rank);
> }
>
> +#ifdef CONFIG_GICV3_ESPI
> +struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v, unsigned int b,
> + unsigned int n, unsigned int s)
> +{
> + unsigned int rank = REG_RANK_NR(b, (n >> s));
> +
> + return vgic_get_rank(v, rank + EXT_RANK_MIN);
> +}
> +#endif
> +
> struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq)
> {
> unsigned int rank = irq / 32;
> @@ -117,6 +148,29 @@ int domain_vgic_register(struct domain *d, unsigned int *mmio_count)
> return 0;
> }
>
> +#ifdef CONFIG_GICV3_ESPI
> +static int init_vgic_espi(struct domain *d)
> +{
> + int i;
> +
> + if ( d->arch.vgic.nr_espis == 0 )
> + return 0;
> +
> + d->arch.vgic.ext_shared_irqs =
> + xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_EXT_RANKS(d));
> + if ( d->arch.vgic.ext_shared_irqs == NULL )
> + return -ENOMEM;
> +
> + for ( i = 0; i < d->arch.vgic.nr_espis; i++ )
> + vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i + d->arch.vgic.nr_spis], ESPI_IDX2INTID(i));
> +
> + for ( i = 0; i < DOMAIN_NR_EXT_RANKS(d); i++ )
> + vgic_rank_init(&d->arch.vgic.ext_shared_irqs[i], i, 0);
> +
> + return 0;
> +}
> +#endif
> +
> int domain_vgic_init(struct domain *d, unsigned int nr_spis)
> {
> int i;
> @@ -131,6 +185,30 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
> */
> nr_spis = ROUNDUP(nr_spis, 32);
>
> +#ifdef CONFIG_GICV3_ESPI
> + if ( nr_spis > ESPI_MAX_INTID )
> + return -EINVAL;
> +
> + if ( is_espi(nr_spis) )
> + {
> + /*
> + * During domain creation, the toolstack specifies the maximum INTID,
> + * which is defined in the domain config subtracted by 32. To compute the
> + * actual number of eSPI that will be usable for, add back 32.
I think, according to this, your if ( is_espi() ) check is wrong. Should
you add 32 here as well?
> + */
> + d->arch.vgic.nr_espis = min(nr_spis - ESPI_BASE_INTID + 32, 1024U);
> + /* Verify if GIC HW can handle provided INTID */
> + if ( d->arch.vgic.nr_espis > gic_number_espis() )
> + return -EINVAL;
> + /* Set the maximum available number for defult SPI to pass the next check */
> + nr_spis = VGIC_DEF_NR_SPIS;
> + } else
> + {
> + /* Domain will use the regular SPI range */
> + d->arch.vgic.nr_espis = 0;
> + }
> +#endif
> +
> /* Limit the number of virtual SPIs supported to (1020 - 32) = 988 */
> if ( nr_spis > (1020 - NR_LOCAL_IRQS) )
> return -EINVAL;
> @@ -145,7 +223,12 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
> return -ENOMEM;
>
> d->arch.vgic.pending_irqs =
> +#ifdef CONFIG_GICV3_ESPI
> + xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis +
> + d->arch.vgic.nr_espis);
> +#else
> xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis);
> +#endif
> if ( d->arch.vgic.pending_irqs == NULL )
> return -ENOMEM;
>
> @@ -156,12 +239,23 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
> for ( i = 0; i < DOMAIN_NR_RANKS(d); i++ )
> vgic_rank_init(&d->arch.vgic.shared_irqs[i], i + 1, 0);
>
> +#ifdef CONFIG_GICV3_ESPI
> + ret = init_vgic_espi(d);
> + if ( ret )
> + return ret;
> +#endif
> +
> ret = d->arch.vgic.handler->domain_init(d);
> if ( ret )
> return ret;
>
> d->arch.vgic.allocated_irqs =
> +#ifdef CONFIG_GICV3_ESPI
> + xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d) +
> + d->arch.vgic.nr_espis));
> +#else
> xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d)));
> +#endif
> if ( !d->arch.vgic.allocated_irqs )
> return -ENOMEM;
>
> @@ -195,9 +289,27 @@ void domain_vgic_free(struct domain *d)
> }
> }
>
> +#ifdef CONFIG_GICV3_ESPI
> + for ( i = 0; i < (d->arch.vgic.nr_espis); i++ )
> + {
> + struct pending_irq *p = spi_to_pending(d, ESPI_IDX2INTID(i));
> +
> + if ( p->desc )
> + {
> + ret = release_guest_irq(d, p->irq);
> + if ( ret )
> + dprintk(XENLOG_G_WARNING, "d%u: Failed to release virq %u ret = %d\n",
> + d->domain_id, p->irq, ret);
> + }
> + }
> +#endif
> +
> if ( d->arch.vgic.handler )
> d->arch.vgic.handler->domain_free(d);
> xfree(d->arch.vgic.shared_irqs);
> +#ifdef CONFIG_GICV3_ESPI
> + xfree(d->arch.vgic.ext_shared_irqs);
> +#endif
> xfree(d->arch.vgic.pending_irqs);
> xfree(d->arch.vgic.allocated_irqs);
> }
> @@ -331,6 +443,17 @@ void arch_move_irqs(struct vcpu *v)
> if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) )
> irq_set_affinity(p->desc, cpu_mask);
> }
> +
> +#ifdef CONFIG_GICV3_ESPI
> + for ( i = ESPI_BASE_INTID; i < (d)->arch.vgic.nr_espis; i++ )
> + {
> + v_target = vgic_get_target_vcpu(v, i);
> + p = irq_to_pending(v_target, i);
> +
> + if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) )
> + irq_set_affinity(p->desc, cpu_mask);
> + }
> +#endif
> }
>
> void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned int n)
> @@ -538,6 +661,10 @@ struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
> n = &v->arch.vgic.pending_irqs[irq];
> else if ( is_lpi(irq) )
> n = v->domain->arch.vgic.handler->lpi_to_pending(v->domain, irq);
> +#ifdef CONFIG_GICV3_ESPI
> + else if ( is_espi(irq) )
> + n = &v->domain->arch.vgic.pending_irqs[ESPI_INTID2IDX(irq) + v->domain->arch.vgic.nr_spis];
> +#endif
> else
> n = &v->domain->arch.vgic.pending_irqs[irq - 32];
> return n;
> @@ -547,6 +674,14 @@ struct pending_irq *spi_to_pending(struct domain *d, unsigned int irq)
> {
> ASSERT(irq >= NR_LOCAL_IRQS);
>
> +#ifdef CONFIG_GICV3_ESPI
> + if ( is_espi(irq) )
> + {
> + irq = ESPI_INTID2IDX(irq) + d->arch.vgic.nr_spis;
> + return &d->arch.vgic.pending_irqs[irq];
> + }
> +#endif
> +
> return &d->arch.vgic.pending_irqs[irq - 32];
> }
>
> @@ -668,6 +803,11 @@ bool vgic_reserve_virq(struct domain *d, unsigned int virq)
> if ( !vgic_is_valid_irq(d, virq) )
> return false;
>
> +#ifdef CONFIG_GICV3_ESPI
> + if ( is_espi(virq) )
> + return !test_and_set_bit(ESPI_INTID2IDX(virq) + vgic_num_irqs(d), d->arch.vgic.allocated_irqs);
> +#endif
> +
> return !test_and_set_bit(virq, d->arch.vgic.allocated_irqs);
> }
>
> @@ -686,6 +826,11 @@ int vgic_allocate_virq(struct domain *d, bool spi)
> {
> first = 32;
> end = vgic_num_irqs(d);
> +#ifdef CONFIG_GICV3_ESPI
> + /* Take into account extended SPI range */
> + end += d->arch.vgic.nr_espis;
> +#endif
> +
> }
>
> /*
--
WBR, Volodymyr
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 08/10] xen/arm: vgic: add resource management for extended SPIs
2025-08-21 16:43 ` Volodymyr Babchuk
@ 2025-08-22 8:27 ` Leonid Komarianskyi
0 siblings, 0 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-22 8:27 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Hi Volodymyr,
Thank you for your review comments.
On 21.08.25 19:43, Volodymyr Babchuk wrote:
>
> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>
>> This change introduces resource management in the VGIC to handle
>> extended SPIs introduced in GICv3.1. The pending_irqs and
>> allocated_irqs arrays are resized to support the required
>> number of eSPIs, based on what is supported by the hardware and
>> requested by the guest. A new field, ext_shared_irqs, is added
>> to the VGIC structure to store information about eSPIs, similar
>> to how shared_irqs is used for regular SPIs.
>>
>> Since the eSPI range starts at INTID 4096 and INTIDs between 1025
>> and 4095 are reserved, helper macros are introduced to simplify the
>> transformation of indices and to enable easier access to eSPI-specific
>> resources. These changes prepare the VGIC for processing eSPIs as
>> required by future functionality.
>>
>> The initialization and deinitialization paths for vgic have been updated
>> to allocate and free these resources appropriately. Additionally,
>> updated handling of INTIDs greater than 1024, passed from the toolstack
>> during domain creation, and verification logic ensures only valid SPI or
>> eSPI INTIDs are used.
>>
>> The existing SPI behavior remains unaffected when guests do not request
>> eSPIs, GIC hardware does not support them, or the CONFIG_GICV3_ESPI
>> option is disabled.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V2:
>> - change is_espi_rank to is_valid_espi_rank to verify whether the array
>> element ext_shared_irqs exists. The previous version, is_espi_rank,
>> only checked if the rank index was less than the maximum possible eSPI
>> rank index, but this could potentially result in accessing a
>> non-existing array element. To address this, is_valid_espi_rank was
>> introduced, which ensures that the required eSPI rank exists
>> - move gic_number_espis to
>> xen/arm: gicv3: implement handling of GICv3.1 eSPI
>> - update vgic_is_valid_irq checks to allow operating with eSPIs
>> - remove redundant newline in vgic_allocate_virq
>> ---
>> xen/arch/arm/include/asm/vgic.h | 18 ++++
>> xen/arch/arm/vgic.c | 145 ++++++++++++++++++++++++++++++++
>> 2 files changed, 163 insertions(+)
>>
>> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
>> index 45201f4ca5..9fa4523018 100644
>> --- a/xen/arch/arm/include/asm/vgic.h
>> +++ b/xen/arch/arm/include/asm/vgic.h
>> @@ -146,6 +146,10 @@ struct vgic_dist {
>> int nr_spis; /* Number of SPIs */
>> unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
>> struct vgic_irq_rank *shared_irqs;
>> +#ifdef CONFIG_GICV3_ESPI
>> + struct vgic_irq_rank *ext_shared_irqs;
>> + int nr_espis; /* Number of extended SPIs */
>> +#endif
>> /*
>> * SPIs are domain global, SGIs and PPIs are per-VCPU and stored in
>> * struct arch_vcpu.
>> @@ -243,6 +247,14 @@ struct vgic_ops {
>> /* Number of ranks of interrupt registers for a domain */
>> #define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_spis+31)/32)
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> +#define DOMAIN_NR_EXT_RANKS(d) (((d)->arch.vgic.nr_espis+31)/32)
>> +#define EXT_RANK_MIN (ESPI_BASE_INTID/32)
>> +#define EXT_RANK_MAX ((ESPI_MAX_INTID+31)/32)
>> +#define EXT_RANK_NUM2IDX(num) ((num)-EXT_RANK_MIN)
>> +#define EXT_RANK_IDX2NUM(idx) ((idx)+EXT_RANK_MIN)
>> +#endif
>> +
>> #define vgic_lock(v) spin_lock_irq(&(v)->domain->arch.vgic.lock)
>> #define vgic_unlock(v) spin_unlock_irq(&(v)->domain->arch.vgic.lock)
>>
>> @@ -302,6 +314,12 @@ extern struct vgic_irq_rank *vgic_rank_offset(struct vcpu *v,
>> unsigned int b,
>> unsigned int n,
>> unsigned int s);
>> +#ifdef CONFIG_GICV3_ESPI
>> +extern struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v,
>> + unsigned int b,
>> + unsigned int n,
>> + unsigned int s);
>> +#endif
>> extern struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq);
>> extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned int n);
>> extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, unsigned int n);
>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>> index 48fbaf56fb..1a6c765af9 100644
>> --- a/xen/arch/arm/vgic.c
>> +++ b/xen/arch/arm/vgic.c
>> @@ -27,9 +27,26 @@
>>
>> bool vgic_is_valid_irq(struct domain *d, unsigned int virq)
>> {
>> +#ifdef CONFIG_GICV3_ESPI
>> + if ( virq >= ESPI_BASE_INTID && virq < ESPI_IDX2INTID(d->arch.vgic.nr_espis) )
>> + return true;
>> +#endif
>> +
>> return virq < vgic_num_irqs(d);
>> }
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> +/*
>> + * Since eSPI indexes start from 4096 and numbers from 1024 to
>> + * 4095 are forbidden, we need to check both lower and upper
>> + * limits for ranks.
>> + */
>> +static inline bool is_valid_espi_rank(struct domain *d, unsigned int rank)
>> +{
>> + return ( rank >= EXT_RANK_MIN && EXT_RANK_NUM2IDX(rank) < DOMAIN_NR_EXT_RANKS(d) );
>
> I am pretty sure that max line length in 80 symbols, and here you have
> more. As well in couple of other places below.
>
Previously, I had seen code in the mainline with lines longer than 80
characters, so I thought the maximum length was 100.. However, I
rechecked the CODING_STYLE file in Xen and it indeed contains the following:
> Lines should be less than 80 characters in length. Long lines should
> be split at sensible places and the trailing portions indented.
So, yes, my mistake. I will revise the code in the patch series and fix
the places where the line length exceeds 80 characters.
>> +}
>> +#endif
>> +
>> static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
>> unsigned int rank)
>> {
>> @@ -37,6 +54,10 @@ static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
>> return v->arch.vgic.private_irqs;
>> else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
>> return &v->domain->arch.vgic.shared_irqs[rank - 1];
>> +#ifdef CONFIG_GICV3_ESPI
>> + else if ( is_valid_espi_rank(v->domain, rank) )
>> + return &v->domain->arch.vgic.ext_shared_irqs[EXT_RANK_NUM2IDX(rank)];
>> +#endif
>> else
>> return NULL;
>> }
>> @@ -53,6 +74,16 @@ struct vgic_irq_rank *vgic_rank_offset(struct vcpu *v, unsigned int b,
>> return vgic_get_rank(v, rank);
>> }
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> +struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v, unsigned int b,
>> + unsigned int n, unsigned int s)
>> +{
>> + unsigned int rank = REG_RANK_NR(b, (n >> s));
>> +
>> + return vgic_get_rank(v, rank + EXT_RANK_MIN);
>> +}
>> +#endif
>> +
>> struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq)
>> {
>> unsigned int rank = irq / 32;
>> @@ -117,6 +148,29 @@ int domain_vgic_register(struct domain *d, unsigned int *mmio_count)
>> return 0;
>> }
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> +static int init_vgic_espi(struct domain *d)
>> +{
>> + int i;
>> +
>> + if ( d->arch.vgic.nr_espis == 0 )
>> + return 0;
>> +
>> + d->arch.vgic.ext_shared_irqs =
>> + xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_EXT_RANKS(d));
>> + if ( d->arch.vgic.ext_shared_irqs == NULL )
>> + return -ENOMEM;
>> +
>> + for ( i = 0; i < d->arch.vgic.nr_espis; i++ )
>> + vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i + d->arch.vgic.nr_spis], ESPI_IDX2INTID(i));
>> +
>> + for ( i = 0; i < DOMAIN_NR_EXT_RANKS(d); i++ )
>> + vgic_rank_init(&d->arch.vgic.ext_shared_irqs[i], i, 0);
>> +
>> + return 0;
>> +}
>> +#endif
>> +
>> int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>> {
>> int i;
>> @@ -131,6 +185,30 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>> */
>> nr_spis = ROUNDUP(nr_spis, 32);
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> + if ( nr_spis > ESPI_MAX_INTID )
>> + return -EINVAL;
>> +
>> + if ( is_espi(nr_spis) )
>> + {
>> + /*
>> + * During domain creation, the toolstack specifies the maximum INTID,
>> + * which is defined in the domain config subtracted by 32. To compute the
>> + * actual number of eSPI that will be usable for, add back 32.
>
> I think, according to this, your if ( is_espi() ) check is wrong. Should
> you add 32 here as well?
Oh, yes, the verifications above do not take into account this
subtraction of 32, so they will fail for the first 32 eSPI INTIDs and
mistakenly allow processing INTIDs greater than 5119. I will fix that in
V3 by adding 32 to nr_spis if checks and moving this comment above the
if checks.
>
>> + */
>> + d->arch.vgic.nr_espis = min(nr_spis - ESPI_BASE_INTID + 32, 1024U);
>> + /* Verify if GIC HW can handle provided INTID */
>> + if ( d->arch.vgic.nr_espis > gic_number_espis() )
>> + return -EINVAL;
>> + /* Set the maximum available number for defult SPI to pass the next check */
>> + nr_spis = VGIC_DEF_NR_SPIS;
>> + } else
>> + {
>> + /* Domain will use the regular SPI range */
>> + d->arch.vgic.nr_espis = 0;
>> + }
>> +#endif
>> +
>> /* Limit the number of virtual SPIs supported to (1020 - 32) = 988 */
>> if ( nr_spis > (1020 - NR_LOCAL_IRQS) )
>> return -EINVAL;
>> @@ -145,7 +223,12 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>> return -ENOMEM;
>>
>> d->arch.vgic.pending_irqs =
>> +#ifdef CONFIG_GICV3_ESPI
>> + xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis +
>> + d->arch.vgic.nr_espis);
>> +#else
>> xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis);
>> +#endif
>> if ( d->arch.vgic.pending_irqs == NULL )
>> return -ENOMEM;
>>
>> @@ -156,12 +239,23 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>> for ( i = 0; i < DOMAIN_NR_RANKS(d); i++ )
>> vgic_rank_init(&d->arch.vgic.shared_irqs[i], i + 1, 0);
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> + ret = init_vgic_espi(d);
>> + if ( ret )
>> + return ret;
>> +#endif
>> +
>> ret = d->arch.vgic.handler->domain_init(d);
>> if ( ret )
>> return ret;
>>
>> d->arch.vgic.allocated_irqs =
>> +#ifdef CONFIG_GICV3_ESPI
>> + xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d) +
>> + d->arch.vgic.nr_espis));
>> +#else
>> xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d)));
>> +#endif
>> if ( !d->arch.vgic.allocated_irqs )
>> return -ENOMEM;
>>
>> @@ -195,9 +289,27 @@ void domain_vgic_free(struct domain *d)
>> }
>> }
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> + for ( i = 0; i < (d->arch.vgic.nr_espis); i++ )
>> + {
>> + struct pending_irq *p = spi_to_pending(d, ESPI_IDX2INTID(i));
>> +
>> + if ( p->desc )
>> + {
>> + ret = release_guest_irq(d, p->irq);
>> + if ( ret )
>> + dprintk(XENLOG_G_WARNING, "d%u: Failed to release virq %u ret = %d\n",
>> + d->domain_id, p->irq, ret);
>> + }
>> + }
>> +#endif
>> +
>> if ( d->arch.vgic.handler )
>> d->arch.vgic.handler->domain_free(d);
>> xfree(d->arch.vgic.shared_irqs);
>> +#ifdef CONFIG_GICV3_ESPI
>> + xfree(d->arch.vgic.ext_shared_irqs);
>> +#endif
>> xfree(d->arch.vgic.pending_irqs);
>> xfree(d->arch.vgic.allocated_irqs);
>> }
>> @@ -331,6 +443,17 @@ void arch_move_irqs(struct vcpu *v)
>> if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) )
>> irq_set_affinity(p->desc, cpu_mask);
>> }
>> +
>> +#ifdef CONFIG_GICV3_ESPI
>> + for ( i = ESPI_BASE_INTID; i < (d)->arch.vgic.nr_espis; i++ )
>> + {
>> + v_target = vgic_get_target_vcpu(v, i);
>> + p = irq_to_pending(v_target, i);
>> +
>> + if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) )
>> + irq_set_affinity(p->desc, cpu_mask);
>> + }
>> +#endif
>> }
>>
>> void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned int n)
>> @@ -538,6 +661,10 @@ struct pending_irq *irq_to_pending(struct vcpu *v, unsigned int irq)
>> n = &v->arch.vgic.pending_irqs[irq];
>> else if ( is_lpi(irq) )
>> n = v->domain->arch.vgic.handler->lpi_to_pending(v->domain, irq);
>> +#ifdef CONFIG_GICV3_ESPI
>> + else if ( is_espi(irq) )
>> + n = &v->domain->arch.vgic.pending_irqs[ESPI_INTID2IDX(irq) + v->domain->arch.vgic.nr_spis];
>> +#endif
>> else
>> n = &v->domain->arch.vgic.pending_irqs[irq - 32];
>> return n;
>> @@ -547,6 +674,14 @@ struct pending_irq *spi_to_pending(struct domain *d, unsigned int irq)
>> {
>> ASSERT(irq >= NR_LOCAL_IRQS);
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> + if ( is_espi(irq) )
>> + {
>> + irq = ESPI_INTID2IDX(irq) + d->arch.vgic.nr_spis;
>> + return &d->arch.vgic.pending_irqs[irq];
>> + }
>> +#endif
>> +
>> return &d->arch.vgic.pending_irqs[irq - 32];
>> }
>>
>> @@ -668,6 +803,11 @@ bool vgic_reserve_virq(struct domain *d, unsigned int virq)
>> if ( !vgic_is_valid_irq(d, virq) )
>> return false;
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> + if ( is_espi(virq) )
>> + return !test_and_set_bit(ESPI_INTID2IDX(virq) + vgic_num_irqs(d), d->arch.vgic.allocated_irqs);
>> +#endif
>> +
>> return !test_and_set_bit(virq, d->arch.vgic.allocated_irqs);
>> }
>>
>> @@ -686,6 +826,11 @@ int vgic_allocate_virq(struct domain *d, bool spi)
>> {
>> first = 32;
>> end = vgic_num_irqs(d);
>> +#ifdef CONFIG_GICV3_ESPI
>> + /* Take into account extended SPI range */
>> + end += d->arch.vgic.nr_espis;
>> +#endif
>> +
>> }
>>
>> /*
>
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v2 08/10] xen/arm: vgic: add resource management for extended SPIs
2025-08-07 12:33 ` [PATCH v2 08/10] xen/arm: vgic: add resource management for extended SPIs Leonid Komarianskyi
2025-08-21 16:43 ` Volodymyr Babchuk
@ 2025-08-23 14:39 ` Oleksandr Tyshchenko
2025-08-24 18:28 ` Leonid Komarianskyi
1 sibling, 1 reply; 55+ messages in thread
From: Oleksandr Tyshchenko @ 2025-08-23 14:39 UTC (permalink / raw)
To: Leonid Komarianskyi, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini, Julien Grall, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk
On 07.08.25 15:33, Leonid Komarianskyi wrote:
Hello Leonid
> This change introduces resource management in the VGIC to handle
> extended SPIs introduced in GICv3.1. The pending_irqs and
> allocated_irqs arrays are resized to support the required
> number of eSPIs, based on what is supported by the hardware and
> requested by the guest. A new field, ext_shared_irqs, is added
> to the VGIC structure to store information about eSPIs, similar
> to how shared_irqs is used for regular SPIs.
>
> Since the eSPI range starts at INTID 4096 and INTIDs between 1025
> and 4095 are reserved, helper macros are introduced to simplify the
> transformation of indices and to enable easier access to eSPI-specific
> resources. These changes prepare the VGIC for processing eSPIs as
> required by future functionality.
>
> The initialization and deinitialization paths for vgic have been updated
> to allocate and free these resources appropriately. Additionally,
> updated handling of INTIDs greater than 1024, passed from the toolstack
> during domain creation, and verification logic ensures only valid SPI or
> eSPI INTIDs are used.
>
> The existing SPI behavior remains unaffected when guests do not request
> eSPIs, GIC hardware does not support them, or the CONFIG_GICV3_ESPI
> option is disabled.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>
> ---
> Changes in V2:
> - change is_espi_rank to is_valid_espi_rank to verify whether the array
> element ext_shared_irqs exists. The previous version, is_espi_rank,
> only checked if the rank index was less than the maximum possible eSPI
> rank index, but this could potentially result in accessing a
> non-existing array element. To address this, is_valid_espi_rank was
> introduced, which ensures that the required eSPI rank exists
> - move gic_number_espis to
> xen/arm: gicv3: implement handling of GICv3.1 eSPI
> - update vgic_is_valid_irq checks to allow operating with eSPIs
> - remove redundant newline in vgic_allocate_virq
> ---
> xen/arch/arm/include/asm/vgic.h | 18 ++++
> xen/arch/arm/vgic.c | 145 ++++++++++++++++++++++++++++++++
> 2 files changed, 163 insertions(+)
>
> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
> index 45201f4ca5..9fa4523018 100644
> --- a/xen/arch/arm/include/asm/vgic.h
> +++ b/xen/arch/arm/include/asm/vgic.h
> @@ -146,6 +146,10 @@ struct vgic_dist {
> int nr_spis; /* Number of SPIs */
> unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
> struct vgic_irq_rank *shared_irqs;
> +#ifdef CONFIG_GICV3_ESPI
> + struct vgic_irq_rank *ext_shared_irqs;
> + int nr_espis; /* Number of extended SPIs */
> +#endif
> /*
> * SPIs are domain global, SGIs and PPIs are per-VCPU and stored in
> * struct arch_vcpu.
> @@ -243,6 +247,14 @@ struct vgic_ops {
> /* Number of ranks of interrupt registers for a domain */
> #define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_spis+31)/32)
>
> +#ifdef CONFIG_GICV3_ESPI
> +#define DOMAIN_NR_EXT_RANKS(d) (((d)->arch.vgic.nr_espis+31)/32)
> +#define EXT_RANK_MIN (ESPI_BASE_INTID/32)
> +#define EXT_RANK_MAX ((ESPI_MAX_INTID+31)/32)
> +#define EXT_RANK_NUM2IDX(num) ((num)-EXT_RANK_MIN)
> +#define EXT_RANK_IDX2NUM(idx) ((idx)+EXT_RANK_MIN)
> +#endif
> +
> #define vgic_lock(v) spin_lock_irq(&(v)->domain->arch.vgic.lock)
> #define vgic_unlock(v) spin_unlock_irq(&(v)->domain->arch.vgic.lock)
>
> @@ -302,6 +314,12 @@ extern struct vgic_irq_rank *vgic_rank_offset(struct vcpu *v,
> unsigned int b,
> unsigned int n,
> unsigned int s);
> +#ifdef CONFIG_GICV3_ESPI
> +extern struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v,
> + unsigned int b,
> + unsigned int n,
> + unsigned int s);
> +#endif
> extern struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq);
> extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned int n);
> extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, unsigned int n);
> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
> index 48fbaf56fb..1a6c765af9 100644
> --- a/xen/arch/arm/vgic.c
> +++ b/xen/arch/arm/vgic.c
> @@ -27,9 +27,26 @@
>
> bool vgic_is_valid_irq(struct domain *d, unsigned int virq)
> {
> +#ifdef CONFIG_GICV3_ESPI
> + if ( virq >= ESPI_BASE_INTID && virq < ESPI_IDX2INTID(d->arch.vgic.nr_espis) )
> + return true;
> +#endif
> +
> return virq < vgic_num_irqs(d);
> }
>
> +#ifdef CONFIG_GICV3_ESPI
> +/*
> + * Since eSPI indexes start from 4096 and numbers from 1024 to
> + * 4095 are forbidden, we need to check both lower and upper
> + * limits for ranks.
> + */
> +static inline bool is_valid_espi_rank(struct domain *d, unsigned int rank)
> +{
> + return ( rank >= EXT_RANK_MIN && EXT_RANK_NUM2IDX(rank) < DOMAIN_NR_EXT_RANKS(d) );
> +}
> +#endif
> +
> static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
> unsigned int rank)
> {
> @@ -37,6 +54,10 @@ static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
> return v->arch.vgic.private_irqs;
> else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
> return &v->domain->arch.vgic.shared_irqs[rank - 1];
> +#ifdef CONFIG_GICV3_ESPI
> + else if ( is_valid_espi_rank(v->domain, rank) )
> + return &v->domain->arch.vgic.ext_shared_irqs[EXT_RANK_NUM2IDX(rank)];
> +#endif
> else
> return NULL;
> }
> @@ -53,6 +74,16 @@ struct vgic_irq_rank *vgic_rank_offset(struct vcpu *v, unsigned int b,
> return vgic_get_rank(v, rank);
> }
>
> +#ifdef CONFIG_GICV3_ESPI
> +struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v, unsigned int b,
> + unsigned int n, unsigned int s)
> +{
> + unsigned int rank = REG_RANK_NR(b, (n >> s));
> +
> + return vgic_get_rank(v, rank + EXT_RANK_MIN);
> +}
> +#endif
> +
> struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq)
> {
> unsigned int rank = irq / 32;
> @@ -117,6 +148,29 @@ int domain_vgic_register(struct domain *d, unsigned int *mmio_count)
> return 0;
> }
>
> +#ifdef CONFIG_GICV3_ESPI
> +static int init_vgic_espi(struct domain *d)
> +{
> + int i;
please use unsigned int if "i" cannot be negative
> +
> + if ( d->arch.vgic.nr_espis == 0 )
> + return 0;
> +
> + d->arch.vgic.ext_shared_irqs =
> + xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_EXT_RANKS(d));
> + if ( d->arch.vgic.ext_shared_irqs == NULL )
> + return -ENOMEM;
> +
> + for ( i = 0; i < d->arch.vgic.nr_espis; i++ )
> + vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i + d->arch.vgic.nr_spis], ESPI_IDX2INTID(i));
> +
> + for ( i = 0; i < DOMAIN_NR_EXT_RANKS(d); i++ )
> + vgic_rank_init(&d->arch.vgic.ext_shared_irqs[i], i, 0);
> +
> + return 0;
> +}
> +#endif
> +
> int domain_vgic_init(struct domain *d, unsigned int nr_spis)
> {
> int i;
> @@ -131,6 +185,30 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
> */
> nr_spis = ROUNDUP(nr_spis, 32);
>
> +#ifdef CONFIG_GICV3_ESPI
> + if ( nr_spis > ESPI_MAX_INTID )
> + return -EINVAL;
> +
> + if ( is_espi(nr_spis) )
> + {
> + /*
> + * During domain creation, the toolstack specifies the maximum INTID,
domain_vgic_init() is also called for dom0less DomUs (if present), which
are created at Xen boot. So the toolstack might even be absent on the
target system.
> + * which is defined in the domain config subtracted by 32. To compute the
> + * actual number of eSPI that will be usable for, add back 32.
> + */
> + d->arch.vgic.nr_espis = min(nr_spis - ESPI_BASE_INTID + 32, 1024U);
> + /* Verify if GIC HW can handle provided INTID */
> + if ( d->arch.vgic.nr_espis > gic_number_espis() )
> + return -EINVAL;
> + /* Set the maximum available number for defult SPI to pass the next check */
> + nr_spis = VGIC_DEF_NR_SPIS;
> + } else
> + {
> + /* Domain will use the regular SPI range */
> + d->arch.vgic.nr_espis = 0;
> + }
> +#endif
> +
> /* Limit the number of virtual SPIs supported to (1020 - 32) = 988 */
> if ( nr_spis > (1020 - NR_LOCAL_IRQS) )
> return -EINVAL;
> @@ -145,7 +223,12 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
> return -ENOMEM;
>
> d->arch.vgic.pending_irqs =
> +#ifdef CONFIG_GICV3_ESPI
> + xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis +
> + d->arch.vgic.nr_espis);
> +#else
> xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis);
> +#endif
> if ( d->arch.vgic.pending_irqs == NULL )
> return -ENOMEM;
>
> @@ -156,12 +239,23 @@ int domain_vgic_init(struct domain *d, unsigned int nr_spis)
> for ( i = 0; i < DOMAIN_NR_RANKS(d); i++ )
> vgic_rank_init(&d->arch.vgic.shared_irqs[i], i + 1, 0);
>
> +#ifdef CONFIG_GICV3_ESPI
> + ret = init_vgic_espi(d);
> + if ( ret )
> + return ret;
> +#endif
> +
> ret = d->arch.vgic.handler->domain_init(d);
> if ( ret )
> return ret;
>
> d->arch.vgic.allocated_irqs =
> +#ifdef CONFIG_GICV3_ESPI
> + xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d) +
> + d->arch.vgic.nr_espis));
> +#else
> xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d)));
> +#endif
> if ( !d->arch.vgic.allocated_irqs )
> return -ENOMEM;
>
> @@ -195,9 +289,27 @@ void domain_vgic_free(struct domain *d)
> }
> }
>
> +#ifdef CONFIG_GICV3_ESPI
> + for ( i = 0; i < (d->arch.vgic.nr_espis); i++ )
NIT: no need for () around d->arch.vgic.nr_espis
> + {
> + struct pending_irq *p = spi_to_pending(d, ESPI_IDX2INTID(i));
> +
> + if ( p->desc )
> + {
> + ret = release_guest_irq(d, p->irq);
> + if ( ret )
> + dprintk(XENLOG_G_WARNING, "d%u: Failed to release virq %u ret = %d\n",
> + d->domain_id, p->irq, ret);
> + }
> + }
> +#endif
> +
> if ( d->arch.vgic.handler )
> d->arch.vgic.handler->domain_free(d);
> xfree(d->arch.vgic.shared_irqs);
> +#ifdef CONFIG_GICV3_ESPI
> + xfree(d->arch.vgic.ext_shared_irqs);
> +#endif
> xfree(d->arch.vgic.pending_irqs);
> xfree(d->arch.vgic.allocated_irqs);
> }
> @@ -331,6 +443,17 @@ void arch_move_irqs(struct vcpu *v)
> if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) )
> irq_set_affinity(p->desc, cpu_mask);
> }
> +
> +#ifdef CONFIG_GICV3_ESPI
> + for ( i = ESPI_BASE_INTID; i < (d)->arch.vgic.nr_espis; i++ )
NIT: no need for () around d
[snip]
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 08/10] xen/arm: vgic: add resource management for extended SPIs
2025-08-23 14:39 ` Oleksandr Tyshchenko
@ 2025-08-24 18:28 ` Leonid Komarianskyi
0 siblings, 0 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-24 18:28 UTC (permalink / raw)
To: Oleksandr Tyshchenko, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini, Julien Grall, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk
Hello Oleksandr,
Thank you for your close review.
On 23.08.25 17:39, Oleksandr Tyshchenko wrote:
>
>
> On 07.08.25 15:33, Leonid Komarianskyi wrote:
>
>
> Hello Leonid
>
>> This change introduces resource management in the VGIC to handle
>> extended SPIs introduced in GICv3.1. The pending_irqs and
>> allocated_irqs arrays are resized to support the required
>> number of eSPIs, based on what is supported by the hardware and
>> requested by the guest. A new field, ext_shared_irqs, is added
>> to the VGIC structure to store information about eSPIs, similar
>> to how shared_irqs is used for regular SPIs.
>>
>> Since the eSPI range starts at INTID 4096 and INTIDs between 1025
>> and 4095 are reserved, helper macros are introduced to simplify the
>> transformation of indices and to enable easier access to eSPI-specific
>> resources. These changes prepare the VGIC for processing eSPIs as
>> required by future functionality.
>>
>> The initialization and deinitialization paths for vgic have been updated
>> to allocate and free these resources appropriately. Additionally,
>> updated handling of INTIDs greater than 1024, passed from the toolstack
>> during domain creation, and verification logic ensures only valid SPI or
>> eSPI INTIDs are used.
>>
>> The existing SPI behavior remains unaffected when guests do not request
>> eSPIs, GIC hardware does not support them, or the CONFIG_GICV3_ESPI
>> option is disabled.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V2:
>> - change is_espi_rank to is_valid_espi_rank to verify whether the array
>> element ext_shared_irqs exists. The previous version, is_espi_rank,
>> only checked if the rank index was less than the maximum possible eSPI
>> rank index, but this could potentially result in accessing a
>> non-existing array element. To address this, is_valid_espi_rank was
>> introduced, which ensures that the required eSPI rank exists
>> - move gic_number_espis to
>> xen/arm: gicv3: implement handling of GICv3.1 eSPI
>> - update vgic_is_valid_irq checks to allow operating with eSPIs
>> - remove redundant newline in vgic_allocate_virq
>> ---
>> xen/arch/arm/include/asm/vgic.h | 18 ++++
>> xen/arch/arm/vgic.c | 145 ++++++++++++++++++++++++++++++++
>> 2 files changed, 163 insertions(+)
>>
>> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/
>> asm/vgic.h
>> index 45201f4ca5..9fa4523018 100644
>> --- a/xen/arch/arm/include/asm/vgic.h
>> +++ b/xen/arch/arm/include/asm/vgic.h
>> @@ -146,6 +146,10 @@ struct vgic_dist {
>> int nr_spis; /* Number of SPIs */
>> unsigned long *allocated_irqs; /* bitmap of IRQs allocated */
>> struct vgic_irq_rank *shared_irqs;
>> +#ifdef CONFIG_GICV3_ESPI
>> + struct vgic_irq_rank *ext_shared_irqs;
>> + int nr_espis; /* Number of extended SPIs */
>> +#endif
>> /*
>> * SPIs are domain global, SGIs and PPIs are per-VCPU and stored in
>> * struct arch_vcpu.
>> @@ -243,6 +247,14 @@ struct vgic_ops {
>> /* Number of ranks of interrupt registers for a domain */
>> #define DOMAIN_NR_RANKS(d) (((d)->arch.vgic.nr_spis+31)/32)
>> +#ifdef CONFIG_GICV3_ESPI
>> +#define DOMAIN_NR_EXT_RANKS(d) (((d)->arch.vgic.nr_espis+31)/32)
>> +#define EXT_RANK_MIN (ESPI_BASE_INTID/32)
>> +#define EXT_RANK_MAX ((ESPI_MAX_INTID+31)/32)
>> +#define EXT_RANK_NUM2IDX(num) ((num)-EXT_RANK_MIN)
>> +#define EXT_RANK_IDX2NUM(idx) ((idx)+EXT_RANK_MIN)
>> +#endif
>> +
>> #define vgic_lock(v) spin_lock_irq(&(v)->domain->arch.vgic.lock)
>> #define vgic_unlock(v) spin_unlock_irq(&(v)->domain->arch.vgic.lock)
>> @@ -302,6 +314,12 @@ extern struct vgic_irq_rank
>> *vgic_rank_offset(struct vcpu *v,
>> unsigned int b,
>> unsigned int n,
>> unsigned int s);
>> +#ifdef CONFIG_GICV3_ESPI
>> +extern struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v,
>> + unsigned int b,
>> + unsigned int n,
>> + unsigned int s);
>> +#endif
>> extern struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned
>> int irq);
>> extern void vgic_disable_irqs(struct vcpu *v, uint32_t r, unsigned
>> int n);
>> extern void vgic_enable_irqs(struct vcpu *v, uint32_t r, unsigned
>> int n);
>> diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c
>> index 48fbaf56fb..1a6c765af9 100644
>> --- a/xen/arch/arm/vgic.c
>> +++ b/xen/arch/arm/vgic.c
>> @@ -27,9 +27,26 @@
>> bool vgic_is_valid_irq(struct domain *d, unsigned int virq)
>> {
>> +#ifdef CONFIG_GICV3_ESPI
>> + if ( virq >= ESPI_BASE_INTID && virq < ESPI_IDX2INTID(d-
>> >arch.vgic.nr_espis) )
>> + return true;
>> +#endif
>> +
>> return virq < vgic_num_irqs(d);
>> }
>> +#ifdef CONFIG_GICV3_ESPI
>> +/*
>> + * Since eSPI indexes start from 4096 and numbers from 1024 to
>> + * 4095 are forbidden, we need to check both lower and upper
>> + * limits for ranks.
>> + */
>> +static inline bool is_valid_espi_rank(struct domain *d, unsigned int
>> rank)
>> +{
>> + return ( rank >= EXT_RANK_MIN && EXT_RANK_NUM2IDX(rank) <
>> DOMAIN_NR_EXT_RANKS(d) );
>> +}
>> +#endif
>> +
>> static inline struct vgic_irq_rank *vgic_get_rank(struct vcpu *v,
>> unsigned int rank)
>> {
>> @@ -37,6 +54,10 @@ static inline struct vgic_irq_rank
>> *vgic_get_rank(struct vcpu *v,
>> return v->arch.vgic.private_irqs;
>> else if ( rank <= DOMAIN_NR_RANKS(v->domain) )
>> return &v->domain->arch.vgic.shared_irqs[rank - 1];
>> +#ifdef CONFIG_GICV3_ESPI
>> + else if ( is_valid_espi_rank(v->domain, rank) )
>> + return &v->domain-
>> >arch.vgic.ext_shared_irqs[EXT_RANK_NUM2IDX(rank)];
>> +#endif
>> else
>> return NULL;
>> }
>> @@ -53,6 +74,16 @@ struct vgic_irq_rank *vgic_rank_offset(struct vcpu
>> *v, unsigned int b,
>> return vgic_get_rank(v, rank);
>> }
>> +#ifdef CONFIG_GICV3_ESPI
>> +struct vgic_irq_rank *vgic_ext_rank_offset(struct vcpu *v, unsigned
>> int b,
>> + unsigned int n, unsigned
>> int s)
>> +{
>> + unsigned int rank = REG_RANK_NR(b, (n >> s));
>> +
>> + return vgic_get_rank(v, rank + EXT_RANK_MIN);
>> +}
>> +#endif
>> +
>> struct vgic_irq_rank *vgic_rank_irq(struct vcpu *v, unsigned int irq)
>> {
>> unsigned int rank = irq / 32;
>> @@ -117,6 +148,29 @@ int domain_vgic_register(struct domain *d,
>> unsigned int *mmio_count)
>> return 0;
>> }
>> +#ifdef CONFIG_GICV3_ESPI
>> +static int init_vgic_espi(struct domain *d)
>> +{
>> + int i;
>
> please use unsigned int if "i" cannot be negative
>
Sure, I will change the type to unsigned int in V3.
>> +
>> + if ( d->arch.vgic.nr_espis == 0 )
>> + return 0;
>> +
>> + d->arch.vgic.ext_shared_irqs =
>> + xzalloc_array(struct vgic_irq_rank, DOMAIN_NR_EXT_RANKS(d));
>> + if ( d->arch.vgic.ext_shared_irqs == NULL )
>> + return -ENOMEM;
>> +
>> + for ( i = 0; i < d->arch.vgic.nr_espis; i++ )
>> + vgic_init_pending_irq(&d->arch.vgic.pending_irqs[i + d-
>> >arch.vgic.nr_spis], ESPI_IDX2INTID(i));
>> +
>> + for ( i = 0; i < DOMAIN_NR_EXT_RANKS(d); i++ )
>> + vgic_rank_init(&d->arch.vgic.ext_shared_irqs[i], i, 0);
>> +
>> + return 0;
>> +}
>> +#endif
>> +
>> int domain_vgic_init(struct domain *d, unsigned int nr_spis)
>> {
>> int i;
>> @@ -131,6 +185,30 @@ int domain_vgic_init(struct domain *d, unsigned
>> int nr_spis)
>> */
>> nr_spis = ROUNDUP(nr_spis, 32);
>> +#ifdef CONFIG_GICV3_ESPI
>> + if ( nr_spis > ESPI_MAX_INTID )
>> + return -EINVAL;
>> +
>> + if ( is_espi(nr_spis) )
>> + {
>> + /*
>> + * During domain creation, the toolstack specifies the
>> maximum INTID,
>
> domain_vgic_init() is also called for dom0less DomUs (if present), which
> are created at Xen boot. So the toolstack might even be absent on the
> target system.
>
Thank you for pointing this out. I revised the patch series and found
that I did not change the behavior for dom0less builds. I will fix it in
V3 and update the comment.
>> + * which is defined in the domain config subtracted by 32. To
>> compute the
>> + * actual number of eSPI that will be usable for, add back 32.
>> + */
>> + d->arch.vgic.nr_espis = min(nr_spis - ESPI_BASE_INTID + 32,
>> 1024U);
>> + /* Verify if GIC HW can handle provided INTID */
>> + if ( d->arch.vgic.nr_espis > gic_number_espis() )
>> + return -EINVAL;
>> + /* Set the maximum available number for defult SPI to pass
>> the next check */
>> + nr_spis = VGIC_DEF_NR_SPIS;
>> + } else
>> + {
>> + /* Domain will use the regular SPI range */
>> + d->arch.vgic.nr_espis = 0;
>> + }
>> +#endif
>> +
>> /* Limit the number of virtual SPIs supported to (1020 - 32) =
>> 988 */
>> if ( nr_spis > (1020 - NR_LOCAL_IRQS) )
>> return -EINVAL;
>> @@ -145,7 +223,12 @@ int domain_vgic_init(struct domain *d, unsigned
>> int nr_spis)
>> return -ENOMEM;
>> d->arch.vgic.pending_irqs =
>> +#ifdef CONFIG_GICV3_ESPI
>> + xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis +
>> + d->arch.vgic.nr_espis);
>> +#else
>> xzalloc_array(struct pending_irq, d->arch.vgic.nr_spis);
>> +#endif
>> if ( d->arch.vgic.pending_irqs == NULL )
>> return -ENOMEM;
>> @@ -156,12 +239,23 @@ int domain_vgic_init(struct domain *d, unsigned
>> int nr_spis)
>> for ( i = 0; i < DOMAIN_NR_RANKS(d); i++ )
>> vgic_rank_init(&d->arch.vgic.shared_irqs[i], i + 1, 0);
>> +#ifdef CONFIG_GICV3_ESPI
>> + ret = init_vgic_espi(d);
>> + if ( ret )
>> + return ret;
>> +#endif
>> +
>> ret = d->arch.vgic.handler->domain_init(d);
>> if ( ret )
>> return ret;
>> d->arch.vgic.allocated_irqs =
>> +#ifdef CONFIG_GICV3_ESPI
>> + xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d) +
>> + d->arch.vgic.nr_espis));
>> +#else
>> xzalloc_array(unsigned long, BITS_TO_LONGS(vgic_num_irqs(d)));
>> +#endif
>> if ( !d->arch.vgic.allocated_irqs )
>> return -ENOMEM;
>> @@ -195,9 +289,27 @@ void domain_vgic_free(struct domain *d)
>> }
>> }
>> +#ifdef CONFIG_GICV3_ESPI
>> + for ( i = 0; i < (d->arch.vgic.nr_espis); i++ )
>
> NIT: no need for () around d->arch.vgic.nr_espis
>
I will remove them in V3.
>> + {
>> + struct pending_irq *p = spi_to_pending(d, ESPI_IDX2INTID(i));
>> +
>> + if ( p->desc )
>> + {
>> + ret = release_guest_irq(d, p->irq);
>> + if ( ret )
>> + dprintk(XENLOG_G_WARNING, "d%u: Failed to release
>> virq %u ret = %d\n",
>> + d->domain_id, p->irq, ret);
>> + }
>> + }
>> +#endif
>> +
>> if ( d->arch.vgic.handler )
>> d->arch.vgic.handler->domain_free(d);
>> xfree(d->arch.vgic.shared_irqs);
>> +#ifdef CONFIG_GICV3_ESPI
>> + xfree(d->arch.vgic.ext_shared_irqs);
>> +#endif
>> xfree(d->arch.vgic.pending_irqs);
>> xfree(d->arch.vgic.allocated_irqs);
>> }
>> @@ -331,6 +443,17 @@ void arch_move_irqs(struct vcpu *v)
>> if ( v_target == v && !test_bit(GIC_IRQ_GUEST_MIGRATING, &p-
>> >status) )
>> irq_set_affinity(p->desc, cpu_mask);
>> }
>> +
>> +#ifdef CONFIG_GICV3_ESPI
>> + for ( i = ESPI_BASE_INTID; i < (d)->arch.vgic.nr_espis; i++ )
>
> NIT: no need for () around d
I will remove them in V3.
>
> [snip]
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread
* [PATCH v2 10/10] xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers
2025-08-07 12:33 ` [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations Leonid Komarianskyi
` (6 preceding siblings ...)
2025-08-07 12:33 ` [PATCH v2 08/10] xen/arm: vgic: add resource management for extended SPIs Leonid Komarianskyi
@ 2025-08-07 12:33 ` Leonid Komarianskyi
2025-08-21 16:17 ` Oleksandr Tyshchenko
2025-08-21 17:26 ` Volodymyr Babchuk
2025-08-07 12:33 ` [PATCH v2 09/10] xen/arm: domain_build: adjust Dom0 IRQ handling to support eSPIs Leonid Komarianskyi
` (2 subsequent siblings)
10 siblings, 2 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-07 12:33 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Leonid Komarianskyi, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk
Implemented support for GICv3.1 extended SPI registers for vGICv3,
allowing the emulation of eSPI-specific behavior for guest domains.
The implementation includes read and write emulation for eSPI-related
registers (e.g., GICD_ISENABLERnE, GICD_IROUTERnE, and others),
following a similar approach to the handling of regular SPIs.
The eSPI registers, previously located in reserved address ranges,
are now adjusted to support MMIO read and write operations correctly
when CONFIG_GICV3_ESPI is enabled.
The availability of eSPIs and the number of emulated extended SPIs
for guest domains is reported by setting the appropriate bits in the
GICD_TYPER register, based on the number of eSPIs requested by the
domain and supported by the hardware. In cases where the configuration
option is disabled, the hardware does not support eSPIs, or the domain
does not request such interrupts, the functionality remains unchanged.
Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
---
Changes in V2:
- add missing rank index conversion for pending and inflight irqs
---
xen/arch/arm/vgic-v3.c | 248 ++++++++++++++++++++++++++++++++++++++++-
1 file changed, 245 insertions(+), 3 deletions(-)
diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 4369c55177..1cacbb6e43 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -111,7 +111,7 @@ static uint64_t vgic_fetch_irouter(struct vgic_irq_rank *rank,
* Note the offset will be aligned to the appropriate boundary.
*/
static void vgic_store_irouter(struct domain *d, struct vgic_irq_rank *rank,
- unsigned int offset, uint64_t irouter)
+ unsigned int offset, uint64_t irouter, bool espi)
{
struct vcpu *new_vcpu, *old_vcpu;
unsigned int virq;
@@ -123,7 +123,8 @@ static void vgic_store_irouter(struct domain *d, struct vgic_irq_rank *rank,
* The IROUTER0-31, used for SGIs/PPIs, are reserved and should
* never call this function.
*/
- ASSERT(virq >= 32);
+ if ( !espi )
+ ASSERT(virq >= 32);
/* Get the index in the rank */
offset = virq & INTERRUPT_RANK_MASK;
@@ -146,6 +147,11 @@ static void vgic_store_irouter(struct domain *d, struct vgic_irq_rank *rank,
/* Only migrate the IRQ if the target vCPU has changed */
if ( new_vcpu != old_vcpu )
{
+#ifdef CONFIG_GICV3_ESPI
+ /* Convert virq index to eSPI range */
+ if ( espi )
+ virq = ESPI_IDX2INTID(virq);
+#endif
if ( vgic_migrate_irq(old_vcpu, new_vcpu, virq) )
write_atomic(&rank->vcpu[offset], new_vcpu->vcpu_id);
}
@@ -685,6 +691,9 @@ static int __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
{
case VRANGE32(GICD_IGROUPR, GICD_IGROUPRN):
case VRANGE32(GICD_IGRPMODR, GICD_IGRPMODRN):
+#ifdef CONFIG_GICV3_ESPI
+ case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
+#endif
/* We do not implement security extensions for guests, read zero */
if ( dabt.size != DABT_WORD ) goto bad_width;
goto read_as_zero;
@@ -710,11 +719,19 @@ static int __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
/* Read the pending status of an IRQ via GICD/GICR is not supported */
case VRANGE32(GICD_ISPENDR, GICD_ISPENDRN):
case VRANGE32(GICD_ICPENDR, GICD_ICPENDRN):
+#ifdef CONFIG_GICV3_ESPI
+ case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
+ case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
+#endif
goto read_as_zero;
/* Read the active status of an IRQ via GICD/GICR is not supported */
case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
case VRANGE32(GICD_ICACTIVER, GICD_ICACTIVERN):
+#ifdef CONFIG_GICV3_ESPI
+ case VRANGE32(GICD_ISACTIVERnE, GICD_ISACTIVERnEN):
+ case VRANGE32(GICD_ICACTIVERnE, GICD_ICACTIVERnEN):
+#endif
goto read_as_zero;
case VRANGE32(GICD_IPRIORITYR, GICD_IPRIORITYRN):
@@ -752,6 +769,61 @@ static int __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
return 1;
}
+#ifdef CONFIG_GICV3_ESPI
+ case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ rank = vgic_ext_rank_offset(v, 1, reg - GICD_ISENABLERnE, DABT_WORD);
+ if ( rank == NULL ) goto read_as_zero;
+ vgic_lock_rank(v, rank, flags);
+ *r = vreg_reg32_extract(rank->ienable, info);
+ vgic_unlock_rank(v, rank, flags);
+ return 1;
+
+ case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ rank = vgic_ext_rank_offset(v, 1, reg - GICD_ICENABLERnE, DABT_WORD);
+ if ( rank == NULL ) goto read_as_zero;
+ vgic_lock_rank(v, rank, flags);
+ *r = vreg_reg32_extract(rank->ienable, info);
+ vgic_unlock_rank(v, rank, flags);
+ return 1;
+
+ case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
+ {
+ uint32_t ipriorityr;
+ uint8_t rank_index;
+
+ if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+ rank = vgic_ext_rank_offset(v, 8, reg - GICD_IPRIORITYRnE, DABT_WORD);
+ if ( rank == NULL ) goto read_as_zero;
+ rank_index = REG_RANK_INDEX(8, reg - GICD_IPRIORITYRnE, DABT_WORD);
+
+ vgic_lock_rank(v, rank, flags);
+ ipriorityr = ACCESS_ONCE(rank->ipriorityr[rank_index]);
+ vgic_unlock_rank(v, rank, flags);
+
+ *r = vreg_reg32_extract(ipriorityr, info);
+
+ return 1;
+ }
+
+ case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
+ {
+ uint32_t icfgr;
+
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ rank = vgic_ext_rank_offset(v, 2, reg - GICD_ICFGRnE, DABT_WORD);
+ if ( rank == NULL ) goto read_as_zero;
+ vgic_lock_rank(v, rank, flags);
+ icfgr = rank->icfg[REG_RANK_INDEX(2, reg - GICD_ICFGRnE, DABT_WORD)];
+ vgic_unlock_rank(v, rank, flags);
+
+ *r = vreg_reg32_extract(icfgr, info);
+
+ return 1;
+ }
+#endif
+
default:
printk(XENLOG_G_ERR
"%pv: %s: unhandled read r%d offset %#08x\n",
@@ -782,6 +854,9 @@ static int __vgic_v3_distr_common_mmio_write(const char *name, struct vcpu *v,
{
case VRANGE32(GICD_IGROUPR, GICD_IGROUPRN):
case VRANGE32(GICD_IGRPMODR, GICD_IGRPMODRN):
+#ifdef CONFIG_GICV3_ESPI
+ case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
+#endif
/* We do not implement security extensions for guests, write ignore */
goto write_ignore_32;
@@ -871,6 +946,87 @@ static int __vgic_v3_distr_common_mmio_write(const char *name, struct vcpu *v,
vgic_unlock_rank(v, rank, flags);
return 1;
+#ifdef CONFIG_GICV3_ESPI
+ case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ rank = vgic_ext_rank_offset(v, 1, reg - GICD_ISENABLERnE, DABT_WORD);
+ if ( rank == NULL ) goto write_ignore;
+ vgic_lock_rank(v, rank, flags);
+ tr = rank->ienable;
+ vreg_reg32_setbits(&rank->ienable, r, info);
+ vgic_enable_irqs(v, (rank->ienable) & (~tr), EXT_RANK_IDX2NUM(rank->index));
+ vgic_unlock_rank(v, rank, flags);
+ return 1;
+
+ case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ rank = vgic_ext_rank_offset(v, 1, reg - GICD_ICENABLERnE, DABT_WORD);
+ if ( rank == NULL ) goto write_ignore;
+ vgic_lock_rank(v, rank, flags);
+ tr = rank->ienable;
+ vreg_reg32_clearbits(&rank->ienable, r, info);
+ vgic_disable_irqs(v, (~rank->ienable) & tr, EXT_RANK_IDX2NUM(rank->index));
+ vgic_unlock_rank(v, rank, flags);
+ return 1;
+
+ case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ rank = vgic_ext_rank_offset(v, 1, reg - GICD_ISPENDRnE, DABT_WORD);
+ if ( rank == NULL ) goto write_ignore;
+
+ vgic_set_irqs_pending(v, r, EXT_RANK_IDX2NUM(rank->index));
+
+ return 1;
+
+ case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ rank = vgic_ext_rank_offset(v, 1, reg - GICD_ICPENDRnE, DABT_WORD);
+ if ( rank == NULL ) goto write_ignore;
+
+ vgic_check_inflight_irqs_pending(v, EXT_RANK_IDX2NUM(rank->index), r);
+
+ goto write_ignore;
+ case VRANGE32(GICD_ISACTIVERnE, GICD_ISACTIVERnEN):
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ printk(XENLOG_G_ERR
+ "%pv: %s: unhandled word write %#"PRIregister" to ISACTIVER%d\n",
+ v, name, r, reg - GICD_ISACTIVERnE);
+ return 0;
+
+ case VRANGE32(GICD_ICACTIVERnE, GICD_ICACTIVERnEN):
+ printk(XENLOG_G_ERR
+ "%pv: %s: unhandled word write %#"PRIregister" to ICACTIVER%d\n",
+ v, name, r, reg - GICD_ICACTIVER);
+ goto write_ignore_32;
+
+ case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
+ {
+ uint32_t *ipriorityr, priority;
+
+ if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
+ rank = vgic_ext_rank_offset(v, 8, reg - GICD_IPRIORITYRnE, DABT_WORD);
+ if ( rank == NULL ) goto write_ignore;
+ vgic_lock_rank(v, rank, flags);
+ ipriorityr = &rank->ipriorityr[REG_RANK_INDEX(8, reg - GICD_IPRIORITYRnE,
+ DABT_WORD)];
+ priority = ACCESS_ONCE(*ipriorityr);
+ vreg_reg32_update(&priority, r, info);
+ ACCESS_ONCE(*ipriorityr) = priority;
+ vgic_unlock_rank(v, rank, flags);
+ return 1;
+ }
+ case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
+ if ( dabt.size != DABT_WORD ) goto bad_width;
+ rank = vgic_ext_rank_offset(v, 2, reg - GICD_ICFGRnE, DABT_WORD);
+ if ( rank == NULL ) goto write_ignore;
+ vgic_lock_rank(v, rank, flags);
+ vreg_reg32_update(&rank->icfg[REG_RANK_INDEX(2, reg - GICD_ICFGRnE,
+ DABT_WORD)],
+ r, info);
+ vgic_unlock_rank(v, rank, flags);
+ return 1;
+#endif
+
default:
printk(XENLOG_G_ERR
"%pv: %s: unhandled write r%d=%"PRIregister" offset %#08x\n",
@@ -1129,6 +1285,16 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info,
typer |= GICD_TYPE_LPIS;
typer |= (v->domain->arch.vgic.intid_bits - 1) << GICD_TYPE_ID_BITS_SHIFT;
+#ifdef CONFIG_GICV3_ESPI
+ if ( v->domain->arch.vgic.nr_espis > 0 )
+ {
+ /* Set eSPI support bit for the domain */
+ typer |= GICD_TYPER_ESPI;
+ /* Set ESPI range bits */
+ typer |= (DIV_ROUND_UP(v->domain->arch.vgic.nr_espis, 32) - 1)
+ << GICD_TYPER_ESPI_RANGE_SHIFT;
+ }
+#endif
*r = vreg_reg32_extract(typer, info);
@@ -1194,6 +1360,18 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info,
case VRANGE32(GICD_IPRIORITYR, GICD_IPRIORITYRN):
case VRANGE32(GICD_ICFGR, GICD_ICFGRN):
case VRANGE32(GICD_IGRPMODR, GICD_IGRPMODRN):
+
+#ifdef CONFIG_GICV3_ESPI
+ case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
+ case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
+ case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
+ case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
+ case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
+ case VRANGE32(GICD_ISACTIVERnE, GICD_ISACTIVERnEN):
+ case VRANGE32(GICD_ICACTIVERnE, GICD_ICACTIVERnEN):
+ case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
+ case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
+#endif
/*
* Above all register are common with GICR and GICD
* Manage in common
@@ -1216,7 +1394,11 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info,
/* Replaced with GICR_ISPENDR0. So ignore write */
goto read_as_zero_32;
+#ifdef CONFIG_GICV3_ESPI
+ case VRANGE32(0x3100, 0x60FC):
+#else
case VRANGE32(0x0F30, 0x60FC):
+#endif
goto read_reserved;
case VRANGE64(GICD_IROUTER32, GICD_IROUTER1019):
@@ -1235,8 +1417,30 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info,
return 1;
}
+#ifdef CONFIG_GICV3_ESPI
+ case VRANGE64(GICD_IROUTERnE, GICD_IROUTERnEN):
+ {
+ uint64_t irouter;
+
+ if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
+ rank = vgic_ext_rank_offset(v, 64, gicd_reg - GICD_IROUTERnE,
+ DABT_DOUBLE_WORD);
+ if ( rank == NULL ) goto read_as_zero;
+ vgic_lock_rank(v, rank, flags);
+ irouter = vgic_fetch_irouter(rank, gicd_reg - GICD_IROUTERnE);
+ vgic_unlock_rank(v, rank, flags);
+
+ *r = vreg_reg64_extract(irouter, info);
+
+ return 1;
+ }
+#endif
+#ifdef CONFIG_GICV3_ESPI
+ case VRANGE32(0xA004, 0xBFFC):
+#else
case VRANGE32(0x7FE0, 0xBFFC):
+#endif
goto read_reserved;
case VRANGE32(0xC000, 0xFFCC):
@@ -1382,6 +1586,18 @@ static int vgic_v3_distr_mmio_write(struct vcpu *v, mmio_info_t *info,
case VRANGE32(GICD_IPRIORITYR, GICD_IPRIORITYRN):
case VRANGE32(GICD_ICFGR, GICD_ICFGRN):
case VRANGE32(GICD_IGRPMODR, GICD_IGRPMODRN):
+
+#ifdef CONFIG_GICV3_ESPI
+ case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
+ case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
+ case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
+ case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
+ case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
+ case VRANGE32(GICD_ISACTIVERnE, GICD_ISACTIVERnEN):
+ case VRANGE32(GICD_ICACTIVERnE, GICD_ICACTIVERnEN):
+ case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
+ case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
+#endif
/* Above registers are common with GICR and GICD
* Manage in common */
return __vgic_v3_distr_common_mmio_write("vGICD", v, info,
@@ -1405,7 +1621,11 @@ static int vgic_v3_distr_mmio_write(struct vcpu *v, mmio_info_t *info,
if ( dabt.size != DABT_WORD ) goto bad_width;
return 0;
+#ifdef CONFIG_GICV3_ESPI
+ case VRANGE32(0x3100, 0x60FC):
+#else
case VRANGE32(0x0F30, 0x60FC):
+#endif
goto write_reserved;
case VRANGE64(GICD_IROUTER32, GICD_IROUTER1019):
@@ -1419,12 +1639,34 @@ static int vgic_v3_distr_mmio_write(struct vcpu *v, mmio_info_t *info,
vgic_lock_rank(v, rank, flags);
irouter = vgic_fetch_irouter(rank, gicd_reg - GICD_IROUTER);
vreg_reg64_update(&irouter, r, info);
- vgic_store_irouter(v->domain, rank, gicd_reg - GICD_IROUTER, irouter);
+ vgic_store_irouter(v->domain, rank, gicd_reg - GICD_IROUTER, irouter, false);
+ vgic_unlock_rank(v, rank, flags);
+ return 1;
+ }
+
+#ifdef CONFIG_GICV3_ESPI
+ case VRANGE64(GICD_IROUTERnE, GICD_IROUTERnEN):
+ {
+ uint64_t irouter;
+
+ if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
+ rank = vgic_ext_rank_offset(v, 64, gicd_reg - GICD_IROUTERnE,
+ DABT_DOUBLE_WORD);
+ if ( rank == NULL ) goto write_ignore;
+ vgic_lock_rank(v, rank, flags);
+ irouter = vgic_fetch_irouter(rank, gicd_reg - GICD_IROUTERnE);
+ vreg_reg64_update(&irouter, r, info);
+ vgic_store_irouter(v->domain, rank, gicd_reg - GICD_IROUTERnE, irouter, true);
vgic_unlock_rank(v, rank, flags);
return 1;
}
+#endif
+#ifdef CONFIG_GICV3_ESPI
+ case VRANGE32(0xA004, 0xBFFC):
+#else
case VRANGE32(0x7FE0, 0xBFFC):
+#endif
goto write_reserved;
case VRANGE32(0xC000, 0xFFCC):
--
2.34.1
^ permalink raw reply related [flat|nested] 55+ messages in thread* Re: [PATCH v2 10/10] xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers
2025-08-07 12:33 ` [PATCH v2 10/10] xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers Leonid Komarianskyi
@ 2025-08-21 16:17 ` Oleksandr Tyshchenko
2025-08-22 10:47 ` Leonid Komarianskyi
2025-08-21 17:26 ` Volodymyr Babchuk
1 sibling, 1 reply; 55+ messages in thread
From: Oleksandr Tyshchenko @ 2025-08-21 16:17 UTC (permalink / raw)
To: Leonid Komarianskyi, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini, Julien Grall, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk
On 07.08.25 15:33, Leonid Komarianskyi wrote:
Hello Leonid
> Implemented support for GICv3.1 extended SPI registers for vGICv3,
> allowing the emulation of eSPI-specific behavior for guest domains.
> The implementation includes read and write emulation for eSPI-related
> registers (e.g., GICD_ISENABLERnE, GICD_IROUTERnE, and others),
> following a similar approach to the handling of regular SPIs.
>
> The eSPI registers, previously located in reserved address ranges,
> are now adjusted to support MMIO read and write operations correctly
> when CONFIG_GICV3_ESPI is enabled.
>
> The availability of eSPIs and the number of emulated extended SPIs
> for guest domains is reported by setting the appropriate bits in the
> GICD_TYPER register, based on the number of eSPIs requested by the
> domain and supported by the hardware. In cases where the configuration
> option is disabled, the hardware does not support eSPIs, or the domain
> does not request such interrupts, the functionality remains unchanged.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>
> ---
> Changes in V2:
> - add missing rank index conversion for pending and inflight irqs
> ---
> xen/arch/arm/vgic-v3.c | 248 ++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 245 insertions(+), 3 deletions(-)
>
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index 4369c55177..1cacbb6e43 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -111,7 +111,7 @@ static uint64_t vgic_fetch_irouter(struct vgic_irq_rank *rank,
> * Note the offset will be aligned to the appropriate boundary.
> */
> static void vgic_store_irouter(struct domain *d, struct vgic_irq_rank *rank,
> - unsigned int offset, uint64_t irouter)
> + unsigned int offset, uint64_t irouter, bool espi)
> {
> struct vcpu *new_vcpu, *old_vcpu;
> unsigned int virq;
> @@ -123,7 +123,8 @@ static void vgic_store_irouter(struct domain *d, struct vgic_irq_rank *rank,
> * The IROUTER0-31, used for SGIs/PPIs, are reserved and should
> * never call this function.
> */
> - ASSERT(virq >= 32);
> + if ( !espi )
> + ASSERT(virq >= 32);
>
> /* Get the index in the rank */
> offset = virq & INTERRUPT_RANK_MASK;
> @@ -146,6 +147,11 @@ static void vgic_store_irouter(struct domain *d, struct vgic_irq_rank *rank,
> /* Only migrate the IRQ if the target vCPU has changed */
> if ( new_vcpu != old_vcpu )
> {
> +#ifdef CONFIG_GICV3_ESPI
> + /* Convert virq index to eSPI range */
> + if ( espi )
> + virq = ESPI_IDX2INTID(virq);
> +#endif
> if ( vgic_migrate_irq(old_vcpu, new_vcpu, virq) )
> write_atomic(&rank->vcpu[offset], new_vcpu->vcpu_id);
> }
> @@ -685,6 +691,9 @@ static int __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
> {
> case VRANGE32(GICD_IGROUPR, GICD_IGROUPRN):
> case VRANGE32(GICD_IGRPMODR, GICD_IGRPMODRN):
> +#ifdef CONFIG_GICV3_ESPI
> + case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
> +#endif
> /* We do not implement security extensions for guests, read zero */
> if ( dabt.size != DABT_WORD ) goto bad_width;
> goto read_as_zero;
> @@ -710,11 +719,19 @@ static int __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
> /* Read the pending status of an IRQ via GICD/GICR is not supported */
> case VRANGE32(GICD_ISPENDR, GICD_ISPENDRN):
> case VRANGE32(GICD_ICPENDR, GICD_ICPENDRN):
> +#ifdef CONFIG_GICV3_ESPI
> + case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
> + case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
> +#endif
> goto read_as_zero;
>
> /* Read the active status of an IRQ via GICD/GICR is not supported */
> case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
> case VRANGE32(GICD_ICACTIVER, GICD_ICACTIVERN):
> +#ifdef CONFIG_GICV3_ESPI
> + case VRANGE32(GICD_ISACTIVERnE, GICD_ISACTIVERnEN):
> + case VRANGE32(GICD_ICACTIVERnE, GICD_ICACTIVERnEN):
> +#endif
> goto read_as_zero;
>
> case VRANGE32(GICD_IPRIORITYR, GICD_IPRIORITYRN):
> @@ -752,6 +769,61 @@ static int __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
> return 1;
> }
>
> +#ifdef CONFIG_GICV3_ESPI
> + case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
> + if ( dabt.size != DABT_WORD ) goto bad_width;
NIT: If I am not mistaken, the goto should be on the next line (here and
in similar places throughout the added code).
> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ISENABLERnE, DABT_WORD);
> + if ( rank == NULL ) goto read_as_zero;
> + vgic_lock_rank(v, rank, flags);
> + *r = vreg_reg32_extract(rank->ienable, info);
> + vgic_unlock_rank(v, rank, flags);
> + return 1;
> +
> + case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ICENABLERnE, DABT_WORD);
> + if ( rank == NULL ) goto read_as_zero;
> + vgic_lock_rank(v, rank, flags);
> + *r = vreg_reg32_extract(rank->ienable, info);
> + vgic_unlock_rank(v, rank, flags);
> + return 1;
> +
> + case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
> + {
> + uint32_t ipriorityr;
> + uint8_t rank_index;
> +
> + if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 8, reg - GICD_IPRIORITYRnE, DABT_WORD);
> + if ( rank == NULL ) goto read_as_zero;
> + rank_index = REG_RANK_INDEX(8, reg - GICD_IPRIORITYRnE, DABT_WORD);
> +
> + vgic_lock_rank(v, rank, flags);
> + ipriorityr = ACCESS_ONCE(rank->ipriorityr[rank_index]);
> + vgic_unlock_rank(v, rank, flags);
> +
> + *r = vreg_reg32_extract(ipriorityr, info);
> +
> + return 1;
> + }
> +
> + case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
> + {
> + uint32_t icfgr;
> +
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 2, reg - GICD_ICFGRnE, DABT_WORD);
> + if ( rank == NULL ) goto read_as_zero;
> + vgic_lock_rank(v, rank, flags);
> + icfgr = rank->icfg[REG_RANK_INDEX(2, reg - GICD_ICFGRnE, DABT_WORD)];
> + vgic_unlock_rank(v, rank, flags);
> +
> + *r = vreg_reg32_extract(icfgr, info);
> +
> + return 1;
> + }
> +#endif
> +
> default:
> printk(XENLOG_G_ERR
> "%pv: %s: unhandled read r%d offset %#08x\n",
> @@ -782,6 +854,9 @@ static int __vgic_v3_distr_common_mmio_write(const char *name, struct vcpu *v,
> {
> case VRANGE32(GICD_IGROUPR, GICD_IGROUPRN):
> case VRANGE32(GICD_IGRPMODR, GICD_IGRPMODRN):
> +#ifdef CONFIG_GICV3_ESPI
> + case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
> +#endif
> /* We do not implement security extensions for guests, write ignore */
> goto write_ignore_32;
>
> @@ -871,6 +946,87 @@ static int __vgic_v3_distr_common_mmio_write(const char *name, struct vcpu *v,
> vgic_unlock_rank(v, rank, flags);
> return 1;
>
> +#ifdef CONFIG_GICV3_ESPI
> + case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ISENABLERnE, DABT_WORD);
> + if ( rank == NULL ) goto write_ignore;
> + vgic_lock_rank(v, rank, flags);
> + tr = rank->ienable;
> + vreg_reg32_setbits(&rank->ienable, r, info);
> + vgic_enable_irqs(v, (rank->ienable) & (~tr), EXT_RANK_IDX2NUM(rank->index));
> + vgic_unlock_rank(v, rank, flags);
> + return 1;
> +
> + case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ICENABLERnE, DABT_WORD);
> + if ( rank == NULL ) goto write_ignore;
> + vgic_lock_rank(v, rank, flags);
> + tr = rank->ienable;
> + vreg_reg32_clearbits(&rank->ienable, r, info);
> + vgic_disable_irqs(v, (~rank->ienable) & tr, EXT_RANK_IDX2NUM(rank->index));
> + vgic_unlock_rank(v, rank, flags);
> + return 1;
> +
> + case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ISPENDRnE, DABT_WORD);
> + if ( rank == NULL ) goto write_ignore;
> +
> + vgic_set_irqs_pending(v, r, EXT_RANK_IDX2NUM(rank->index));
> +
> + return 1;
> +
> + case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ICPENDRnE, DABT_WORD);
> + if ( rank == NULL ) goto write_ignore;
> +
> + vgic_check_inflight_irqs_pending(v, EXT_RANK_IDX2NUM(rank->index), r);
> +
> + goto write_ignore;
> + case VRANGE32(GICD_ISACTIVERnE, GICD_ISACTIVERnEN):
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + printk(XENLOG_G_ERR
> + "%pv: %s: unhandled word write %#"PRIregister" to ISACTIVER%d\n",
I would use ISACTIVER%dE in the printed message to distinguish between
normal and "extended" registers (here and in similar places throughout
the added code).
> + v, name, r, reg - GICD_ISACTIVERnE);
> + return 0;
> +
> + case VRANGE32(GICD_ICACTIVERnE, GICD_ICACTIVERnEN):
> + printk(XENLOG_G_ERR
> + "%pv: %s: unhandled word write %#"PRIregister" to ICACTIVER%d\n",
> + v, name, r, reg - GICD_ICACTIVER);
s/GICD_ICACTIVER/GICD_ICACTIVERnE
[snip]
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 10/10] xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers
2025-08-21 16:17 ` Oleksandr Tyshchenko
@ 2025-08-22 10:47 ` Leonid Komarianskyi
0 siblings, 0 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-22 10:47 UTC (permalink / raw)
To: Oleksandr Tyshchenko, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini, Julien Grall, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk
Hello Oleksandr,
Thank you for your review comments.
On 21.08.25 19:17, Oleksandr Tyshchenko wrote:
>
>
> On 07.08.25 15:33, Leonid Komarianskyi wrote:
>
> Hello Leonid
>
>
>> Implemented support for GICv3.1 extended SPI registers for vGICv3,
>> allowing the emulation of eSPI-specific behavior for guest domains.
>> The implementation includes read and write emulation for eSPI-related
>> registers (e.g., GICD_ISENABLERnE, GICD_IROUTERnE, and others),
>> following a similar approach to the handling of regular SPIs.
>>
>> The eSPI registers, previously located in reserved address ranges,
>> are now adjusted to support MMIO read and write operations correctly
>> when CONFIG_GICV3_ESPI is enabled.
>>
>> The availability of eSPIs and the number of emulated extended SPIs
>> for guest domains is reported by setting the appropriate bits in the
>> GICD_TYPER register, based on the number of eSPIs requested by the
>> domain and supported by the hardware. In cases where the configuration
>> option is disabled, the hardware does not support eSPIs, or the domain
>> does not request such interrupts, the functionality remains unchanged.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V2:
>> - add missing rank index conversion for pending and inflight irqs
>> ---
>> xen/arch/arm/vgic-v3.c | 248 ++++++++++++++++++++++++++++++++++++++++-
>> 1 file changed, 245 insertions(+), 3 deletions(-)
>>
>> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
>> index 4369c55177..1cacbb6e43 100644
>> --- a/xen/arch/arm/vgic-v3.c
>> +++ b/xen/arch/arm/vgic-v3.c
>> @@ -111,7 +111,7 @@ static uint64_t vgic_fetch_irouter(struct
>> vgic_irq_rank *rank,
>> * Note the offset will be aligned to the appropriate boundary.
>> */
>> static void vgic_store_irouter(struct domain *d, struct
>> vgic_irq_rank *rank,
>> - unsigned int offset, uint64_t irouter)
>> + unsigned int offset, uint64_t irouter,
>> bool espi)
>> {
>> struct vcpu *new_vcpu, *old_vcpu;
>> unsigned int virq;
>> @@ -123,7 +123,8 @@ static void vgic_store_irouter(struct domain *d,
>> struct vgic_irq_rank *rank,
>> * The IROUTER0-31, used for SGIs/PPIs, are reserved and should
>> * never call this function.
>> */
>> - ASSERT(virq >= 32);
>> + if ( !espi )
>> + ASSERT(virq >= 32);
>> /* Get the index in the rank */
>> offset = virq & INTERRUPT_RANK_MASK;
>> @@ -146,6 +147,11 @@ static void vgic_store_irouter(struct domain *d,
>> struct vgic_irq_rank *rank,
>> /* Only migrate the IRQ if the target vCPU has changed */
>> if ( new_vcpu != old_vcpu )
>> {
>> +#ifdef CONFIG_GICV3_ESPI
>> + /* Convert virq index to eSPI range */
>> + if ( espi )
>> + virq = ESPI_IDX2INTID(virq);
>> +#endif
>> if ( vgic_migrate_irq(old_vcpu, new_vcpu, virq) )
>> write_atomic(&rank->vcpu[offset], new_vcpu->vcpu_id);
>> }
>> @@ -685,6 +691,9 @@ static int __vgic_v3_distr_common_mmio_read(const
>> char *name, struct vcpu *v,
>> {
>> case VRANGE32(GICD_IGROUPR, GICD_IGROUPRN):
>> case VRANGE32(GICD_IGRPMODR, GICD_IGRPMODRN):
>> +#ifdef CONFIG_GICV3_ESPI
>> + case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
>> +#endif
>> /* We do not implement security extensions for guests, read
>> zero */
>> if ( dabt.size != DABT_WORD ) goto bad_width;
>> goto read_as_zero;
>> @@ -710,11 +719,19 @@ static int
>> __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
>> /* Read the pending status of an IRQ via GICD/GICR is not
>> supported */
>> case VRANGE32(GICD_ISPENDR, GICD_ISPENDRN):
>> case VRANGE32(GICD_ICPENDR, GICD_ICPENDRN):
>> +#ifdef CONFIG_GICV3_ESPI
>> + case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
>> + case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
>> +#endif
>> goto read_as_zero;
>> /* Read the active status of an IRQ via GICD/GICR is not
>> supported */
>> case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
>> case VRANGE32(GICD_ICACTIVER, GICD_ICACTIVERN):
>> +#ifdef CONFIG_GICV3_ESPI
>> + case VRANGE32(GICD_ISACTIVERnE, GICD_ISACTIVERnEN):
>> + case VRANGE32(GICD_ICACTIVERnE, GICD_ICACTIVERnEN):
>> +#endif
>> goto read_as_zero;
>> case VRANGE32(GICD_IPRIORITYR, GICD_IPRIORITYRN):
>> @@ -752,6 +769,61 @@ static int __vgic_v3_distr_common_mmio_read(const
>> char *name, struct vcpu *v,
>> return 1;
>> }
>> +#ifdef CONFIG_GICV3_ESPI
>> + case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
>> + if ( dabt.size != DABT_WORD ) goto bad_width;
>
>
> NIT: If I am not mistaken, the goto should be on the next line (here and
> in similar places throughout the added code).
>
I just replicated some existing code with formatting for regular SPIs in
eSPI. According to CODING_STYLE, there is no explicit rule against
putting such code on one line (or I missed this point). However, in
examples for a single-statement block after an if:
> Braces should be omitted for blocks with a single statement. e.g.,
>
> if ( condition )
> single_statement();
>
The single statement is placed on a new line. So, I would prefer to
address your nit and add newlines here and in similar places in V3.
>> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ISENABLERnE,
>> DABT_WORD);
>> + if ( rank == NULL ) goto read_as_zero;
>> + vgic_lock_rank(v, rank, flags);
>> + *r = vreg_reg32_extract(rank->ienable, info);
>> + vgic_unlock_rank(v, rank, flags);
>> + return 1;
>> +
>> + case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
>> + if ( dabt.size != DABT_WORD ) goto bad_width;
>> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ICENABLERnE,
>> DABT_WORD);
>> + if ( rank == NULL ) goto read_as_zero;
>> + vgic_lock_rank(v, rank, flags);
>> + *r = vreg_reg32_extract(rank->ienable, info);
>> + vgic_unlock_rank(v, rank, flags);
>> + return 1;
>> +
>> + case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
>> + {
>> + uint32_t ipriorityr;
>> + uint8_t rank_index;
>> +
>> + if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto
>> bad_width;
>> + rank = vgic_ext_rank_offset(v, 8, reg - GICD_IPRIORITYRnE,
>> DABT_WORD);
>> + if ( rank == NULL ) goto read_as_zero;
>> + rank_index = REG_RANK_INDEX(8, reg - GICD_IPRIORITYRnE,
>> DABT_WORD);
>> +
>> + vgic_lock_rank(v, rank, flags);
>> + ipriorityr = ACCESS_ONCE(rank->ipriorityr[rank_index]);
>> + vgic_unlock_rank(v, rank, flags);
>> +
>> + *r = vreg_reg32_extract(ipriorityr, info);
>> +
>> + return 1;
>> + }
>> +
>> + case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
>> + {
>> + uint32_t icfgr;
>> +
>> + if ( dabt.size != DABT_WORD ) goto bad_width;
>> + rank = vgic_ext_rank_offset(v, 2, reg - GICD_ICFGRnE,
>> DABT_WORD);
>> + if ( rank == NULL ) goto read_as_zero;
>> + vgic_lock_rank(v, rank, flags);
>> + icfgr = rank->icfg[REG_RANK_INDEX(2, reg - GICD_ICFGRnE,
>> DABT_WORD)];
>> + vgic_unlock_rank(v, rank, flags);
>> +
>> + *r = vreg_reg32_extract(icfgr, info);
>> +
>> + return 1;
>> + }
>> +#endif
>> +
>> default:
>> printk(XENLOG_G_ERR
>> "%pv: %s: unhandled read r%d offset %#08x\n",
>> @@ -782,6 +854,9 @@ static int __vgic_v3_distr_common_mmio_write(const
>> char *name, struct vcpu *v,
>> {
>> case VRANGE32(GICD_IGROUPR, GICD_IGROUPRN):
>> case VRANGE32(GICD_IGRPMODR, GICD_IGRPMODRN):
>> +#ifdef CONFIG_GICV3_ESPI
>> + case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
>> +#endif
>> /* We do not implement security extensions for guests, write
>> ignore */
>> goto write_ignore_32;
>> @@ -871,6 +946,87 @@ static int
>> __vgic_v3_distr_common_mmio_write(const char *name, struct vcpu *v,
>> vgic_unlock_rank(v, rank, flags);
>> return 1;
>> +#ifdef CONFIG_GICV3_ESPI
>> + case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
>> + if ( dabt.size != DABT_WORD ) goto bad_width;
>> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ISENABLERnE,
>> DABT_WORD);
>> + if ( rank == NULL ) goto write_ignore;
>> + vgic_lock_rank(v, rank, flags);
>> + tr = rank->ienable;
>> + vreg_reg32_setbits(&rank->ienable, r, info);
>> + vgic_enable_irqs(v, (rank->ienable) & (~tr),
>> EXT_RANK_IDX2NUM(rank->index));
>> + vgic_unlock_rank(v, rank, flags);
>> + return 1;
>> +
>> + case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
>> + if ( dabt.size != DABT_WORD ) goto bad_width;
>> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ICENABLERnE,
>> DABT_WORD);
>> + if ( rank == NULL ) goto write_ignore;
>> + vgic_lock_rank(v, rank, flags);
>> + tr = rank->ienable;
>> + vreg_reg32_clearbits(&rank->ienable, r, info);
>> + vgic_disable_irqs(v, (~rank->ienable) & tr,
>> EXT_RANK_IDX2NUM(rank->index));
>> + vgic_unlock_rank(v, rank, flags);
>> + return 1;
>> +
>> + case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
>> + if ( dabt.size != DABT_WORD ) goto bad_width;
>> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ISPENDRnE,
>> DABT_WORD);
>> + if ( rank == NULL ) goto write_ignore;
>> +
>> + vgic_set_irqs_pending(v, r, EXT_RANK_IDX2NUM(rank->index));
>> +
>> + return 1;
>> +
>> + case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
>> + if ( dabt.size != DABT_WORD ) goto bad_width;
>> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ICPENDRnE,
>> DABT_WORD);
>> + if ( rank == NULL ) goto write_ignore;
>> +
>> + vgic_check_inflight_irqs_pending(v, EXT_RANK_IDX2NUM(rank-
>> >index), r);
>> +
>> + goto write_ignore;
>> + case VRANGE32(GICD_ISACTIVERnE, GICD_ISACTIVERnEN):
>> + if ( dabt.size != DABT_WORD ) goto bad_width;
>> + printk(XENLOG_G_ERR
>> + "%pv: %s: unhandled word write %#"PRIregister" to
>> ISACTIVER%d\n",
>
> I would use ISACTIVER%dE in the printed message to distinguish between
> normal and "extended" registers (here and in similar places throughout
> the added code).
>
>> + v, name, r, reg - GICD_ISACTIVERnE);
>> + return 0;
>> +
>> + case VRANGE32(GICD_ICACTIVERnE, GICD_ICACTIVERnEN):
>> + printk(XENLOG_G_ERR
>> + "%pv: %s: unhandled word write %#"PRIregister" to
>> ICACTIVER%d\n",
>> + v, name, r, reg - GICD_ICACTIVER);
>
> s/GICD_ICACTIVER/GICD_ICACTIVERnE
>
>
> [snip]
Oh, I missed this. I will fix it in V3. Thank you.
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v2 10/10] xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers
2025-08-07 12:33 ` [PATCH v2 10/10] xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers Leonid Komarianskyi
2025-08-21 16:17 ` Oleksandr Tyshchenko
@ 2025-08-21 17:26 ` Volodymyr Babchuk
2025-08-22 10:00 ` Leonid Komarianskyi
2025-08-25 14:07 ` Leonid Komarianskyi
1 sibling, 2 replies; 55+ messages in thread
From: Volodymyr Babchuk @ 2025-08-21 17:26 UTC (permalink / raw)
To: Leonid Komarianskyi
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Hi Leonid,
Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
> Implemented support for GICv3.1 extended SPI registers for vGICv3,
> allowing the emulation of eSPI-specific behavior for guest domains.
> The implementation includes read and write emulation for eSPI-related
> registers (e.g., GICD_ISENABLERnE, GICD_IROUTERnE, and others),
> following a similar approach to the handling of regular SPIs.
>
> The eSPI registers, previously located in reserved address ranges,
> are now adjusted to support MMIO read and write operations correctly
> when CONFIG_GICV3_ESPI is enabled.
>
> The availability of eSPIs and the number of emulated extended SPIs
> for guest domains is reported by setting the appropriate bits in the
> GICD_TYPER register, based on the number of eSPIs requested by the
> domain and supported by the hardware. In cases where the configuration
> option is disabled, the hardware does not support eSPIs, or the domain
> does not request such interrupts, the functionality remains unchanged.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>
> ---
> Changes in V2:
> - add missing rank index conversion for pending and inflight irqs
> ---
> xen/arch/arm/vgic-v3.c | 248 ++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 245 insertions(+), 3 deletions(-)
>
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index 4369c55177..1cacbb6e43 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -111,7 +111,7 @@ static uint64_t vgic_fetch_irouter(struct vgic_irq_rank *rank,
> * Note the offset will be aligned to the appropriate boundary.
> */
> static void vgic_store_irouter(struct domain *d, struct vgic_irq_rank *rank,
> - unsigned int offset, uint64_t irouter)
> + unsigned int offset, uint64_t irouter, bool espi)
I am wondering: maybe it is better to pass virq instead of offset into
this function? In that case you can get rid of espi parameter.
> {
> struct vcpu *new_vcpu, *old_vcpu;
> unsigned int virq;
> @@ -123,7 +123,8 @@ static void vgic_store_irouter(struct domain *d, struct vgic_irq_rank *rank,
> * The IROUTER0-31, used for SGIs/PPIs, are reserved and should
> * never call this function.
> */
> - ASSERT(virq >= 32);
> + if ( !espi )
> + ASSERT(virq >= 32);
better to write
ASSERT (!espi & (virq>=32))
and probably you need symmetrical ASSERT for espi == true
> /* Get the index in the rank */
> offset = virq & INTERRUPT_RANK_MASK;
> @@ -146,6 +147,11 @@ static void vgic_store_irouter(struct domain *d, struct vgic_irq_rank *rank,
> /* Only migrate the IRQ if the target vCPU has changed */
> if ( new_vcpu != old_vcpu )
> {
> +#ifdef CONFIG_GICV3_ESPI
> + /* Convert virq index to eSPI range */
> + if ( espi )
> + virq = ESPI_IDX2INTID(virq);
> +#endif
If you define ESPI_IDX2INTID() uncoditionally, you can get rid of #ifdef
CONFIG_GICV3_ESPI here
> if ( vgic_migrate_irq(old_vcpu, new_vcpu, virq) )
> write_atomic(&rank->vcpu[offset], new_vcpu->vcpu_id);
> }
> @@ -685,6 +691,9 @@ static int __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
> {
> case VRANGE32(GICD_IGROUPR, GICD_IGROUPRN):
> case VRANGE32(GICD_IGRPMODR, GICD_IGRPMODRN):
> +#ifdef CONFIG_GICV3_ESPI
Do you really need ifdef here?
> + case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
> +#endif
> /* We do not implement security extensions for guests, read zero */
> if ( dabt.size != DABT_WORD ) goto bad_width;
> goto read_as_zero;
> @@ -710,11 +719,19 @@ static int __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
> /* Read the pending status of an IRQ via GICD/GICR is not supported */
> case VRANGE32(GICD_ISPENDR, GICD_ISPENDRN):
> case VRANGE32(GICD_ICPENDR, GICD_ICPENDRN):
> +#ifdef CONFIG_GICV3_ESPI
Same as here
> + case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
> + case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
> +#endif
> goto read_as_zero;
>
> /* Read the active status of an IRQ via GICD/GICR is not supported */
> case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
> case VRANGE32(GICD_ICACTIVER, GICD_ICACTIVERN):
> +#ifdef CONFIG_GICV3_ESPI
... and here and in all other places
> + case VRANGE32(GICD_ISACTIVERnE, GICD_ISACTIVERnEN):
> + case VRANGE32(GICD_ICACTIVERnE, GICD_ICACTIVERnEN):
> +#endif
> goto read_as_zero;
>
> case VRANGE32(GICD_IPRIORITYR, GICD_IPRIORITYRN):
> @@ -752,6 +769,61 @@ static int __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
> return 1;
> }
>
> +#ifdef CONFIG_GICV3_ESPI
> + case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ISENABLERnE, DABT_WORD);
> + if ( rank == NULL ) goto read_as_zero;
> + vgic_lock_rank(v, rank, flags);
> + *r = vreg_reg32_extract(rank->ienable, info);
> + vgic_unlock_rank(v, rank, flags);
> + return 1;
> +
> + case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ICENABLERnE, DABT_WORD);
> + if ( rank == NULL ) goto read_as_zero;
> + vgic_lock_rank(v, rank, flags);
> + *r = vreg_reg32_extract(rank->ienable, info);
> + vgic_unlock_rank(v, rank, flags);
> + return 1;
> +
> + case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
> + {
> + uint32_t ipriorityr;
> + uint8_t rank_index;
> +
> + if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 8, reg - GICD_IPRIORITYRnE, DABT_WORD);
> + if ( rank == NULL ) goto read_as_zero;
> + rank_index = REG_RANK_INDEX(8, reg - GICD_IPRIORITYRnE, DABT_WORD);
> +
> + vgic_lock_rank(v, rank, flags);
> + ipriorityr = ACCESS_ONCE(rank->ipriorityr[rank_index]);
> + vgic_unlock_rank(v, rank, flags);
> +
> + *r = vreg_reg32_extract(ipriorityr, info);
> +
> + return 1;
> + }
> +
> + case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
> + {
> + uint32_t icfgr;
> +
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 2, reg - GICD_ICFGRnE, DABT_WORD);
> + if ( rank == NULL ) goto read_as_zero;
> + vgic_lock_rank(v, rank, flags);
> + icfgr = rank->icfg[REG_RANK_INDEX(2, reg - GICD_ICFGRnE, DABT_WORD)];
> + vgic_unlock_rank(v, rank, flags);
> +
> + *r = vreg_reg32_extract(icfgr, info);
> +
> + return 1;
> + }
> +#endif
> +
> default:
> printk(XENLOG_G_ERR
> "%pv: %s: unhandled read r%d offset %#08x\n",
> @@ -782,6 +854,9 @@ static int __vgic_v3_distr_common_mmio_write(const char *name, struct vcpu *v,
> {
> case VRANGE32(GICD_IGROUPR, GICD_IGROUPRN):
> case VRANGE32(GICD_IGRPMODR, GICD_IGRPMODRN):
> +#ifdef CONFIG_GICV3_ESPI
> + case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
> +#endif
> /* We do not implement security extensions for guests, write ignore */
> goto write_ignore_32;
>
> @@ -871,6 +946,87 @@ static int __vgic_v3_distr_common_mmio_write(const char *name, struct vcpu *v,
> vgic_unlock_rank(v, rank, flags);
> return 1;
>
> +#ifdef CONFIG_GICV3_ESPI
> + case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ISENABLERnE, DABT_WORD);
> + if ( rank == NULL ) goto write_ignore;
> + vgic_lock_rank(v, rank, flags);
> + tr = rank->ienable;
> + vreg_reg32_setbits(&rank->ienable, r, info);
> + vgic_enable_irqs(v, (rank->ienable) & (~tr), EXT_RANK_IDX2NUM(rank->index));
> + vgic_unlock_rank(v, rank, flags);
> + return 1;
> +
> + case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ICENABLERnE, DABT_WORD);
> + if ( rank == NULL ) goto write_ignore;
> + vgic_lock_rank(v, rank, flags);
> + tr = rank->ienable;
> + vreg_reg32_clearbits(&rank->ienable, r, info);
> + vgic_disable_irqs(v, (~rank->ienable) & tr, EXT_RANK_IDX2NUM(rank->index));
> + vgic_unlock_rank(v, rank, flags);
> + return 1;
> +
> + case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ISPENDRnE, DABT_WORD);
> + if ( rank == NULL ) goto write_ignore;
> +
> + vgic_set_irqs_pending(v, r, EXT_RANK_IDX2NUM(rank->index));
> +
> + return 1;
> +
> + case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 1, reg - GICD_ICPENDRnE, DABT_WORD);
> + if ( rank == NULL ) goto write_ignore;
> +
> + vgic_check_inflight_irqs_pending(v, EXT_RANK_IDX2NUM(rank->index), r);
> +
> + goto write_ignore;
> + case VRANGE32(GICD_ISACTIVERnE, GICD_ISACTIVERnEN):
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + printk(XENLOG_G_ERR
> + "%pv: %s: unhandled word write %#"PRIregister" to ISACTIVER%d\n",
> + v, name, r, reg - GICD_ISACTIVERnE);
> + return 0;
> +
> + case VRANGE32(GICD_ICACTIVERnE, GICD_ICACTIVERnEN):
> + printk(XENLOG_G_ERR
> + "%pv: %s: unhandled word write %#"PRIregister" to ICACTIVER%d\n",
> + v, name, r, reg - GICD_ICACTIVER);
> + goto write_ignore_32;
> +
> + case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
> + {
> + uint32_t *ipriorityr, priority;
> +
> + if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 8, reg - GICD_IPRIORITYRnE, DABT_WORD);
> + if ( rank == NULL ) goto write_ignore;
> + vgic_lock_rank(v, rank, flags);
> + ipriorityr = &rank->ipriorityr[REG_RANK_INDEX(8, reg - GICD_IPRIORITYRnE,
> + DABT_WORD)];
> + priority = ACCESS_ONCE(*ipriorityr);
> + vreg_reg32_update(&priority, r, info);
> + ACCESS_ONCE(*ipriorityr) = priority;
> + vgic_unlock_rank(v, rank, flags);
> + return 1;
> + }
> + case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
> + if ( dabt.size != DABT_WORD ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 2, reg - GICD_ICFGRnE, DABT_WORD);
> + if ( rank == NULL ) goto write_ignore;
> + vgic_lock_rank(v, rank, flags);
> + vreg_reg32_update(&rank->icfg[REG_RANK_INDEX(2, reg - GICD_ICFGRnE,
> + DABT_WORD)],
> + r, info);
> + vgic_unlock_rank(v, rank, flags);
> + return 1;
> +#endif
> +
> default:
> printk(XENLOG_G_ERR
> "%pv: %s: unhandled write r%d=%"PRIregister" offset %#08x\n",
> @@ -1129,6 +1285,16 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info,
> typer |= GICD_TYPE_LPIS;
>
> typer |= (v->domain->arch.vgic.intid_bits - 1) << GICD_TYPE_ID_BITS_SHIFT;
> +#ifdef CONFIG_GICV3_ESPI
> + if ( v->domain->arch.vgic.nr_espis > 0 )
> + {
> + /* Set eSPI support bit for the domain */
> + typer |= GICD_TYPER_ESPI;
> + /* Set ESPI range bits */
> + typer |= (DIV_ROUND_UP(v->domain->arch.vgic.nr_espis, 32) - 1)
> + << GICD_TYPER_ESPI_RANGE_SHIFT;
> + }
> +#endif
>
> *r = vreg_reg32_extract(typer, info);
>
> @@ -1194,6 +1360,18 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info,
> case VRANGE32(GICD_IPRIORITYR, GICD_IPRIORITYRN):
> case VRANGE32(GICD_ICFGR, GICD_ICFGRN):
> case VRANGE32(GICD_IGRPMODR, GICD_IGRPMODRN):
> +
> +#ifdef CONFIG_GICV3_ESPI
> + case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
> + case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
> + case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
> + case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
> + case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
> + case VRANGE32(GICD_ISACTIVERnE, GICD_ISACTIVERnEN):
> + case VRANGE32(GICD_ICACTIVERnE, GICD_ICACTIVERnEN):
> + case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
> + case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
> +#endif
> /*
> * Above all register are common with GICR and GICD
> * Manage in common
> @@ -1216,7 +1394,11 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info,
> /* Replaced with GICR_ISPENDR0. So ignore write */
> goto read_as_zero_32;
>
> +#ifdef CONFIG_GICV3_ESPI
> + case VRANGE32(0x3100, 0x60FC):
> +#else
> case VRANGE32(0x0F30, 0x60FC):
> +#endif
> goto read_reserved;
>
> case VRANGE64(GICD_IROUTER32, GICD_IROUTER1019):
> @@ -1235,8 +1417,30 @@ static int vgic_v3_distr_mmio_read(struct vcpu *v, mmio_info_t *info,
>
> return 1;
> }
> +#ifdef CONFIG_GICV3_ESPI
> + case VRANGE64(GICD_IROUTERnE, GICD_IROUTERnEN):
> + {
> + uint64_t irouter;
> +
> + if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 64, gicd_reg - GICD_IROUTERnE,
> + DABT_DOUBLE_WORD);
> + if ( rank == NULL ) goto read_as_zero;
> + vgic_lock_rank(v, rank, flags);
> + irouter = vgic_fetch_irouter(rank, gicd_reg - GICD_IROUTERnE);
> + vgic_unlock_rank(v, rank, flags);
> +
> + *r = vreg_reg64_extract(irouter, info);
> +
> + return 1;
> + }
> +#endif
>
> +#ifdef CONFIG_GICV3_ESPI
> + case VRANGE32(0xA004, 0xBFFC):
> +#else
> case VRANGE32(0x7FE0, 0xBFFC):
> +#endif
> goto read_reserved;
>
> case VRANGE32(0xC000, 0xFFCC):
> @@ -1382,6 +1586,18 @@ static int vgic_v3_distr_mmio_write(struct vcpu *v, mmio_info_t *info,
> case VRANGE32(GICD_IPRIORITYR, GICD_IPRIORITYRN):
> case VRANGE32(GICD_ICFGR, GICD_ICFGRN):
> case VRANGE32(GICD_IGRPMODR, GICD_IGRPMODRN):
> +
> +#ifdef CONFIG_GICV3_ESPI
> + case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
> + case VRANGE32(GICD_ISENABLERnE, GICD_ISENABLERnEN):
> + case VRANGE32(GICD_ICENABLERnE, GICD_ICENABLERnEN):
> + case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
> + case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
> + case VRANGE32(GICD_ISACTIVERnE, GICD_ISACTIVERnEN):
> + case VRANGE32(GICD_ICACTIVERnE, GICD_ICACTIVERnEN):
> + case VRANGE32(GICD_IPRIORITYRnE, GICD_IPRIORITYRnEN):
> + case VRANGE32(GICD_ICFGRnE, GICD_ICFGRnEN):
> +#endif
> /* Above registers are common with GICR and GICD
> * Manage in common */
> return __vgic_v3_distr_common_mmio_write("vGICD", v, info,
> @@ -1405,7 +1621,11 @@ static int vgic_v3_distr_mmio_write(struct vcpu *v, mmio_info_t *info,
> if ( dabt.size != DABT_WORD ) goto bad_width;
> return 0;
>
> +#ifdef CONFIG_GICV3_ESPI
> + case VRANGE32(0x3100, 0x60FC):
> +#else
> case VRANGE32(0x0F30, 0x60FC):
> +#endif
> goto write_reserved;
>
> case VRANGE64(GICD_IROUTER32, GICD_IROUTER1019):
> @@ -1419,12 +1639,34 @@ static int vgic_v3_distr_mmio_write(struct vcpu *v, mmio_info_t *info,
> vgic_lock_rank(v, rank, flags);
> irouter = vgic_fetch_irouter(rank, gicd_reg - GICD_IROUTER);
> vreg_reg64_update(&irouter, r, info);
> - vgic_store_irouter(v->domain, rank, gicd_reg - GICD_IROUTER, irouter);
> + vgic_store_irouter(v->domain, rank, gicd_reg - GICD_IROUTER, irouter, false);
> + vgic_unlock_rank(v, rank, flags);
> + return 1;
> + }
> +
> +#ifdef CONFIG_GICV3_ESPI
> + case VRANGE64(GICD_IROUTERnE, GICD_IROUTERnEN):
> + {
> + uint64_t irouter;
> +
> + if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
> + rank = vgic_ext_rank_offset(v, 64, gicd_reg - GICD_IROUTERnE,
> + DABT_DOUBLE_WORD);
> + if ( rank == NULL ) goto write_ignore;
> + vgic_lock_rank(v, rank, flags);
> + irouter = vgic_fetch_irouter(rank, gicd_reg - GICD_IROUTERnE);
> + vreg_reg64_update(&irouter, r, info);
> + vgic_store_irouter(v->domain, rank, gicd_reg - GICD_IROUTERnE, irouter, true);
> vgic_unlock_rank(v, rank, flags);
> return 1;
> }
> +#endif
>
> +#ifdef CONFIG_GICV3_ESPI
> + case VRANGE32(0xA004, 0xBFFC):
> +#else
> case VRANGE32(0x7FE0, 0xBFFC):
> +#endif
> goto write_reserved;
>
> case VRANGE32(0xC000, 0xFFCC):
--
WBR, Volodymyr
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 10/10] xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers
2025-08-21 17:26 ` Volodymyr Babchuk
@ 2025-08-22 10:00 ` Leonid Komarianskyi
2025-08-25 14:07 ` Leonid Komarianskyi
1 sibling, 0 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-22 10:00 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Hi Volodymyr,
Thank you for your close review.
On 21.08.25 20:26, Volodymyr Babchuk wrote:
>
> Hi Leonid,
>
> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>
>> Implemented support for GICv3.1 extended SPI registers for vGICv3,
>> allowing the emulation of eSPI-specific behavior for guest domains.
>> The implementation includes read and write emulation for eSPI-related
>> registers (e.g., GICD_ISENABLERnE, GICD_IROUTERnE, and others),
>> following a similar approach to the handling of regular SPIs.
>>
>> The eSPI registers, previously located in reserved address ranges,
>> are now adjusted to support MMIO read and write operations correctly
>> when CONFIG_GICV3_ESPI is enabled.
>>
>> The availability of eSPIs and the number of emulated extended SPIs
>> for guest domains is reported by setting the appropriate bits in the
>> GICD_TYPER register, based on the number of eSPIs requested by the
>> domain and supported by the hardware. In cases where the configuration
>> option is disabled, the hardware does not support eSPIs, or the domain
>> does not request such interrupts, the functionality remains unchanged.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V2:
>> - add missing rank index conversion for pending and inflight irqs
>> ---
>> xen/arch/arm/vgic-v3.c | 248 ++++++++++++++++++++++++++++++++++++++++-
>> 1 file changed, 245 insertions(+), 3 deletions(-)
>>
>> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
>> index 4369c55177..1cacbb6e43 100644
>> --- a/xen/arch/arm/vgic-v3.c
>> +++ b/xen/arch/arm/vgic-v3.c
>> @@ -111,7 +111,7 @@ static uint64_t vgic_fetch_irouter(struct vgic_irq_rank *rank,
>> * Note the offset will be aligned to the appropriate boundary.
>> */
>> static void vgic_store_irouter(struct domain *d, struct vgic_irq_rank *rank,
>> - unsigned int offset, uint64_t irouter)
>> + unsigned int offset, uint64_t irouter, bool espi)
>
> I am wondering: maybe it is better to pass virq instead of offset into
> this function? In that case you can get rid of espi parameter.
>
Yes, it makes sense, because in any case we recalculate the offset
later, after getting the virq:
> /* Get the index in the rank */
> offset = virq & INTERRUPT_RANK_MASK;
And as a bonus, I can get rid of the espi parameter. However, it
requires the caller to calculate the virq by itself.
If applicable, I can add one more preparation patch in V3 before
actually implementing eSPI with such changes. Would that be better?
>> {
>> struct vcpu *new_vcpu, *old_vcpu;
>> unsigned int virq;
>> @@ -123,7 +123,8 @@ static void vgic_store_irouter(struct domain *d, struct vgic_irq_rank *rank,
>> * The IROUTER0-31, used for SGIs/PPIs, are reserved and should
>> * never call this function.
>> */
>> - ASSERT(virq >= 32);
>> + if ( !espi )
>> + ASSERT(virq >= 32);
>
> better to write
> ASSERT (!espi & (virq>=32))
>
I agree with that, it looks much better. I will change it in V3 or drop
at all if it would be okay to use virq as a parameter.
> and probably you need symmetrical ASSERT for espi == true
This is not needed for eSPI because, unlike regular SPIs that have
indexes starting from 32, eSPI INTIDs start from 4096 and cover all 1024
IRQ lines.
>
>> /* Get the index in the rank */
>> offset = virq & INTERRUPT_RANK_MASK;
>> @@ -146,6 +147,11 @@ static void vgic_store_irouter(struct domain *d, struct vgic_irq_rank *rank,
>> /* Only migrate the IRQ if the target vCPU has changed */
>> if ( new_vcpu != old_vcpu )
>> {
>> +#ifdef CONFIG_GICV3_ESPI
>> + /* Convert virq index to eSPI range */
>> + if ( espi )
>> + virq = ESPI_IDX2INTID(virq);
>> +#endif
>
> If you define ESPI_IDX2INTID() uncoditionally, you can get rid of #ifdef
> CONFIG_GICV3_ESPI here
>
Actually, ESPI_IDX2INTID is defined under ifdef condition:
> #ifdef CONFIG_GICV3_ESPI
> #define ESPI_BASE_INTID 4096
> #define ESPI_MAX_INTID 5119
> #define ESPI_INTID2IDX(intid) ((intid) - ESPI_BASE_INTID)
> #define ESPI_IDX2INTID(idx) ((idx) + ESPI_BASE_INTID)
> #endif
So it should be used under CONFIG_GICV3_ESPI, like in other places.
>> if ( vgic_migrate_irq(old_vcpu, new_vcpu, virq) )
>> write_atomic(&rank->vcpu[offset], new_vcpu->vcpu_id);
>> }
>> @@ -685,6 +691,9 @@ static int __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
>> {
>> case VRANGE32(GICD_IGROUPR, GICD_IGROUPRN):
>> case VRANGE32(GICD_IGRPMODR, GICD_IGRPMODRN):
>> +#ifdef CONFIG_GICV3_ESPI
>
> Do you really need ifdef here?
>
Oh, yes, this ifdef is redundant because, without CONFIG_GICV3_ESPI
enabled, we cannot reach this function with eSPI-specific offsets from
the caller.
So, yes, I will remove this ifdef in V3.
>> + case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
>> +#endif
>
>
>> /* We do not implement security extensions for guests, read zero */
>> if ( dabt.size != DABT_WORD ) goto bad_width;
>> goto read_as_zero;
>> @@ -710,11 +719,19 @@ static int __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
>> /* Read the pending status of an IRQ via GICD/GICR is not supported */
>> case VRANGE32(GICD_ISPENDR, GICD_ISPENDRN):
>> case VRANGE32(GICD_ICPENDR, GICD_ICPENDRN):
>> +#ifdef CONFIG_GICV3_ESPI
>
> Same as here
I will remove it in V3.
>
>> + case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
>> + case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
>> +#endif
>> goto read_as_zero;
>>
>> /* Read the active status of an IRQ via GICD/GICR is not supported */
>> case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
>> case VRANGE32(GICD_ICACTIVER, GICD_ICACTIVERN):
>> +#ifdef CONFIG_GICV3_ESPI
>
> ... and here and in all other places
>
Thank you, that's a really good point. I will revise all the places with
ifdefs in this patch and fix them in V3.
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 10/10] xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers
2025-08-21 17:26 ` Volodymyr Babchuk
2025-08-22 10:00 ` Leonid Komarianskyi
@ 2025-08-25 14:07 ` Leonid Komarianskyi
1 sibling, 0 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-25 14:07 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Hello Volodymyr,
On 21.08.25 20:26, Volodymyr Babchuk wrote:
>
> Hi Leonid,
>
> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>
>> Implemented support for GICv3.1 extended SPI registers for vGICv3,
>> allowing the emulation of eSPI-specific behavior for guest domains.
>> The implementation includes read and write emulation for eSPI-related
>> registers (e.g., GICD_ISENABLERnE, GICD_IROUTERnE, and others),
>> following a similar approach to the handling of regular SPIs.
>>
>> The eSPI registers, previously located in reserved address ranges,
>> are now adjusted to support MMIO read and write operations correctly
>> when CONFIG_GICV3_ESPI is enabled.
>>
>> The availability of eSPIs and the number of emulated extended SPIs
>> for guest domains is reported by setting the appropriate bits in the
>> GICD_TYPER register, based on the number of eSPIs requested by the
>> domain and supported by the hardware. In cases where the configuration
>> option is disabled, the hardware does not support eSPIs, or the domain
>> does not request such interrupts, the functionality remains unchanged.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V2:
>> - add missing rank index conversion for pending and inflight irqs
>> ---
>> xen/arch/arm/vgic-v3.c | 248 ++++++++++++++++++++++++++++++++++++++++-
>> 1 file changed, 245 insertions(+), 3 deletions(-)
>>
>> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
>> index 4369c55177..1cacbb6e43 100644
>> --- a/xen/arch/arm/vgic-v3.c
>> +++ b/xen/arch/arm/vgic-v3.c
>> @@ -111,7 +111,7 @@ static uint64_t vgic_fetch_irouter(struct vgic_irq_rank *rank,
>> * Note the offset will be aligned to the appropriate boundary.
>> */
>> static void vgic_store_irouter(struct domain *d, struct vgic_irq_rank *rank,
>> - unsigned int offset, uint64_t irouter)
>> + unsigned int offset, uint64_t irouter, bool espi)
>
> I am wondering: maybe it is better to pass virq instead of offset into
> this function? In that case you can get rid of espi parameter.
>
>> {
>> struct vcpu *new_vcpu, *old_vcpu;
>> unsigned int virq;
>> @@ -123,7 +123,8 @@ static void vgic_store_irouter(struct domain *d, struct vgic_irq_rank *rank,
>> * The IROUTER0-31, used for SGIs/PPIs, are reserved and should
>> * never call this function.
>> */
>> - ASSERT(virq >= 32);
>> + if ( !espi )
>> + ASSERT(virq >= 32);
>
> better to write
> ASSERT (!espi & (virq>=32))
>
> and probably you need symmetrical ASSERT for espi == true
>
>> /* Get the index in the rank */
>> offset = virq & INTERRUPT_RANK_MASK;
>> @@ -146,6 +147,11 @@ static void vgic_store_irouter(struct domain *d, struct vgic_irq_rank *rank,
>> /* Only migrate the IRQ if the target vCPU has changed */
>> if ( new_vcpu != old_vcpu )
>> {
>> +#ifdef CONFIG_GICV3_ESPI
>> + /* Convert virq index to eSPI range */
>> + if ( espi )
>> + virq = ESPI_IDX2INTID(virq);
>> +#endif
>
> If you define ESPI_IDX2INTID() uncoditionally, you can get rid of #ifdef
> CONFIG_GICV3_ESPI here
>
>> if ( vgic_migrate_irq(old_vcpu, new_vcpu, virq) )
>> write_atomic(&rank->vcpu[offset], new_vcpu->vcpu_id);
>> }
>> @@ -685,6 +691,9 @@ static int __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
>> {
>> case VRANGE32(GICD_IGROUPR, GICD_IGROUPRN):
>> case VRANGE32(GICD_IGRPMODR, GICD_IGRPMODRN):
>> +#ifdef CONFIG_GICV3_ESPI
>
> Do you really need ifdef here?
>
>> + case VRANGE32(GICD_IGROUPRnE, GICD_IGROUPRnEN):
>> +#endif
>
>
>> /* We do not implement security extensions for guests, read zero */
>> if ( dabt.size != DABT_WORD ) goto bad_width;
>> goto read_as_zero;
>> @@ -710,11 +719,19 @@ static int __vgic_v3_distr_common_mmio_read(const char *name, struct vcpu *v,
>> /* Read the pending status of an IRQ via GICD/GICR is not supported */
>> case VRANGE32(GICD_ISPENDR, GICD_ISPENDRN):
>> case VRANGE32(GICD_ICPENDR, GICD_ICPENDRN):
>> +#ifdef CONFIG_GICV3_ESPI
>
> Same as here
>
>> + case VRANGE32(GICD_ISPENDRnE, GICD_ISPENDRnEN):
>> + case VRANGE32(GICD_ICPENDRnE, GICD_ICPENDRnEN):
>> +#endif
>> goto read_as_zero;
>>
>> /* Read the active status of an IRQ via GICD/GICR is not supported */
>> case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
>> case VRANGE32(GICD_ICACTIVER, GICD_ICACTIVERN):
>> +#ifdef CONFIG_GICV3_ESPI
>
> ... and here and in all other places
>
I have rechecked the idea of removing some of the #ifdefs, and
unfortunately, I cannot simply do this as is here, because in this case,
I would need to move the register offsets out of the CONFIG_GICV3_ESPI
guard in the previous patch in the series. Please see:
[PATCH v2 05/10] xen/arm: gicv3: implement handling of GICv3.1 eSPI
As a result, with CONFIG_GICV3_ESPI disabled, we would have many unused
defines and also unreachable code in __vgic_v3_distr_common_mmio_read
and __vgic_v3_distr_common_mmio_write (I think the MISRA team will not
be happy about this..). Thus, I prefer to leave the #ifdefs as they are.
However, if you think it is better to remove the #ifdefs, I will update
the appropriate patch to make these changes compilable with fewer
#ifdefs and CONFIG_GICV3_ESPI disabled.
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread
* [PATCH v2 09/10] xen/arm: domain_build: adjust Dom0 IRQ handling to support eSPIs
2025-08-07 12:33 ` [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations Leonid Komarianskyi
` (7 preceding siblings ...)
2025-08-07 12:33 ` [PATCH v2 10/10] xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers Leonid Komarianskyi
@ 2025-08-07 12:33 ` Leonid Komarianskyi
2025-08-21 16:46 ` Volodymyr Babchuk
2025-08-23 13:02 ` Oleksandr Tyshchenko
2025-08-21 16:00 ` [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations Volodymyr Babchuk
2025-08-21 16:14 ` Julien Grall
10 siblings, 2 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-07 12:33 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Leonid Komarianskyi, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk
The Dom0 configuration logic in create_dom0() has been updated
to account for extended SPIs when supported by the hardware and
enabled with CONFIG_GICV3_ESPI. These changes ensure the proper
calculation of the maximum number of SPIs and eSPIs available for Dom0.
When eSPIs are supported by the hardware and CONFIG_GICV3_ESPI is
enabled, the maximum number of eSPI interrupts is calculated using
the ESPI_BASE_INTID offset (4096) and limited at 1024, with 32 IRQs
subtracted. To ensure compatibility with non-Dom0 domains, this
adjustment is applied by the toolstack during domain creation, while
for Dom0 it is handled directly during VGIC initialization. If eSPIs
are not supported, the calculation defaults to using the standard SPI
range, with a maximum value of 992 interrupt lines as it works now.
Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
---
Changes in V2:
- no changes
---
xen/arch/arm/domain_build.c | 10 ++++++++++
xen/arch/arm/include/asm/vgic.h | 11 +++++++++++
2 files changed, 21 insertions(+)
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index d91a71acfd..fa5abf2dfb 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2055,6 +2055,16 @@ void __init create_dom0(void)
/* The vGIC for DOM0 is exactly emulating the hardware GIC */
dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
dom0_cfg.arch.nr_spis = VGIC_DEF_NR_SPIS;
+#ifdef CONFIG_GICV3_ESPI
+ /*
+ * Check if the hardware supports extended SPIs (even if the appropriate config is set).
+ * If not, the common SPI range will be used. Otherwise overwrite the nr_spis with the
+ * maximum available INTID from eSPI range. In that case, the number of regular SPIs will
+ * be adjusted to the maximum value during vGIC initialization.
+ */
+ if ( gic_number_espis() > 0 )
+ dom0_cfg.arch.nr_spis = VGIC_DEF_NR_ESPIS;
+#endif
dom0_cfg.arch.tee_type = tee_get_type();
dom0_cfg.max_vcpus = dom0_max_vcpus();
diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
index 9fa4523018..117b3aa92c 100644
--- a/xen/arch/arm/include/asm/vgic.h
+++ b/xen/arch/arm/include/asm/vgic.h
@@ -353,6 +353,17 @@ extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
/* Default number of vGIC SPIs. 32 are substracted to cover local IRQs. */
#define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
+#ifdef CONFIG_GICV3_ESPI
+/*
+ * Returns the maximum eSPI INTID subtracted by 32. For non-Dom0 domains, the
+ * toolstack applies the same adjustment to cover local IRQs. We will add back
+ * this value during VGIC initialization. This ensures consistent handling for Dom0
+ * and other domains. For the regular SPI range interrupts in this case, the maximum
+ * value of VGIC_DEF_NR_SPIS will be used.
+ */
+#define VGIC_DEF_NR_ESPIS (ESPI_BASE_INTID + min(gic_number_espis(), 1024U) - 32)
+#endif
+
extern bool vgic_is_valid_irq(struct domain *d, unsigned int virq);
static inline bool vgic_is_shared_irq(struct domain *d, unsigned int virq)
--
2.34.1
^ permalink raw reply related [flat|nested] 55+ messages in thread* Re: [PATCH v2 09/10] xen/arm: domain_build: adjust Dom0 IRQ handling to support eSPIs
2025-08-07 12:33 ` [PATCH v2 09/10] xen/arm: domain_build: adjust Dom0 IRQ handling to support eSPIs Leonid Komarianskyi
@ 2025-08-21 16:46 ` Volodymyr Babchuk
2025-08-22 7:08 ` Leonid Komarianskyi
2025-08-23 13:02 ` Oleksandr Tyshchenko
1 sibling, 1 reply; 55+ messages in thread
From: Volodymyr Babchuk @ 2025-08-21 16:46 UTC (permalink / raw)
To: Leonid Komarianskyi
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
> The Dom0 configuration logic in create_dom0() has been updated
> to account for extended SPIs when supported by the hardware and
> enabled with CONFIG_GICV3_ESPI. These changes ensure the proper
> calculation of the maximum number of SPIs and eSPIs available for Dom0.
>
> When eSPIs are supported by the hardware and CONFIG_GICV3_ESPI is
> enabled, the maximum number of eSPI interrupts is calculated using
> the ESPI_BASE_INTID offset (4096) and limited at 1024, with 32 IRQs
> subtracted. To ensure compatibility with non-Dom0 domains, this
> adjustment is applied by the toolstack during domain creation, while
> for Dom0 it is handled directly during VGIC initialization. If eSPIs
> are not supported, the calculation defaults to using the standard SPI
> range, with a maximum value of 992 interrupt lines as it works now.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>
> ---
> Changes in V2:
> - no changes
> ---
> xen/arch/arm/domain_build.c | 10 ++++++++++
> xen/arch/arm/include/asm/vgic.h | 11 +++++++++++
> 2 files changed, 21 insertions(+)
>
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index d91a71acfd..fa5abf2dfb 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -2055,6 +2055,16 @@ void __init create_dom0(void)
> /* The vGIC for DOM0 is exactly emulating the hardware GIC */
> dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
> dom0_cfg.arch.nr_spis = VGIC_DEF_NR_SPIS;
> +#ifdef CONFIG_GICV3_ESPI
> + /*
> + * Check if the hardware supports extended SPIs (even if the appropriate config is set).
> + * If not, the common SPI range will be used. Otherwise overwrite the nr_spis with the
> + * maximum available INTID from eSPI range. In that case, the number of regular SPIs will
> + * be adjusted to the maximum value during vGIC initialization.
> + */
> + if ( gic_number_espis() > 0 )
> + dom0_cfg.arch.nr_spis = VGIC_DEF_NR_ESPIS;
> +#endif
> dom0_cfg.arch.tee_type = tee_get_type();
> dom0_cfg.max_vcpus = dom0_max_vcpus();
>
> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
> index 9fa4523018..117b3aa92c 100644
> --- a/xen/arch/arm/include/asm/vgic.h
> +++ b/xen/arch/arm/include/asm/vgic.h
> @@ -353,6 +353,17 @@ extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
> /* Default number of vGIC SPIs. 32 are substracted to cover local IRQs. */
> #define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
>
> +#ifdef CONFIG_GICV3_ESPI
> +/*
> + * Returns the maximum eSPI INTID subtracted by 32. For non-Dom0 domains, the
> + * toolstack applies the same adjustment to cover local IRQs. We will add back
> + * this value during VGIC initialization. This ensures consistent handling for Dom0
> + * and other domains. For the regular SPI range interrupts in this case, the maximum
> + * value of VGIC_DEF_NR_SPIS will be used.
> + */
> +#define VGIC_DEF_NR_ESPIS (ESPI_BASE_INTID + min(gic_number_espis(), 1024U) - 32)
Name of the define is wrong, as it is not number of eSPIs. Actually, this is
maximum SPI (including eSPIs) number.
> +#endif
> +
> extern bool vgic_is_valid_irq(struct domain *d, unsigned int virq);
>
> static inline bool vgic_is_shared_irq(struct domain *d, unsigned int virq)
--
WBR, Volodymyr
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v2 09/10] xen/arm: domain_build: adjust Dom0 IRQ handling to support eSPIs
2025-08-21 16:46 ` Volodymyr Babchuk
@ 2025-08-22 7:08 ` Leonid Komarianskyi
2025-08-22 12:26 ` Volodymyr Babchuk
0 siblings, 1 reply; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-22 7:08 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Hi Volodymyr,
On 21.08.25 19:46, Volodymyr Babchuk wrote:
>
>
>
> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>
>> The Dom0 configuration logic in create_dom0() has been updated
>> to account for extended SPIs when supported by the hardware and
>> enabled with CONFIG_GICV3_ESPI. These changes ensure the proper
>> calculation of the maximum number of SPIs and eSPIs available for Dom0.
>>
>> When eSPIs are supported by the hardware and CONFIG_GICV3_ESPI is
>> enabled, the maximum number of eSPI interrupts is calculated using
>> the ESPI_BASE_INTID offset (4096) and limited at 1024, with 32 IRQs
>> subtracted. To ensure compatibility with non-Dom0 domains, this
>> adjustment is applied by the toolstack during domain creation, while
>> for Dom0 it is handled directly during VGIC initialization. If eSPIs
>> are not supported, the calculation defaults to using the standard SPI
>> range, with a maximum value of 992 interrupt lines as it works now.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V2:
>> - no changes
>> ---
>> xen/arch/arm/domain_build.c | 10 ++++++++++
>> xen/arch/arm/include/asm/vgic.h | 11 +++++++++++
>> 2 files changed, 21 insertions(+)
>>
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index d91a71acfd..fa5abf2dfb 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -2055,6 +2055,16 @@ void __init create_dom0(void)
>> /* The vGIC for DOM0 is exactly emulating the hardware GIC */
>> dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
>> dom0_cfg.arch.nr_spis = VGIC_DEF_NR_SPIS;
>> +#ifdef CONFIG_GICV3_ESPI
>> + /*
>> + * Check if the hardware supports extended SPIs (even if the appropriate config is set).
>> + * If not, the common SPI range will be used. Otherwise overwrite the nr_spis with the
>> + * maximum available INTID from eSPI range. In that case, the number of regular SPIs will
>> + * be adjusted to the maximum value during vGIC initialization.
>> + */
>> + if ( gic_number_espis() > 0 )
>> + dom0_cfg.arch.nr_spis = VGIC_DEF_NR_ESPIS;
>> +#endif
>> dom0_cfg.arch.tee_type = tee_get_type();
>> dom0_cfg.max_vcpus = dom0_max_vcpus();
>>
>> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
>> index 9fa4523018..117b3aa92c 100644
>> --- a/xen/arch/arm/include/asm/vgic.h
>> +++ b/xen/arch/arm/include/asm/vgic.h
>> @@ -353,6 +353,17 @@ extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
>> /* Default number of vGIC SPIs. 32 are substracted to cover local IRQs. */
>> #define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
>>
>> +#ifdef CONFIG_GICV3_ESPI
>> +/*
>> + * Returns the maximum eSPI INTID subtracted by 32. For non-Dom0 domains, the
>> + * toolstack applies the same adjustment to cover local IRQs. We will add back
>> + * this value during VGIC initialization. This ensures consistent handling for Dom0
>> + * and other domains. For the regular SPI range interrupts in this case, the maximum
>> + * value of VGIC_DEF_NR_SPIS will be used.
>> + */
>> +#define VGIC_DEF_NR_ESPIS (ESPI_BASE_INTID + min(gic_number_espis(), 1024U) - 32)
> Name of the define is wrong, as it is not number of eSPIs. Actually, this is
> maximum SPI (including eSPIs) number.
Thank you for your review.
Would it be okay if I rename this macro to VGIC_DEF_MAX_SPI?
>
>> +#endif
>> +
>> extern bool vgic_is_valid_irq(struct domain *d, unsigned int virq);
>>
>> static inline bool vgic_is_shared_irq(struct domain *d, unsigned int virq)
>
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v2 09/10] xen/arm: domain_build: adjust Dom0 IRQ handling to support eSPIs
2025-08-22 7:08 ` Leonid Komarianskyi
@ 2025-08-22 12:26 ` Volodymyr Babchuk
2025-08-22 15:03 ` Leonid Komarianskyi
0 siblings, 1 reply; 55+ messages in thread
From: Volodymyr Babchuk @ 2025-08-22 12:26 UTC (permalink / raw)
To: Leonid Komarianskyi
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Hi Leonid,
Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
> Hi Volodymyr,
>
> On 21.08.25 19:46, Volodymyr Babchuk wrote:
>>
>>
>>
>> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>>
>>> The Dom0 configuration logic in create_dom0() has been updated
>>> to account for extended SPIs when supported by the hardware and
>>> enabled with CONFIG_GICV3_ESPI. These changes ensure the proper
>>> calculation of the maximum number of SPIs and eSPIs available for Dom0.
>>>
>>> When eSPIs are supported by the hardware and CONFIG_GICV3_ESPI is
>>> enabled, the maximum number of eSPI interrupts is calculated using
>>> the ESPI_BASE_INTID offset (4096) and limited at 1024, with 32 IRQs
>>> subtracted. To ensure compatibility with non-Dom0 domains, this
>>> adjustment is applied by the toolstack during domain creation, while
>>> for Dom0 it is handled directly during VGIC initialization. If eSPIs
>>> are not supported, the calculation defaults to using the standard SPI
>>> range, with a maximum value of 992 interrupt lines as it works now.
>>>
>>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>>
>>> ---
>>> Changes in V2:
>>> - no changes
>>> ---
>>> xen/arch/arm/domain_build.c | 10 ++++++++++
>>> xen/arch/arm/include/asm/vgic.h | 11 +++++++++++
>>> 2 files changed, 21 insertions(+)
>>>
>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>> index d91a71acfd..fa5abf2dfb 100644
>>> --- a/xen/arch/arm/domain_build.c
>>> +++ b/xen/arch/arm/domain_build.c
>>> @@ -2055,6 +2055,16 @@ void __init create_dom0(void)
>>> /* The vGIC for DOM0 is exactly emulating the hardware GIC */
>>> dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
>>> dom0_cfg.arch.nr_spis = VGIC_DEF_NR_SPIS;
>>> +#ifdef CONFIG_GICV3_ESPI
>>> + /*
>>> + * Check if the hardware supports extended SPIs (even if the appropriate config is set).
>>> + * If not, the common SPI range will be used. Otherwise overwrite the nr_spis with the
>>> + * maximum available INTID from eSPI range. In that case, the number of regular SPIs will
>>> + * be adjusted to the maximum value during vGIC initialization.
>>> + */
>>> + if ( gic_number_espis() > 0 )
>>> + dom0_cfg.arch.nr_spis = VGIC_DEF_NR_ESPIS;
>>> +#endif
>>> dom0_cfg.arch.tee_type = tee_get_type();
>>> dom0_cfg.max_vcpus = dom0_max_vcpus();
>>>
>>> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
>>> index 9fa4523018..117b3aa92c 100644
>>> --- a/xen/arch/arm/include/asm/vgic.h
>>> +++ b/xen/arch/arm/include/asm/vgic.h
>>> @@ -353,6 +353,17 @@ extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
>>> /* Default number of vGIC SPIs. 32 are substracted to cover local IRQs. */
>>> #define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
>>>
>>> +#ifdef CONFIG_GICV3_ESPI
>>> +/*
>>> + * Returns the maximum eSPI INTID subtracted by 32. For non-Dom0 domains, the
>>> + * toolstack applies the same adjustment to cover local IRQs. We will add back
>>> + * this value during VGIC initialization. This ensures consistent handling for Dom0
>>> + * and other domains. For the regular SPI range interrupts in this case, the maximum
>>> + * value of VGIC_DEF_NR_SPIS will be used.
>>> + */
>>> +#define VGIC_DEF_NR_ESPIS (ESPI_BASE_INTID + min(gic_number_espis(), 1024U) - 32)
>> Name of the define is wrong, as it is not number of eSPIs. Actually, this is
>> maximum SPI (including eSPIs) number.
>
>
> Thank you for your review.
> Would it be okay if I rename this macro to VGIC_DEF_MAX_SPI?
Yes, I think this is better name.
--
WBR, Volodymyr
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v2 09/10] xen/arm: domain_build: adjust Dom0 IRQ handling to support eSPIs
2025-08-22 12:26 ` Volodymyr Babchuk
@ 2025-08-22 15:03 ` Leonid Komarianskyi
0 siblings, 0 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-22 15:03 UTC (permalink / raw)
To: Volodymyr Babchuk
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Hi Volodymyr,
On 22.08.25 15:26, Volodymyr Babchuk wrote:
>
> Hi Leonid,
>
> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>
>> Hi Volodymyr,
>>
>> On 21.08.25 19:46, Volodymyr Babchuk wrote:
>>>
>>>
>>>
>>> Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
>>>
>>>> The Dom0 configuration logic in create_dom0() has been updated
>>>> to account for extended SPIs when supported by the hardware and
>>>> enabled with CONFIG_GICV3_ESPI. These changes ensure the proper
>>>> calculation of the maximum number of SPIs and eSPIs available for Dom0.
>>>>
>>>> When eSPIs are supported by the hardware and CONFIG_GICV3_ESPI is
>>>> enabled, the maximum number of eSPI interrupts is calculated using
>>>> the ESPI_BASE_INTID offset (4096) and limited at 1024, with 32 IRQs
>>>> subtracted. To ensure compatibility with non-Dom0 domains, this
>>>> adjustment is applied by the toolstack during domain creation, while
>>>> for Dom0 it is handled directly during VGIC initialization. If eSPIs
>>>> are not supported, the calculation defaults to using the standard SPI
>>>> range, with a maximum value of 992 interrupt lines as it works now.
>>>>
>>>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>>>
>>>> ---
>>>> Changes in V2:
>>>> - no changes
>>>> ---
>>>> xen/arch/arm/domain_build.c | 10 ++++++++++
>>>> xen/arch/arm/include/asm/vgic.h | 11 +++++++++++
>>>> 2 files changed, 21 insertions(+)
>>>>
>>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>>> index d91a71acfd..fa5abf2dfb 100644
>>>> --- a/xen/arch/arm/domain_build.c
>>>> +++ b/xen/arch/arm/domain_build.c
>>>> @@ -2055,6 +2055,16 @@ void __init create_dom0(void)
>>>> /* The vGIC for DOM0 is exactly emulating the hardware GIC */
>>>> dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
>>>> dom0_cfg.arch.nr_spis = VGIC_DEF_NR_SPIS;
>>>> +#ifdef CONFIG_GICV3_ESPI
>>>> + /*
>>>> + * Check if the hardware supports extended SPIs (even if the appropriate config is set).
>>>> + * If not, the common SPI range will be used. Otherwise overwrite the nr_spis with the
>>>> + * maximum available INTID from eSPI range. In that case, the number of regular SPIs will
>>>> + * be adjusted to the maximum value during vGIC initialization.
>>>> + */
>>>> + if ( gic_number_espis() > 0 )
>>>> + dom0_cfg.arch.nr_spis = VGIC_DEF_NR_ESPIS;
>>>> +#endif
>>>> dom0_cfg.arch.tee_type = tee_get_type();
>>>> dom0_cfg.max_vcpus = dom0_max_vcpus();
>>>>
>>>> diff --git a/xen/arch/arm/include/asm/vgic.h b/xen/arch/arm/include/asm/vgic.h
>>>> index 9fa4523018..117b3aa92c 100644
>>>> --- a/xen/arch/arm/include/asm/vgic.h
>>>> +++ b/xen/arch/arm/include/asm/vgic.h
>>>> @@ -353,6 +353,17 @@ extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
>>>> /* Default number of vGIC SPIs. 32 are substracted to cover local IRQs. */
>>>> #define VGIC_DEF_NR_SPIS (min(gic_number_lines(), VGIC_MAX_IRQS) - 32)
>>>>
>>>> +#ifdef CONFIG_GICV3_ESPI
>>>> +/*
>>>> + * Returns the maximum eSPI INTID subtracted by 32. For non-Dom0 domains, the
>>>> + * toolstack applies the same adjustment to cover local IRQs. We will add back
>>>> + * this value during VGIC initialization. This ensures consistent handling for Dom0
>>>> + * and other domains. For the regular SPI range interrupts in this case, the maximum
>>>> + * value of VGIC_DEF_NR_SPIS will be used.
>>>> + */
>>>> +#define VGIC_DEF_NR_ESPIS (ESPI_BASE_INTID + min(gic_number_espis(), 1024U) - 32)
>>> Name of the define is wrong, as it is not number of eSPIs. Actually, this is
>>> maximum SPI (including eSPIs) number.
>>
>>
>> Thank you for your review.
>> Would it be okay if I rename this macro to VGIC_DEF_MAX_SPI?
>
> Yes, I think this is better name.
>
Thank you for your confirmation. I will rename this macro to
VGIC_DEF_MAX_SPI in V3.
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v2 09/10] xen/arm: domain_build: adjust Dom0 IRQ handling to support eSPIs
2025-08-07 12:33 ` [PATCH v2 09/10] xen/arm: domain_build: adjust Dom0 IRQ handling to support eSPIs Leonid Komarianskyi
2025-08-21 16:46 ` Volodymyr Babchuk
@ 2025-08-23 13:02 ` Oleksandr Tyshchenko
2025-08-24 18:47 ` Leonid Komarianskyi
1 sibling, 1 reply; 55+ messages in thread
From: Oleksandr Tyshchenko @ 2025-08-23 13:02 UTC (permalink / raw)
To: xen-devel, Leonid Komarianskyi
On 07.08.25 15:33, Leonid Komarianskyi wrote:
Hello Leonid
> The Dom0 configuration logic in create_dom0() has been updated
> to account for extended SPIs when supported by the hardware and
> enabled with CONFIG_GICV3_ESPI. These changes ensure the proper
> calculation of the maximum number of SPIs and eSPIs available for Dom0.
>
> When eSPIs are supported by the hardware and CONFIG_GICV3_ESPI is
> enabled, the maximum number of eSPI interrupts is calculated using
> the ESPI_BASE_INTID offset (4096) and limited at 1024, with 32 IRQs
> subtracted. To ensure compatibility with non-Dom0 domains, this
> adjustment is applied by the toolstack during domain creation, while
> for Dom0 it is handled directly during VGIC initialization. If eSPIs
> are not supported, the calculation defaults to using the standard SPI
> range, with a maximum value of 992 interrupt lines as it works now.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>
> ---
> Changes in V2:
> - no changes
> ---
> xen/arch/arm/domain_build.c | 10 ++++++++++
> xen/arch/arm/include/asm/vgic.h | 11 +++++++++++
> 2 files changed, 21 insertions(+)
>
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index d91a71acfd..fa5abf2dfb 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -2055,6 +2055,16 @@ void __init create_dom0(void)
> /* The vGIC for DOM0 is exactly emulating the hardware GIC */
> dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
> dom0_cfg.arch.nr_spis = VGIC_DEF_NR_SPIS;
> +#ifdef CONFIG_GICV3_ESPI
> + /*
> + * Check if the hardware supports extended SPIs (even if the appropriate config is set).
> + * If not, the common SPI range will be used. Otherwise overwrite the nr_spis with the
> + * maximum available INTID from eSPI range. In that case, the number of regular SPIs will
> + * be adjusted to the maximum value during vGIC initialization.
> + */
> + if ( gic_number_espis() > 0 )
> + dom0_cfg.arch.nr_spis = VGIC_DEF_NR_ESPIS;
> +#endif
I might miss something, but within the whole series, you seem not to
update "nr_spis" field for dom0less DomUs (in dom0less-build.c). Please
clarify the reason?
> dom0_cfg.arch.tee_type = tee_get_type();
> dom0_cfg.max_vcpus = dom0_max_vcpus();
>
[snip]
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v2 09/10] xen/arm: domain_build: adjust Dom0 IRQ handling to support eSPIs
2025-08-23 13:02 ` Oleksandr Tyshchenko
@ 2025-08-24 18:47 ` Leonid Komarianskyi
0 siblings, 0 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-24 18:47 UTC (permalink / raw)
To: Oleksandr Tyshchenko, xen-devel@lists.xenproject.org
Hello Oleksandr,
Thank you for your question.
On 23.08.25 16:02, Oleksandr Tyshchenko wrote:
> [You don't often get email from olekstysh@gmail.com. Learn why this is
> important at https://aka.ms/LearnAboutSenderIdentification ]
>
> On 07.08.25 15:33, Leonid Komarianskyi wrote:
>
> Hello Leonid
>
>> The Dom0 configuration logic in create_dom0() has been updated
>> to account for extended SPIs when supported by the hardware and
>> enabled with CONFIG_GICV3_ESPI. These changes ensure the proper
>> calculation of the maximum number of SPIs and eSPIs available for Dom0.
>>
>> When eSPIs are supported by the hardware and CONFIG_GICV3_ESPI is
>> enabled, the maximum number of eSPI interrupts is calculated using
>> the ESPI_BASE_INTID offset (4096) and limited at 1024, with 32 IRQs
>> subtracted. To ensure compatibility with non-Dom0 domains, this
>> adjustment is applied by the toolstack during domain creation, while
>> for Dom0 it is handled directly during VGIC initialization. If eSPIs
>> are not supported, the calculation defaults to using the standard SPI
>> range, with a maximum value of 992 interrupt lines as it works now.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V2:
>> - no changes
>> ---
>> xen/arch/arm/domain_build.c | 10 ++++++++++
>> xen/arch/arm/include/asm/vgic.h | 11 +++++++++++
>> 2 files changed, 21 insertions(+)
>>
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index d91a71acfd..fa5abf2dfb 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -2055,6 +2055,16 @@ void __init create_dom0(void)
>> /* The vGIC for DOM0 is exactly emulating the hardware GIC */
>> dom0_cfg.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
>> dom0_cfg.arch.nr_spis = VGIC_DEF_NR_SPIS;
>> +#ifdef CONFIG_GICV3_ESPI
>> + /*
>> + * Check if the hardware supports extended SPIs (even if the
>> appropriate config is set).
>> + * If not, the common SPI range will be used. Otherwise overwrite
>> the nr_spis with the
>> + * maximum available INTID from eSPI range. In that case, the
>> number of regular SPIs will
>> + * be adjusted to the maximum value during vGIC initialization.
>> + */
>> + if ( gic_number_espis() > 0 )
>> + dom0_cfg.arch.nr_spis = VGIC_DEF_NR_ESPIS;
>> +#endif
>
> I might miss something, but within the whole series, you seem not to
> update "nr_spis" field for dom0less DomUs (in dom0less-build.c). Please
> clarify the reason?
>
Yes, I missed this because I was mostly focused on the build (with Dom0)
that I currently have on the platform with eSPI. I rechecked the code,
and the dom0less code has the same behavior, so I believe it should work
similarly with the same changes in arch_create_domUs. I will update it
in V3.
>> dom0_cfg.arch.tee_type = tee_get_type();
>> dom0_cfg.max_vcpus = dom0_max_vcpus();
>>
>
> [snip]
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations
2025-08-07 12:33 ` [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations Leonid Komarianskyi
` (8 preceding siblings ...)
2025-08-07 12:33 ` [PATCH v2 09/10] xen/arm: domain_build: adjust Dom0 IRQ handling to support eSPIs Leonid Komarianskyi
@ 2025-08-21 16:00 ` Volodymyr Babchuk
2025-08-21 16:14 ` Julien Grall
10 siblings, 0 replies; 55+ messages in thread
From: Volodymyr Babchuk @ 2025-08-21 16:00 UTC (permalink / raw)
To: Leonid Komarianskyi
Cc: xen-devel@lists.xenproject.org, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel
Hi,
Leonid Komarianskyi <Leonid_Komarianskyi@epam.com> writes:
> Currently, many common functions perform the same operations to calculate
> GIC register addresses. This patch consolidates the similar code into
> a separate helper function to improve maintainability and reduce duplication.
> This refactoring also simplifies the implementation of eSPI support in future
> changes.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
>
> ---
> Changes in V2:
> - no changes
> ---
> xen/arch/arm/gic-v3.c | 99 ++++++++++++++++++++++------------
> xen/arch/arm/include/asm/irq.h | 1 +
> 2 files changed, 67 insertions(+), 33 deletions(-)
>
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index cd3e1acf79..8fd78aba44 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -445,17 +445,62 @@ static void gicv3_dump_state(const struct vcpu *v)
> }
> }
>
> +static void __iomem *get_addr_by_offset(struct irq_desc *irqd, u32 offset)
> +{
> + switch ( irqd->irq )
> + {
> + case 0 ... (NR_GIC_LOCAL_IRQS - 1):
> + switch ( offset )
> + {
> + case GICD_ISENABLER:
> + case GICD_ICENABLER:
> + case GICD_ISPENDR:
> + case GICD_ICPENDR:
> + case GICD_ISACTIVER:
> + case GICD_ICACTIVER:
> + return (GICD_RDIST_SGI_BASE + offset);
> + case GICD_ICFGR:
> + return (GICD_RDIST_SGI_BASE + GICR_ICFGR1);
> + case GICD_IPRIORITYR:
> + return (GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + irqd->irq);
> + default:
> + break;
> + }
> + case NR_GIC_LOCAL_IRQS ... SPI_MAX_INTID:
> + switch ( offset )
> + {
> + case GICD_ISENABLER:
> + case GICD_ICENABLER:
> + case GICD_ISPENDR:
> + case GICD_ICPENDR:
> + case GICD_ISACTIVER:
> + case GICD_ICACTIVER:
> + return (GICD + offset + (irqd->irq / 32) * 4);
> + case GICD_ICFGR:
> + return (GICD + GICD_ICFGR + (irqd->irq / 16) * 4);
> + case GICD_IROUTER:
> + return (GICD + GICD_IROUTER + irqd->irq * 8);
> + case GICD_IPRIORITYR:
> + return (GICD + GICD_IPRIORITYR + irqd->irq);
> + default:
> + break;
> + }
> + default:
> + break;
> + }
> +
> + /* Something went wrong, we shouldn't be able to reach here */
> + panic("Invalid offset 0x%x for IRQ#%d", offset, irqd->irq);
> +
> + return NULL;
> +}
> +
> static void gicv3_poke_irq(struct irq_desc *irqd, u32 offset, bool wait_for_rwp)
> {
> u32 mask = 1U << (irqd->irq % 32);
> - void __iomem *base;
> -
> - if ( irqd->irq < NR_GIC_LOCAL_IRQS )
> - base = GICD_RDIST_SGI_BASE;
> - else
> - base = GICD;
> + void __iomem *addr = get_addr_by_offset(irqd, offset);
>
> - writel_relaxed(mask, base + offset + (irqd->irq / 32) * 4);
> + writel_relaxed(mask, addr);
>
> if ( wait_for_rwp )
> gicv3_wait_for_rwp(irqd->irq);
> @@ -463,15 +508,9 @@ static void gicv3_poke_irq(struct irq_desc *irqd, u32 offset, bool wait_for_rwp)
>
> static bool gicv3_peek_irq(struct irq_desc *irqd, u32 offset)
> {
> - void __iomem *base;
> - unsigned int irq = irqd->irq;
> -
> - if ( irq >= NR_GIC_LOCAL_IRQS)
> - base = GICD + (irq / 32) * 4;
> - else
> - base = GICD_RDIST_SGI_BASE;
> + void __iomem *addr = get_addr_by_offset(irqd, offset);
>
> - return !!(readl(base + offset) & (1U << (irq % 32)));
> + return !!(readl(addr) & (1U << (irqd->irq % 32)));
> }
>
> static void gicv3_unmask_irq(struct irq_desc *irqd)
> @@ -558,30 +597,26 @@ static inline uint64_t gicv3_mpidr_to_affinity(int cpu)
> static void gicv3_set_irq_type(struct irq_desc *desc, unsigned int type)
> {
> uint32_t cfg, actual, edgebit;
> - void __iomem *base;
> - unsigned int irq = desc->irq;
> + void __iomem *addr;
>
> /* SGI's are always edge-triggered not need to call GICD_ICFGR0 */
> - ASSERT(irq >= NR_GIC_SGI);
> + ASSERT(desc->irq >= NR_GIC_SGI);
>
> spin_lock(&gicv3.lock);
>
> - if ( irq >= NR_GIC_LOCAL_IRQS)
> - base = GICD + GICD_ICFGR + (irq / 16) * 4;
> - else
> - base = GICD_RDIST_SGI_BASE + GICR_ICFGR1;
> + addr = get_addr_by_offset(desc, GICD_ICFGR);
>
> - cfg = readl_relaxed(base);
> + cfg = readl_relaxed(addr);
>
> - edgebit = 2u << (2 * (irq % 16));
> + edgebit = 2u << (2 * (desc->irq % 16));
> if ( type & IRQ_TYPE_LEVEL_MASK )
> cfg &= ~edgebit;
> else if ( type & IRQ_TYPE_EDGE_BOTH )
> cfg |= edgebit;
>
> - writel_relaxed(cfg, base);
> + writel_relaxed(cfg, addr);
>
> - actual = readl_relaxed(base);
> + actual = readl_relaxed(addr);
> if ( ( cfg & edgebit ) ^ ( actual & edgebit ) )
> {
> printk(XENLOG_WARNING "GICv3: WARNING: "
> @@ -600,15 +635,12 @@ static void gicv3_set_irq_type(struct irq_desc *desc, unsigned int type)
> static void gicv3_set_irq_priority(struct irq_desc *desc,
> unsigned int priority)
> {
> - unsigned int irq = desc->irq;
> + void __iomem *addr;
>
> spin_lock(&gicv3.lock);
>
> - /* Set priority */
> - if ( irq < NR_GIC_LOCAL_IRQS )
> - writeb_relaxed(priority, GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + irq);
> - else
> - writeb_relaxed(priority, GICD + GICD_IPRIORITYR + irq);
> + addr = get_addr_by_offset(desc, GICD_IPRIORITYR);
> + writeb_relaxed(priority, addr);
>
> spin_unlock(&gicv3.lock);
> }
> @@ -1273,6 +1305,7 @@ static void gicv3_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
> {
> unsigned int cpu;
> uint64_t affinity;
> + void __iomem *addr = get_addr_by_offset(desc, GICD_IROUTER);
>
> ASSERT(!cpumask_empty(mask));
>
> @@ -1284,7 +1317,7 @@ static void gicv3_irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
> affinity &= ~GICD_IROUTER_SPI_MODE_ANY;
>
> if ( desc->irq >= NR_GIC_LOCAL_IRQS )
> - writeq_relaxed_non_atomic(affinity, (GICD + GICD_IROUTER + desc->irq * 8));
> + writeq_relaxed_non_atomic(affinity, addr);
>
> spin_unlock(&gicv3.lock);
> }
> diff --git a/xen/arch/arm/include/asm/irq.h b/xen/arch/arm/include/asm/irq.h
> index fce7e42a33..5bc6475eb4 100644
> --- a/xen/arch/arm/include/asm/irq.h
> +++ b/xen/arch/arm/include/asm/irq.h
> @@ -29,6 +29,7 @@ struct arch_irq_desc {
> */
> #define NR_IRQS 1024
>
> +#define SPI_MAX_INTID 1019
> #define LPI_OFFSET 8192
>
> /* LPIs are always numbered starting at 8192, so 0 is a good invalid case. */
--
WBR, Volodymyr
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations
2025-08-07 12:33 ` [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations Leonid Komarianskyi
` (9 preceding siblings ...)
2025-08-21 16:00 ` [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations Volodymyr Babchuk
@ 2025-08-21 16:14 ` Julien Grall
2025-08-22 9:09 ` Leonid Komarianskyi
10 siblings, 1 reply; 55+ messages in thread
From: Julien Grall @ 2025-08-21 16:14 UTC (permalink / raw)
To: Leonid Komarianskyi, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk
Hi Leonid,
On 07/08/2025 13:33, Leonid Komarianskyi wrote:
> Currently, many common functions perform the same operations to calculate
> GIC register addresses. This patch consolidates the similar code into
> a separate helper function to improve maintainability and reduce duplication.
> This refactoring also simplifies the implementation of eSPI support in future
> changes.
>
> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>
> ---
> Changes in V2:
> - no changes
I am a bit surprised this is just saying no changes given the discussion
in v1. I feel you should have pinged on the v1 to close off the
discussion before sending v2.
While I understand your point and could accept we consolidate the code...
> ---
> xen/arch/arm/gic-v3.c | 99 ++++++++++++++++++++++------------
> xen/arch/arm/include/asm/irq.h | 1 +
> 2 files changed, 67 insertions(+), 33 deletions(-)
>
> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
> index cd3e1acf79..8fd78aba44 100644
> --- a/xen/arch/arm/gic-v3.c
> +++ b/xen/arch/arm/gic-v3.c
> @@ -445,17 +445,62 @@ static void gicv3_dump_state(const struct vcpu *v)
> }
> }
>
> +static void __iomem *get_addr_by_offset(struct irq_desc *irqd, u32 offset)
> +{
> + switch ( irqd->irq )
> + {
> + case 0 ... (NR_GIC_LOCAL_IRQS - 1):
> + switch ( offset )
> + {
> + case GICD_ISENABLER:
> + case GICD_ICENABLER:
> + case GICD_ISPENDR:
> + case GICD_ICPENDR:
> + case GICD_ISACTIVER:
> + case GICD_ICACTIVER:
> + return (GICD_RDIST_SGI_BASE + offset);
> + case GICD_ICFGR:
> + return (GICD_RDIST_SGI_BASE + GICR_ICFGR1);
> + case GICD_IPRIORITYR:
> + return (GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + irqd->irq);
> + default:
> + break;
> + }
> + case NR_GIC_LOCAL_IRQS ... SPI_MAX_INTID:
> + switch ( offset )
> + {
> + case GICD_ISENABLER:
> + case GICD_ICENABLER:
> + case GICD_ISPENDR:
> + case GICD_ICPENDR:
> + case GICD_ISACTIVER:
> + case GICD_ICACTIVER:
> + return (GICD + offset + (irqd->irq / 32) * 4);
> + case GICD_ICFGR:
> + return (GICD + GICD_ICFGR + (irqd->irq / 16) * 4);
> + case GICD_IROUTER:
> + return (GICD + GICD_IROUTER + irqd->irq * 8);
> + case GICD_IPRIORITYR:
> + return (GICD + GICD_IPRIORITYR + irqd->irq);
> + default:
> + break;
> + }
> + default:
> + break;
> + }
> +
> + /* Something went wrong, we shouldn't be able to reach here */
> + panic("Invalid offset 0x%x for IRQ#%d", offset, irqd->irq);
... I still quite concerned about using panic here. We need to try to
handle the error more gracefully in production.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations
2025-08-21 16:14 ` Julien Grall
@ 2025-08-22 9:09 ` Leonid Komarianskyi
2025-08-22 9:38 ` Julien Grall
0 siblings, 1 reply; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-22 9:09 UTC (permalink / raw)
To: Julien Grall, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk
Hi Julien,
Thank you for your close review.
On 21.08.25 19:14, Julien Grall wrote:
> Hi Leonid,
>
> On 07/08/2025 13:33, Leonid Komarianskyi wrote:
>> Currently, many common functions perform the same operations to calculate
>> GIC register addresses. This patch consolidates the similar code into
>> a separate helper function to improve maintainability and reduce
>> duplication.
>> This refactoring also simplifies the implementation of eSPI support in
>> future
>> changes.
>>
>> Signed-off-by: Leonid Komarianskyi <leonid_komarianskyi@epam.com>
>>
>> ---
>> Changes in V2:
>> - no changes
>
> I am a bit surprised this is just saying no changes given the discussion
> in v1. I feel you should have pinged on the v1 to close off the
> discussion before sending v2.
>
Sorry, my bad. I should have reached out to you and clarified the
unfinished discussion in V1. I just wanted to publish the changes in V2
that were agreed upon in V1. But yes, I understand - it was a bad idea
to push V2 without completing the discussion in V1. I apologize for that.
> While I understand your point and could accept we consolidate the code...
>
>
Thank you for your understanding and acceptance :) It really simplifies
the changes in the next patches.
>> ---
>> xen/arch/arm/gic-v3.c | 99 ++++++++++++++++++++++------------
>> xen/arch/arm/include/asm/irq.h | 1 +
>> 2 files changed, 67 insertions(+), 33 deletions(-)
>>
>> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
>> index cd3e1acf79..8fd78aba44 100644
>> --- a/xen/arch/arm/gic-v3.c
>> +++ b/xen/arch/arm/gic-v3.c
>> @@ -445,17 +445,62 @@ static void gicv3_dump_state(const struct vcpu *v)
>> }
>> }
>> +static void __iomem *get_addr_by_offset(struct irq_desc *irqd, u32
>> offset)
>> +{
>> + switch ( irqd->irq )
>> + {
>> + case 0 ... (NR_GIC_LOCAL_IRQS - 1):
>> + switch ( offset )
>> + {
>> + case GICD_ISENABLER:
>> + case GICD_ICENABLER:
>> + case GICD_ISPENDR:
>> + case GICD_ICPENDR:
>> + case GICD_ISACTIVER:
>> + case GICD_ICACTIVER:
>> + return (GICD_RDIST_SGI_BASE + offset);
>> + case GICD_ICFGR:
>> + return (GICD_RDIST_SGI_BASE + GICR_ICFGR1);
>> + case GICD_IPRIORITYR:
>> + return (GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + irqd->irq);
>> + default:
>> + break;
>> + }
>> + case NR_GIC_LOCAL_IRQS ... SPI_MAX_INTID:
>> + switch ( offset )
>> + {
>> + case GICD_ISENABLER:
>> + case GICD_ICENABLER:
>> + case GICD_ISPENDR:
>> + case GICD_ICPENDR:
>> + case GICD_ISACTIVER:
>> + case GICD_ICACTIVER:
>> + return (GICD + offset + (irqd->irq / 32) * 4);
>> + case GICD_ICFGR:
>> + return (GICD + GICD_ICFGR + (irqd->irq / 16) * 4);
>> + case GICD_IROUTER:
>> + return (GICD + GICD_IROUTER + irqd->irq * 8);
>> + case GICD_IPRIORITYR:
>> + return (GICD + GICD_IPRIORITYR + irqd->irq);
>> + default:
>> + break;
>> + }
>> + default:
>> + break;
>> + }
>> +
>> + /* Something went wrong, we shouldn't be able to reach here */
> > + panic("Invalid offset 0x%x for IRQ#%d", offset, irqd->irq);
>
> ... I still quite concerned about using panic here. We need to try to
> handle the error more gracefully in production.
>
> Cheers,
>
I was thinking about this again, and maybe it would be better to change
the panic into simply printing an error using printk(XENLOG_G_ERR ...)
and adding proper checks to ensure the return value is not NULL in the
callers.
Also, in the case of gicv3_peek_irq, which must return a boolean value
(due to the generic API for gicv3_read_pending_state), we could return
false with an additional warning message that we are unable to read the
actual value due to incorrect parameters; therefore, we return false.
What do you think about this approach?
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations
2025-08-22 9:09 ` Leonid Komarianskyi
@ 2025-08-22 9:38 ` Julien Grall
2025-08-22 10:09 ` Leonid Komarianskyi
0 siblings, 1 reply; 55+ messages in thread
From: Julien Grall @ 2025-08-22 9:38 UTC (permalink / raw)
To: Leonid Komarianskyi, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk
Hi Leonid,
On 22/08/2025 10:09, Leonid Komarianskyi wrote:
> Thank you for your close review.>>> ---
>>> xen/arch/arm/gic-v3.c | 99 ++++++++++++++++++++++------------
>>> xen/arch/arm/include/asm/irq.h | 1 +
>>> 2 files changed, 67 insertions(+), 33 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
>>> index cd3e1acf79..8fd78aba44 100644
>>> --- a/xen/arch/arm/gic-v3.c
>>> +++ b/xen/arch/arm/gic-v3.c
>>> @@ -445,17 +445,62 @@ static void gicv3_dump_state(const struct vcpu *v)
>>> }
>>> }
>>> +static void __iomem *get_addr_by_offset(struct irq_desc *irqd, u32
>>> offset)
>>> +{
>>> + switch ( irqd->irq )
>>> + {
>>> + case 0 ... (NR_GIC_LOCAL_IRQS - 1):
>>> + switch ( offset )
>>> + {
>>> + case GICD_ISENABLER:
>>> + case GICD_ICENABLER:
>>> + case GICD_ISPENDR:
>>> + case GICD_ICPENDR:
>>> + case GICD_ISACTIVER:
>>> + case GICD_ICACTIVER:
>>> + return (GICD_RDIST_SGI_BASE + offset);
>>> + case GICD_ICFGR:
>>> + return (GICD_RDIST_SGI_BASE + GICR_ICFGR1);
>>> + case GICD_IPRIORITYR:
>>> + return (GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + irqd->irq);
>>> + default:
>>> + break;
>>> + }
>>> + case NR_GIC_LOCAL_IRQS ... SPI_MAX_INTID:
>>> + switch ( offset )
>>> + {
>>> + case GICD_ISENABLER:
>>> + case GICD_ICENABLER:
>>> + case GICD_ISPENDR:
>>> + case GICD_ICPENDR:
>>> + case GICD_ISACTIVER:
>>> + case GICD_ICACTIVER:
>>> + return (GICD + offset + (irqd->irq / 32) * 4);
>>> + case GICD_ICFGR:
>>> + return (GICD + GICD_ICFGR + (irqd->irq / 16) * 4);
>>> + case GICD_IROUTER:
>>> + return (GICD + GICD_IROUTER + irqd->irq * 8);
>>> + case GICD_IPRIORITYR:
>>> + return (GICD + GICD_IPRIORITYR + irqd->irq);
>>> + default:
>>> + break;
>>> + }
>>> + default:
>>> + break;
>>> + }
>>> +
>>> + /* Something went wrong, we shouldn't be able to reach here */
>> > + panic("Invalid offset 0x%x for IRQ#%d", offset, irqd->irq);
>>
>> ... I still quite concerned about using panic here. We need to try to
>> handle the error more gracefully in production.
>>
>> Cheers,
>>
>
> I was thinking about this again, and maybe it would be better to change
> the panic into simply printing an error using printk(XENLOG_G_ERR ...)
> and adding proper checks to ensure the return value is not NULL in the
> callers.
Given the error is not meant to happen, after the printk() I would add
an ASSERT_UNREACHABLE() so we can catch issue in DEBUG build more easily.
> Also, in the case of gicv3_peek_irq, which must return a boolean value
> (due to the generic API for gicv3_read_pending_state), we could return
> false with an additional warning message that we are unable to read the
> actual value due to incorrect parameters; therefore, we return false.
> What do you think about this approach?
It makes sense to read false as the interrupt technically doesn't exist.
But I don't think we should add an extra warning. The one in
get_addr_by_offset() should be sufficient.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 55+ messages in thread* Re: [PATCH v2 01/10] xen/arm: gicv3: refactor obtaining GIC addresses for common operations
2025-08-22 9:38 ` Julien Grall
@ 2025-08-22 10:09 ` Leonid Komarianskyi
0 siblings, 0 replies; 55+ messages in thread
From: Leonid Komarianskyi @ 2025-08-22 10:09 UTC (permalink / raw)
To: Julien Grall, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk
Hi Julien,
On 22.08.25 12:38, Julien Grall wrote:
> Hi Leonid,
>
> On 22/08/2025 10:09, Leonid Komarianskyi wrote:
> > Thank you for your close review.>>> ---
>>>> xen/arch/arm/gic-v3.c | 99 +++++++++++++++++++++
>>>> +------------
>>>> xen/arch/arm/include/asm/irq.h | 1 +
>>>> 2 files changed, 67 insertions(+), 33 deletions(-)
>>>>
>>>> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
>>>> index cd3e1acf79..8fd78aba44 100644
>>>> --- a/xen/arch/arm/gic-v3.c
>>>> +++ b/xen/arch/arm/gic-v3.c
>>>> @@ -445,17 +445,62 @@ static void gicv3_dump_state(const struct vcpu
>>>> *v)
>>>> }
>>>> }
>>>> +static void __iomem *get_addr_by_offset(struct irq_desc *irqd, u32
>>>> offset)
>>>> +{
>>>> + switch ( irqd->irq )
>>>> + {
>>>> + case 0 ... (NR_GIC_LOCAL_IRQS - 1):
>>>> + switch ( offset )
>>>> + {
>>>> + case GICD_ISENABLER:
>>>> + case GICD_ICENABLER:
>>>> + case GICD_ISPENDR:
>>>> + case GICD_ICPENDR:
>>>> + case GICD_ISACTIVER:
>>>> + case GICD_ICACTIVER:
>>>> + return (GICD_RDIST_SGI_BASE + offset);
>>>> + case GICD_ICFGR:
>>>> + return (GICD_RDIST_SGI_BASE + GICR_ICFGR1);
>>>> + case GICD_IPRIORITYR:
>>>> + return (GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + irqd-
>>>> >irq);
>>>> + default:
>>>> + break;
>>>> + }
>>>> + case NR_GIC_LOCAL_IRQS ... SPI_MAX_INTID:
>>>> + switch ( offset )
>>>> + {
>>>> + case GICD_ISENABLER:
>>>> + case GICD_ICENABLER:
>>>> + case GICD_ISPENDR:
>>>> + case GICD_ICPENDR:
>>>> + case GICD_ISACTIVER:
>>>> + case GICD_ICACTIVER:
>>>> + return (GICD + offset + (irqd->irq / 32) * 4);
>>>> + case GICD_ICFGR:
>>>> + return (GICD + GICD_ICFGR + (irqd->irq / 16) * 4);
>>>> + case GICD_IROUTER:
>>>> + return (GICD + GICD_IROUTER + irqd->irq * 8);
>>>> + case GICD_IPRIORITYR:
>>>> + return (GICD + GICD_IPRIORITYR + irqd->irq);
>>>> + default:
>>>> + break;
>>>> + }
>>>> + default:
>>>> + break;
>>>> + }
>>>> +
>>>> + /* Something went wrong, we shouldn't be able to reach here */
>>> > + panic("Invalid offset 0x%x for IRQ#%d", offset, irqd->irq);
>>>
>>> ... I still quite concerned about using panic here. We need to try to
>>> handle the error more gracefully in production.
>>>
>>> Cheers,
>>>
>>
>> I was thinking about this again, and maybe it would be better to change
>> the panic into simply printing an error using printk(XENLOG_G_ERR ...)
>> and adding proper checks to ensure the return value is not NULL in the
>> callers.
>
> Given the error is not meant to happen, after the printk() I would add
> an ASSERT_UNREACHABLE() so we can catch issue in DEBUG build more easily.
>
Okay, so I will change the panic to printk + ASSERT_UNREACHABLE() in V3.
Thank you.
>> Also, in the case of gicv3_peek_irq, which must return a boolean value
>> (due to the generic API for gicv3_read_pending_state), we could return
>> false with an additional warning message that we are unable to read the
>> actual value due to incorrect parameters; therefore, we return false.
>> What do you think about this approach?
>
> It makes sense to read false as the interrupt technically doesn't exist.
> But I don't think we should add an extra warning. The one in
> get_addr_by_offset() should be sufficient.
>
> Cheers,
>
Understood, thank you again.
Best regards,
Leonid
^ permalink raw reply [flat|nested] 55+ messages in thread