[RFC PATCH] xen: Introduce extra IRQ count domain creation parameter

[RFC PATCH] xen: Introduce extra IRQ count domain creation parameter

[Teddy Astie]

When doing PCI Passthrough with high-IRQ devices (e.g NVMe drives),
the default limit may be unefficient as not all domains requires
more IRQs.

Introduce a new parameter to allow the toolstack to tune the IRQ
count if more is required.

Signed-off-by: Teddy Astie <teddy.astie@vates.tech>
---
0 extra_irqs is meaningful, though I am not sure how to expose this
special case.

This of course wants libxl support next.
---
 xen/common/domain.c         | 8 +++++---
 xen/include/public/domctl.h | 3 +++
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/xen/common/domain.c b/xen/common/domain.c
index abf1969e60..5c628962fc 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -912,10 +912,12 @@ struct domain *domain_create(domid_t domid,
 
 #ifdef CONFIG_HAS_PIRQ
     if ( !is_hardware_domain(d) )
-        d->nr_pirqs = nr_static_irqs + extra_domU_irqs;
+        d->nr_pirqs = nr_static_irqs + config->extra_irqs ?: extra_domU_irqs;
     else
-        d->nr_pirqs = extra_hwdom_irqs ? nr_static_irqs + extra_hwdom_irqs
-                                       : arch_hwdom_irqs(d);
+    {
+        unsigned int extra_irqs = config->extra_irqs ?: extra_hwdom_irqs;
+        d->nr_pirqs = extra_irqs ? nr_static_irqs + extra_irqs : arch_hwdom_irqs(d);
+    }
     d->nr_pirqs = min(d->nr_pirqs, nr_irqs);
 
     radix_tree_init(&d->pirq_tree);
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 5b2063eed9..e4bb366c78 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -121,6 +121,9 @@ struct xen_domctl_createdomain {
     /* CPU pool to use; specify 0 or a specific existing pool */
     uint32_t cpupool_id;
 
+    /* Additional IRQ for this guest. 0 to use Xen default value. */
+    uint32_t extra_irqs;
+
     struct xen_arch_domainconfig arch;
 };
 
-- 
2.49.0



Teddy Astie | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech

Re: [RFC PATCH] xen: Introduce extra IRQ count domain creation parameter

[Jan Beulich]

On 16.05.2025 15:50, Teddy Astie wrote:
> When doing PCI Passthrough with high-IRQ devices (e.g NVMe drives),
> the default limit may be unefficient as not all domains requires
> more IRQs.
> 
> Introduce a new parameter to allow the toolstack to tune the IRQ
> count if more is required.
> 
> Signed-off-by: Teddy Astie <teddy.astie@vates.tech>

Sure, why not (suitably fleshed out, as e.g. you say ...

> ---
> 0 extra_irqs is meaningful, though I am not sure how to expose this
> special case.
> 
> This of course wants libxl support next.

... here).

> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -912,10 +912,12 @@ struct domain *domain_create(domid_t domid,
>  
>  #ifdef CONFIG_HAS_PIRQ
>      if ( !is_hardware_domain(d) )
> -        d->nr_pirqs = nr_static_irqs + extra_domU_irqs;
> +        d->nr_pirqs = nr_static_irqs + config->extra_irqs ?: extra_domU_irqs;

I think you're missing parentheses here around the ?: expression.

Jan

Re: [RFC PATCH] xen: Introduce extra IRQ count domain creation parameter

[Roger Pau Monné]

On Fri, May 16, 2025 at 01:50:25PM +0000, Teddy Astie wrote:
> When doing PCI Passthrough with high-IRQ devices (e.g NVMe drives),
> the default limit may be unefficient as not all domains requires
> more IRQs.
> 
> Introduce a new parameter to allow the toolstack to tune the IRQ
> count if more is required.
> 
> Signed-off-by: Teddy Astie <teddy.astie@vates.tech>
> ---
> 0 extra_irqs is meaningful, though I am not sure how to expose this
> special case.

You could introduce a new CDF flag to signal the contents of
extra_irqs is valid, or maybe use the top bit of the `extra_irqs`
field to signal the value is set?

> This of course wants libxl support next.

It would be nice if this could come together in a patch series.

> ---
>  xen/common/domain.c         | 8 +++++---
>  xen/include/public/domctl.h | 3 +++
>  2 files changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/common/domain.c b/xen/common/domain.c
> index abf1969e60..5c628962fc 100644
> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -912,10 +912,12 @@ struct domain *domain_create(domid_t domid,
>  
>  #ifdef CONFIG_HAS_PIRQ
>      if ( !is_hardware_domain(d) )
> -        d->nr_pirqs = nr_static_irqs + extra_domU_irqs;
> +        d->nr_pirqs = nr_static_irqs + config->extra_irqs ?: extra_domU_irqs;
>      else
> -        d->nr_pirqs = extra_hwdom_irqs ? nr_static_irqs + extra_hwdom_irqs
> -                                       : arch_hwdom_irqs(d);
> +    {
> +        unsigned int extra_irqs = config->extra_irqs ?: extra_hwdom_irqs;

Newline.

> +        d->nr_pirqs = extra_irqs ? nr_static_irqs + extra_irqs : arch_hwdom_irqs(d);

I think the above line is > 80 characters?

> +    }
>      d->nr_pirqs = min(d->nr_pirqs, nr_irqs);
>  
>      radix_tree_init(&d->pirq_tree);
> diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
> index 5b2063eed9..e4bb366c78 100644
> --- a/xen/include/public/domctl.h
> +++ b/xen/include/public/domctl.h
> @@ -121,6 +121,9 @@ struct xen_domctl_createdomain {
>      /* CPU pool to use; specify 0 or a specific existing pool */
>      uint32_t cpupool_id;
>  
> +    /* Additional IRQ for this guest. 0 to use Xen default value. */
> +    uint32_t extra_irqs;

I think you need a domctl version bump, as the last one was done for
4.19.

Regards, Roger.