* [PATCH v4 1/8] arm/time: Use static irqaction
2025-11-12 10:51 [PATCH v4 0/8] Implement CPU hotplug on Arm Mykyta Poturai
@ 2025-11-12 10:51 ` Mykyta Poturai
2025-11-12 10:51 ` [PATCH v4 2/8] arm/gic: " Mykyta Poturai
` (6 subsequent siblings)
7 siblings, 0 replies; 20+ messages in thread
From: Mykyta Poturai @ 2025-11-12 10:51 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Mykyta Poturai, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk, Mykola Kvach,
Julien Grall
When stopping a core deinit_timer_interrupt is called in non-alloc
context, which causes xfree in release_irq to fail an assert.
To fix this, switch to a statically allocated irqaction that does not
need to be freed in release_irq.
Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
Reviewed-by: Mykola Kvach <mykola_kvach@epam.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
v3->v4:
* make irqactions static
* collect RBs
v2->v3:
* no changes
v1->v2:
* use percpu actions
---
xen/arch/arm/time.c | 21 +++++++++++++++++----
1 file changed, 17 insertions(+), 4 deletions(-)
diff --git a/xen/arch/arm/time.c b/xen/arch/arm/time.c
index e74d30d258..3710eab109 100644
--- a/xen/arch/arm/time.c
+++ b/xen/arch/arm/time.c
@@ -303,9 +303,15 @@ static void check_timer_irq_cfg(unsigned int irq, const char *which)
"WARNING: %s-timer IRQ%u is not level triggered.\n", which, irq);
}
+static DEFINE_PER_CPU_READ_MOSTLY(struct irqaction, irq_hyp);
+static DEFINE_PER_CPU_READ_MOSTLY(struct irqaction, irq_virt);
+
/* Set up the timer interrupt on this CPU */
void init_timer_interrupt(void)
{
+ struct irqaction *hyp_action = &this_cpu(irq_hyp);
+ struct irqaction *virt_action = &this_cpu(irq_virt);
+
/* Sensible defaults */
WRITE_SYSREG64(0, CNTVOFF_EL2); /* No VM-specific offset */
/* Do not let the VMs program the physical timer, only read the physical counter */
@@ -314,10 +320,17 @@ void init_timer_interrupt(void)
WRITE_SYSREG(0, CNTHP_CTL_EL2); /* Hypervisor's timer disabled */
isb();
- request_irq(timer_irq[TIMER_HYP_PPI], 0, htimer_interrupt,
- "hyptimer", NULL);
- request_irq(timer_irq[TIMER_VIRT_PPI], 0, vtimer_interrupt,
- "virtimer", NULL);
+ hyp_action->name = "hyptimer";
+ hyp_action->handler = htimer_interrupt;
+ hyp_action->dev_id = NULL;
+ hyp_action->free_on_release = 0;
+ setup_irq(timer_irq[TIMER_HYP_PPI], 0, hyp_action);
+
+ virt_action->name = "virtimer";
+ virt_action->handler = vtimer_interrupt;
+ virt_action->dev_id = NULL;
+ virt_action->free_on_release = 0;
+ setup_irq(timer_irq[TIMER_VIRT_PPI], 0, virt_action);
check_timer_irq_cfg(timer_irq[TIMER_HYP_PPI], "hypervisor");
check_timer_irq_cfg(timer_irq[TIMER_VIRT_PPI], "virtual");
--
2.51.2
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH v4 2/8] arm/gic: Use static irqaction
2025-11-12 10:51 [PATCH v4 0/8] Implement CPU hotplug on Arm Mykyta Poturai
2025-11-12 10:51 ` [PATCH v4 1/8] arm/time: Use static irqaction Mykyta Poturai
@ 2025-11-12 10:51 ` Mykyta Poturai
2025-11-12 10:51 ` [PATCH v4 3/8] arm/irq: Keep track of irq affinities Mykyta Poturai
` (5 subsequent siblings)
7 siblings, 0 replies; 20+ messages in thread
From: Mykyta Poturai @ 2025-11-12 10:51 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Mykyta Poturai, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk, Mykola Kvach,
Julien Grall
When stopping a core cpu_gic_callback is called in non-alloc
context, which causes xfree in release_irq to fail an assert.
To fix this, switch to a statically allocated irqaction that does not
need to be freed in release_irq.
Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
Reviewed-by: Mykola Kvach <mykola_kvach@epam.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
v3->v4:
* make irqactions static
* collect RBs
v2->v3:
* no changes
v1->v2:
* use percpu actions
---
xen/arch/arm/gic.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c
index 260ee64cca..ee75258fc3 100644
--- a/xen/arch/arm/gic.c
+++ b/xen/arch/arm/gic.c
@@ -386,10 +386,17 @@ void gic_dump_info(struct vcpu *v)
gic_hw_ops->dump_state(v);
}
+static DEFINE_PER_CPU_READ_MOSTLY(struct irqaction, irq_maintenance);
+
void init_maintenance_interrupt(void)
{
- request_irq(gic_hw_ops->info->maintenance_irq, 0, maintenance_interrupt,
- "irq-maintenance", NULL);
+ struct irqaction *maintenance = &this_cpu(irq_maintenance);
+
+ maintenance->name = "irq-maintenance";
+ maintenance->handler = maintenance_interrupt;
+ maintenance->dev_id = NULL;
+ maintenance->free_on_release = 0;
+ setup_irq(gic_hw_ops->info->maintenance_irq, 0, maintenance);
}
int gic_make_hwdom_dt_node(const struct domain *d,
--
2.51.2
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH v4 3/8] arm/irq: Keep track of irq affinities
2025-11-12 10:51 [PATCH v4 0/8] Implement CPU hotplug on Arm Mykyta Poturai
2025-11-12 10:51 ` [PATCH v4 1/8] arm/time: Use static irqaction Mykyta Poturai
2025-11-12 10:51 ` [PATCH v4 2/8] arm/gic: " Mykyta Poturai
@ 2025-11-12 10:51 ` Mykyta Poturai
2025-11-16 10:24 ` Julien Grall
2025-11-12 10:51 ` [PATCH v4 4/8] arm/irq: Migrate IRQs from dyings CPUs Mykyta Poturai
` (4 subsequent siblings)
7 siblings, 1 reply; 20+ messages in thread
From: Mykyta Poturai @ 2025-11-12 10:51 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Mykyta Poturai, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk
Currently on Arm the desc->affinity mask of an irq is never updated,
which makes it hard to know the actual affinity of an interrupt.
Fix this by updating the field in irq_set_affinity.
Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
v3->v4:
* patch introduced
---
xen/arch/arm/irq.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 73e58a5108..28b40331f7 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -219,7 +219,10 @@ static inline struct domain *irq_get_domain(struct irq_desc *desc)
void irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
{
if ( desc != NULL )
+ {
+ cpumask_copy(desc->affinity, mask);
desc->handler->set_affinity(desc, mask);
+ }
}
int request_irq(unsigned int irq, unsigned int irqflags,
--
2.51.2
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [PATCH v4 3/8] arm/irq: Keep track of irq affinities
2025-11-12 10:51 ` [PATCH v4 3/8] arm/irq: Keep track of irq affinities Mykyta Poturai
@ 2025-11-16 10:24 ` Julien Grall
0 siblings, 0 replies; 20+ messages in thread
From: Julien Grall @ 2025-11-16 10:24 UTC (permalink / raw)
To: Mykyta Poturai, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk
Hi Mykyta,
On 12/11/2025 10:51, Mykyta Poturai wrote:
> Currently on Arm the desc->affinity mask of an irq is never updated,
> which makes it hard to know the actual affinity of an interrupt.
>
> Fix this by updating the field in irq_set_affinity.
>
> Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
>
> v3->v4:
> * patch introduced
> ---
> xen/arch/arm/irq.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 73e58a5108..28b40331f7 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -219,7 +219,10 @@ static inline struct domain *irq_get_domain(struct irq_desc *desc)
> void irq_set_affinity(struct irq_desc *desc, const cpumask_t *mask)
> {
> if ( desc != NULL )
> + {
> + cpumask_copy(desc->affinity, mask);
irq_set_affinity() doesn't always seems to be called with ``desc->lock``
taken (see vgic_migrate_irq(), arch_move_irqs()). So I am not sure this
is safe. You probably need to update the callers as well to always take
the lock. irq_set_affinity() probably want to gain an
ASSERT(spin_is_locked()) with some documentation on top of the function.
> desc->handler->set_affinity(desc, mask);
> + }
NIT: As there are multiple lines in the ``if``, I would consider
reworking the logic so we return early when desc is NULL.
> }
>
> int request_irq(unsigned int irq, unsigned int irqflags,
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v4 4/8] arm/irq: Migrate IRQs from dyings CPUs
2025-11-12 10:51 [PATCH v4 0/8] Implement CPU hotplug on Arm Mykyta Poturai
` (2 preceding siblings ...)
2025-11-12 10:51 ` [PATCH v4 3/8] arm/irq: Keep track of irq affinities Mykyta Poturai
@ 2025-11-12 10:51 ` Mykyta Poturai
2025-11-16 11:24 ` Julien Grall
2025-11-12 10:51 ` [PATCH v4 6/8] arm/sysctl: Implement cpu hotplug ops Mykyta Poturai
` (3 subsequent siblings)
7 siblings, 1 reply; 20+ messages in thread
From: Mykyta Poturai @ 2025-11-12 10:51 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Mykyta Poturai, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk
Move IRQs from dying CPU to the online ones.
Guest-bound IRQs are already handled by scheduler in the process of
moving vCPUs to active pCPUs, so we only need to handle IRQs used by Xen
itself.
If IRQ is to be migrated, it's affinity is set to a mask of all online
CPUs. With current GIC implementation, this means they are routed to a
random online CPU. This may cause extra moves if multiple cores are
disabled in sequence, but should prevent all interrupts from piling up
on CPU0 in case of repeated up-down cycles on different cores.
IRQs from CPU 0 are never migrated, as dying CPU 0 means we are either
shutting down compeletely or entering system suspend.
Considering that all Xen-used IRQs are currently allocated during init
on CPU 0, and setup_irq uses smp_processor_id for the initial affinity.
This change is not strictly required for correct operation for now, but
it should future-proof cpu hotplug and system suspend support in case
some kind if IRQ balancing is implemented later.
Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
v3->v4:
* patch introduced
---
xen/arch/arm/include/asm/irq.h | 2 ++
xen/arch/arm/irq.c | 39 ++++++++++++++++++++++++++++++++++
xen/arch/arm/smpboot.c | 2 ++
3 files changed, 43 insertions(+)
diff --git a/xen/arch/arm/include/asm/irq.h b/xen/arch/arm/include/asm/irq.h
index 09788dbfeb..6e6e27bb80 100644
--- a/xen/arch/arm/include/asm/irq.h
+++ b/xen/arch/arm/include/asm/irq.h
@@ -126,6 +126,8 @@ bool irq_type_set_by_domain(const struct domain *d);
void irq_end_none(struct irq_desc *irq);
#define irq_end_none irq_end_none
+void evacuate_irqs(unsigned int from);
+
#endif /* _ASM_HW_IRQ_H */
/*
* Local variables:
diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
index 28b40331f7..b383d71930 100644
--- a/xen/arch/arm/irq.c
+++ b/xen/arch/arm/irq.c
@@ -158,6 +158,45 @@ static int init_local_irq_data(unsigned int cpu)
return 0;
}
+static void evacuate_irq(int irq, unsigned int from)
+{
+ struct irq_desc *desc = irq_to_desc(irq);
+ unsigned long flags;
+
+ /* Don't move irqs from CPU 0 as it is always last to be disabled */
+ if ( from == 0 )
+ return;
+
+ ASSERT(!cpumask_empty(&cpu_online_map));
+ ASSERT(!cpumask_test_cpu(from, &cpu_online_map));
+
+ spin_lock_irqsave(&desc->lock, flags);
+ if ( likely(!desc->action) )
+ goto out;
+
+ if ( likely(test_bit(_IRQ_GUEST, &desc->status) ||
+ test_bit(_IRQ_MOVE_PENDING, &desc->status)) )
+ goto out;
+
+ if ( cpumask_test_cpu(from, desc->affinity) )
+ irq_set_affinity(desc, &cpu_online_map);
+
+out:
+ spin_unlock_irqrestore(&desc->lock, flags);
+ return;
+}
+
+void evacuate_irqs(unsigned int from)
+{
+ int irq;
+
+ for ( irq = NR_LOCAL_IRQS; irq < NR_IRQS; irq++ )
+ evacuate_irq(irq, from);
+
+ for ( irq = ESPI_BASE_INTID; irq < ESPI_MAX_INTID; irq++ )
+ evacuate_irq(irq, from);
+}
+
static int cpu_callback(struct notifier_block *nfb, unsigned long action,
void *hcpu)
{
diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index 7f3cfa812e..46b24783dd 100644
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -425,6 +425,8 @@ void __cpu_disable(void)
smp_mb();
+ evacuate_irqs(cpu);
+
/* Return to caller; eventually the IPI mechanism will unwind and the
* scheduler will drop to the idle loop, which will call stop_cpu(). */
}
--
2.51.2
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [PATCH v4 4/8] arm/irq: Migrate IRQs from dyings CPUs
2025-11-12 10:51 ` [PATCH v4 4/8] arm/irq: Migrate IRQs from dyings CPUs Mykyta Poturai
@ 2025-11-16 11:24 ` Julien Grall
2025-12-12 10:01 ` Mykyta Poturai
0 siblings, 1 reply; 20+ messages in thread
From: Julien Grall @ 2025-11-16 11:24 UTC (permalink / raw)
To: Mykyta Poturai, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk
Hi,
On 12/11/2025 10:51, Mykyta Poturai wrote:
> Move IRQs from dying CPU to the online ones.
> Guest-bound IRQs are already handled by scheduler in the process of
> moving vCPUs to active pCPUs, so we only need to handle IRQs used by Xen
> itself.
>
> If IRQ is to be migrated, it's affinity is set to a mask of all online
> CPUs. With current GIC implementation, this means they are routed to a
> random online CPU. This may cause extra moves if multiple cores are
> disabled in sequence, but should prevent all interrupts from piling up
> on CPU0 in case of repeated up-down cycles on different cores.
Wouldn't they eventually all move to CPU0 in the case of suspend/resume
or if all the CPUs but CPU0 are turned off and then off? If so,
shouldn't we try to rebalance the interrupts?
>
> IRQs from CPU 0 are never migrated, as dying CPU 0 means we are either
> shutting down compeletely or entering system suspend.
I can't find a place where __cpu_disable() is called on CPU0. Do you
have any pointer? In any case, I am not sure I want to bake that
assumption in more places of the code.
>
> Considering that all Xen-used IRQs are currently allocated during init
> on CPU 0, and setup_irq uses smp_processor_id for the initial affinity.
Looking at the SMMU driver, we seems to request IRQs at the time the
device is attached. So are you sure about this?
> This change is not strictly required for correct operation for now, but
> it should future-proof cpu hotplug and system suspend support in case
> some kind if IRQ balancing is implemented later.
>
> Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
>
> v3->v4:
> * patch introduced
> ---
> xen/arch/arm/include/asm/irq.h | 2 ++
> xen/arch/arm/irq.c | 39 ++++++++++++++++++++++++++++++++++
> xen/arch/arm/smpboot.c | 2 ++
> 3 files changed, 43 insertions(+)
>
> diff --git a/xen/arch/arm/include/asm/irq.h b/xen/arch/arm/include/asm/irq.h
> index 09788dbfeb..6e6e27bb80 100644
> --- a/xen/arch/arm/include/asm/irq.h
> +++ b/xen/arch/arm/include/asm/irq.h
> @@ -126,6 +126,8 @@ bool irq_type_set_by_domain(const struct domain *d);
> void irq_end_none(struct irq_desc *irq);
> #define irq_end_none irq_end_none
>
> +void evacuate_irqs(unsigned int from);
> +
> #endif /* _ASM_HW_IRQ_H */
> /*
> * Local variables:
> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
> index 28b40331f7..b383d71930 100644
> --- a/xen/arch/arm/irq.c
> +++ b/xen/arch/arm/irq.c
> @@ -158,6 +158,45 @@ static int init_local_irq_data(unsigned int cpu)
> return 0;
> }
>
> +static void evacuate_irq(int irq, unsigned int from)
Any reason why the 'irq' is signed?
> +{
> + struct irq_desc *desc = irq_to_desc(irq);
> + unsigned long flags;
> +
> + /* Don't move irqs from CPU 0 as it is always last to be disabled */
Per above, I am not convinced that we should special case CPU 0. But if
we do, then shouldn't this be part of evacuate_irqs() so we don't
pointlessly go through all the IRQs?
> + if ( from == 0 )
> + return;
> +
> + ASSERT(!cpumask_empty(&cpu_online_map));
> + ASSERT(!cpumask_test_cpu(from, &cpu_online_map));
> +
> + spin_lock_irqsave(&desc->lock, flags);
> + if ( likely(!desc->action) )
> + goto out;
> +
> + if ( likely(test_bit(_IRQ_GUEST, &desc->status) ||
> + test_bit(_IRQ_MOVE_PENDING, &desc->status)) )
> + goto out;
> +
> + if ( cpumask_test_cpu(from, desc->affinity) )
> + irq_set_affinity(desc, &cpu_online_map);
I think it would be worth explaining why we are routing to any CPU
online rather than checking whether the affinity has other online CPUs.
Just to note, I don't have strong opinion either way. It mainly needs to
be documented.
> +
> +out:
> + spin_unlock_irqrestore(&desc->lock, flags);
> + return;
> +}
> +
> +void evacuate_irqs(unsigned int from)
> +{
> + int irq;
> +> + for ( irq = NR_LOCAL_IRQS; irq < NR_IRQS; irq++ )
> + evacuate_irq(irq, from);
> +
> + for ( irq = ESPI_BASE_INTID; irq < ESPI_MAX_INTID; irq++ )
AFAICT, irq_to_desc() would not be able to cope with ESPI interrupts
when CONFIG_GICV3_ESPI is not set. Has this been tested?
> + evacuate_irq(irq, from);
> +}
> +
> static int cpu_callback(struct notifier_block *nfb, unsigned long action,
> void *hcpu)
> {
> diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
> index 7f3cfa812e..46b24783dd 100644
> --- a/xen/arch/arm/smpboot.c
> +++ b/xen/arch/arm/smpboot.c
> @@ -425,6 +425,8 @@ void __cpu_disable(void)
>
> smp_mb();
>
> + evacuate_irqs(cpu);
I think it would be worth explaining why evacuate_irqs() is called this
late in the process.
> +> /* Return to caller; eventually the IPI mechanism will
unwind and the
> * scheduler will drop to the idle loop, which will call stop_cpu(). */
> }
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 20+ messages in thread* Re: [PATCH v4 4/8] arm/irq: Migrate IRQs from dyings CPUs
2025-11-16 11:24 ` Julien Grall
@ 2025-12-12 10:01 ` Mykyta Poturai
2025-12-31 15:30 ` Julien Grall
0 siblings, 1 reply; 20+ messages in thread
From: Mykyta Poturai @ 2025-12-12 10:01 UTC (permalink / raw)
To: Julien Grall, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk
On 16.11.25 13:24, Julien Grall wrote:
> Hi,
>
> On 12/11/2025 10:51, Mykyta Poturai wrote:
>> Move IRQs from dying CPU to the online ones.
>> Guest-bound IRQs are already handled by scheduler in the process of
>> moving vCPUs to active pCPUs, so we only need to handle IRQs used by Xen
>> itself.
>>
>> If IRQ is to be migrated, it's affinity is set to a mask of all online
>> CPUs. With current GIC implementation, this means they are routed to a
>> random online CPU. This may cause extra moves if multiple cores are
>> disabled in sequence, but should prevent all interrupts from piling up
>> on CPU0 in case of repeated up-down cycles on different cores.
>
> Wouldn't they eventually all move to CPU0 in the case of suspend/resume
> or if all the CPUs but CPU0 are turned off and then off? If so,
> shouldn't we try to rebalance the interrupts?
>
In case of disabling/enabling all cores in a sequence, yes. This was
designed with the idea of achieving some balancing when
enabling/disabling some cores for power saving reasons. I agree that
proper balancing should be implemented, but it is a complex task on its
own and requires a substantial amount of testing on different hardware
to prove it is close to optimal. So I think implementing it is out of
scope for what I’m trying to do here.
If this would be okay, I can implement a relatively simple solution of
just adding onlined CPUs back to the affinity mask for now. I think it
should improve the situation for the “switching all cores” case.
>>
>> IRQs from CPU 0 are never migrated, as dying CPU 0 means we are either
>> shutting down compeletely or entering system suspend.
>
> I can't find a place where __cpu_disable() is called on CPU0. Do you
> have any pointer? In any case, I am not sure I want to bake that
> assumption in more places of the code.
>
I assume it would be called when suspend is implemented. In any case, I
will rework this to replace the hard check for CPU 0 with the “is it the
last CPU online” one.
>>
>> Considering that all Xen-used IRQs are currently allocated during init
>> on CPU 0, and setup_irq uses smp_processor_id for the initial affinity.
>
> Looking at the SMMU driver, we seems to request IRQs at the time the
> device is attached. So are you sure about this?
>
Indeed, I have missed that one. I will remove those statements then.
>> This change is not strictly required for correct operation for now, but
>> it should future-proof cpu hotplug and system suspend support in case
>> some kind if IRQ balancing is implemented later.
>>
>> Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
>>
>> v3->v4:
>> * patch introduced
>> ---
>> xen/arch/arm/include/asm/irq.h | 2 ++
>> xen/arch/arm/irq.c | 39 ++++++++++++++++++++++++++++++++++
>> xen/arch/arm/smpboot.c | 2 ++
>> 3 files changed, 43 insertions(+)
>>
>> diff --git a/xen/arch/arm/include/asm/irq.h b/xen/arch/arm/include/
>> asm/irq.h
>> index 09788dbfeb..6e6e27bb80 100644
>> --- a/xen/arch/arm/include/asm/irq.h
>> +++ b/xen/arch/arm/include/asm/irq.h
>> @@ -126,6 +126,8 @@ bool irq_type_set_by_domain(const struct domain *d);
>> void irq_end_none(struct irq_desc *irq);
>> #define irq_end_none irq_end_none
>> +void evacuate_irqs(unsigned int from);
>> +
>> #endif /* _ASM_HW_IRQ_H */
>> /*
>> * Local variables:
>> diff --git a/xen/arch/arm/irq.c b/xen/arch/arm/irq.c
>> index 28b40331f7..b383d71930 100644
>> --- a/xen/arch/arm/irq.c
>> +++ b/xen/arch/arm/irq.c
>> @@ -158,6 +158,45 @@ static int init_local_irq_data(unsigned int cpu)
>> return 0;
>> }
>> +static void evacuate_irq(int irq, unsigned int from)
>
> Any reason why the 'irq' is signed?
>
>> +{
>> + struct irq_desc *desc = irq_to_desc(irq);
>> + unsigned long flags;
>> +
>> + /* Don't move irqs from CPU 0 as it is always last to be disabled */
>
> Per above, I am not convinced that we should special case CPU 0. But if
> we do, then shouldn't this be part of evacuate_irqs() so we don't
> pointlessly go through all the IRQs?
>
>> + if ( from == 0 )
>> + return;
>> +
>> + ASSERT(!cpumask_empty(&cpu_online_map));
>> + ASSERT(!cpumask_test_cpu(from, &cpu_online_map));
>> +
>> + spin_lock_irqsave(&desc->lock, flags);
>> + if ( likely(!desc->action) )
>> + goto out;
>> +
>> + if ( likely(test_bit(_IRQ_GUEST, &desc->status) ||
>> + test_bit(_IRQ_MOVE_PENDING, &desc->status)) )
>> + goto out;
>> +
>> + if ( cpumask_test_cpu(from, desc->affinity) )
>> + irq_set_affinity(desc, &cpu_online_map);
>
> I think it would be worth explaining why we are routing to any CPU
> online rather than checking whether the affinity has other online CPUs.
>
> Just to note, I don't have strong opinion either way. It mainly needs to
> be documented.
>
>> +
>> +out:
>> + spin_unlock_irqrestore(&desc->lock, flags);
>> + return;
>> +}
>> +
>> +void evacuate_irqs(unsigned int from)
>> +{
>> + int irq;
> > +> + for ( irq = NR_LOCAL_IRQS; irq < NR_IRQS; irq++ )
>> + evacuate_irq(irq, from);
>> +
>> + for ( irq = ESPI_BASE_INTID; irq < ESPI_MAX_INTID; irq++ )
>
> AFAICT, irq_to_desc() would not be able to cope with ESPI interrupts
> when CONFIG_GICV3_ESPI is not set. Has this been tested?
>
>> + evacuate_irq(irq, from);
>> +}
>> +
>> static int cpu_callback(struct notifier_block *nfb, unsigned long
>> action,
>> void *hcpu)
>> {
>> diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
>> index 7f3cfa812e..46b24783dd 100644
>> --- a/xen/arch/arm/smpboot.c
>> +++ b/xen/arch/arm/smpboot.c
>> @@ -425,6 +425,8 @@ void __cpu_disable(void)
>> smp_mb();
>> + evacuate_irqs(cpu);
>
> I think it would be worth explaining why evacuate_irqs() is called this
> late in the process.
>
> > +> /* Return to caller; eventually the IPI mechanism will
> unwind and the
>> * scheduler will drop to the idle loop, which will call
>> stop_cpu(). */
>> }
>
> Cheers,
>
--
Mykyta
^ permalink raw reply [flat|nested] 20+ messages in thread* Re: [PATCH v4 4/8] arm/irq: Migrate IRQs from dyings CPUs
2025-12-12 10:01 ` Mykyta Poturai
@ 2025-12-31 15:30 ` Julien Grall
0 siblings, 0 replies; 20+ messages in thread
From: Julien Grall @ 2025-12-31 15:30 UTC (permalink / raw)
To: Mykyta Poturai, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk
Hi,
On 12/12/2025 10:01, Mykyta Poturai wrote:
> On 16.11.25 13:24, Julien Grall wrote:
>> Hi,
>>
>> On 12/11/2025 10:51, Mykyta Poturai wrote:
>>> Move IRQs from dying CPU to the online ones.
>>> Guest-bound IRQs are already handled by scheduler in the process of
>>> moving vCPUs to active pCPUs, so we only need to handle IRQs used by Xen
>>> itself.
>>>
>>> If IRQ is to be migrated, it's affinity is set to a mask of all online
>>> CPUs. With current GIC implementation, this means they are routed to a
>>> random online CPU. This may cause extra moves if multiple cores are
>>> disabled in sequence, but should prevent all interrupts from piling up
>>> on CPU0 in case of repeated up-down cycles on different cores.
>>
>> Wouldn't they eventually all move to CPU0 in the case of suspend/resume
>> or if all the CPUs but CPU0 are turned off and then off? If so,
>> shouldn't we try to rebalance the interrupts?
>>
>
> In case of disabling/enabling all cores in a sequence, yes. This was
> designed with the idea of achieving some balancing when
> enabling/disabling some cores for power saving reasons.
I understand how this may balance when disabling some cores. But I don't
understand how it helps for enabling cores. Can you provide more details?
> I agree that
> proper balancing should be implemented, but it is a complex task on its
> own and requires a substantial amount of testing on different hardware
> to prove it is close to optimal. So I think implementing it is out of
> scope for what I’m trying to do here.
Can you provide some details about what you are trying and why it would
be ok to avoid the balancing?
> > If this would be okay, I can implement a relatively simple solution of
> just adding onlined CPUs back to the affinity mask for now. I think it
> should improve the situation for the “switching all cores” case.
Do you mean calling gic_irq_set_affinity() with the CPU? If so, Xen is
still going to get one CPU from the affinity mask.
>
>>>
>>> IRQs from CPU 0 are never migrated, as dying CPU 0 means we are either
>>> shutting down compeletely or entering system suspend.
>>
>> I can't find a place where __cpu_disable() is called on CPU0. Do you
>> have any pointer? In any case, I am not sure I want to bake that
>> assumption in more places of the code.
>>
>
> I assume it would be called when suspend is implemented. In any case, I
> will rework this to replace the hard check for CPU 0 with the “is it the
> last CPU online” one.
AFAIK __cpu_disable() is not going to be called during suspend/resume
for CPU0. So the only case is shutting down the platform.
>
>>>
>>> Considering that all Xen-used IRQs are currently allocated during init
>>> on CPU 0, and setup_irq uses smp_processor_id for the initial affinity.
>>
>> Looking at the SMMU driver, we seems to request IRQs at the time the
>> device is attached. So are you sure about this?
>>
>
> Indeed, I have missed that one. I will remove those statements then.
I think you need to have something in the commit message explaining why
you are ignoring the balancing for the SMMU.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v4 6/8] arm/sysctl: Implement cpu hotplug ops
2025-11-12 10:51 [PATCH v4 0/8] Implement CPU hotplug on Arm Mykyta Poturai
` (3 preceding siblings ...)
2025-11-12 10:51 ` [PATCH v4 4/8] arm/irq: Migrate IRQs from dyings CPUs Mykyta Poturai
@ 2025-11-12 10:51 ` Mykyta Poturai
2025-11-13 15:49 ` Grygorii Strashko
2025-11-16 11:34 ` Julien Grall
2025-11-12 10:51 ` [PATCH v4 5/8] smp: Move cpu_up/down helpers to common code Mykyta Poturai
` (2 subsequent siblings)
7 siblings, 2 replies; 20+ messages in thread
From: Mykyta Poturai @ 2025-11-12 10:51 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Mykyta Poturai, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk,
Daniel P. Smith
Implement XEN_SYSCTL_CPU_HOTPLUG_{ONLINE,OFFLINE} calls to allow for
enabling/disabling CPU cores in runtime.
For now this operations only support Arm64. For proper Arm32 support,
there needs to be a mechanism to free per-cpu page tables, allocated in
init_domheap_mappings.
Also, hotplug is not supported if ITS, FFA, or TEE is enabled, as they
use non-static IRQ actions.
Create a Kconfig option RUNTIME_CPU_CONTROL that reflects this
constraints.
Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
v3->v4:
* don't reimplement cpu_up/down helpers
* add Kconfig option
* fixup formatting
v2->v3:
* no changes
v1->v2:
* remove SMT ops
* remove cpu == 0 checks
* add XSM hooks
* only implement for 64bit Arm
---
xen/arch/arm/Kconfig | 4 ++++
xen/arch/arm/sysctl.c | 32 ++++++++++++++++++++++++++++++++
2 files changed, 36 insertions(+)
diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index cf6af68299..931ae51575 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -274,6 +274,10 @@ config PCI_PASSTHROUGH
help
This option enables PCI device passthrough
+config RUNTIME_CPU_CONTROL
+ def_bool y
+ depends on ARM_64 && !TEE && !FFA && !HAS_ITS
+
endmenu
menu "ARM errata workaround via the alternative framework"
diff --git a/xen/arch/arm/sysctl.c b/xen/arch/arm/sysctl.c
index 32cab4feff..3c4e29d82c 100644
--- a/xen/arch/arm/sysctl.c
+++ b/xen/arch/arm/sysctl.c
@@ -12,6 +12,7 @@
#include <xen/dt-overlay.h>
#include <xen/errno.h>
#include <xen/hypercall.h>
+#include <xsm/xsm.h>
#include <asm/arm64/sve.h>
#include <public/sysctl.h>
@@ -23,6 +24,33 @@ void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
XEN_SYSCTL_PHYSCAP_ARM_SVE_MASK);
}
+static long cpu_hotplug_sysctl(struct xen_sysctl_cpu_hotplug *hotplug)
+{
+#ifdef CONFIG_RUNTIME_CPU_CONTROL
+ int ret;
+
+ switch ( hotplug->op )
+ {
+ case XEN_SYSCTL_CPU_HOTPLUG_ONLINE:
+ ret = xsm_resource_plug_core(XSM_HOOK);
+ if ( ret )
+ return ret;
+ return continue_hypercall_on_cpu(0, cpu_up_helper, _p(hotplug->cpu));
+
+ case XEN_SYSCTL_CPU_HOTPLUG_OFFLINE:
+ ret = xsm_resource_unplug_core(XSM_HOOK);
+ if ( ret )
+ return ret;
+ return continue_hypercall_on_cpu(0, cpu_down_helper, _p(hotplug->cpu));
+
+ default:
+ return -EOPNOTSUPP;
+ }
+#else
+ return -EOPNOTSUPP;
+#endif
+}
+
long arch_do_sysctl(struct xen_sysctl *sysctl,
XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl)
{
@@ -34,6 +62,10 @@ long arch_do_sysctl(struct xen_sysctl *sysctl,
ret = dt_overlay_sysctl(&sysctl->u.dt_overlay);
break;
+ case XEN_SYSCTL_cpu_hotplug:
+ ret = cpu_hotplug_sysctl(&sysctl->u.cpu_hotplug);
+ break;
+
default:
ret = -ENOSYS;
break;
--
2.51.2
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [PATCH v4 6/8] arm/sysctl: Implement cpu hotplug ops
2025-11-12 10:51 ` [PATCH v4 6/8] arm/sysctl: Implement cpu hotplug ops Mykyta Poturai
@ 2025-11-13 15:49 ` Grygorii Strashko
2025-11-21 16:12 ` Mykyta Poturai
2025-11-16 11:34 ` Julien Grall
1 sibling, 1 reply; 20+ messages in thread
From: Grygorii Strashko @ 2025-11-13 15:49 UTC (permalink / raw)
To: Mykyta Poturai, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini, Julien Grall, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk, Daniel P. Smith,
Alejandro.GarciaVallejo@amd.com
On 12.11.25 12:51, Mykyta Poturai wrote:
> Implement XEN_SYSCTL_CPU_HOTPLUG_{ONLINE,OFFLINE} calls to allow for
> enabling/disabling CPU cores in runtime.
>
> For now this operations only support Arm64. For proper Arm32 support,
> there needs to be a mechanism to free per-cpu page tables, allocated in
> init_domheap_mappings.
> Also, hotplug is not supported if ITS, FFA, or TEE is enabled, as they
> use non-static IRQ actions.
>
> Create a Kconfig option RUNTIME_CPU_CONTROL that reflects this
> constraints.
>
> Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
>
> v3->v4:
> * don't reimplement cpu_up/down helpers
> * add Kconfig option
> * fixup formatting
>
> v2->v3:
> * no changes
>
> v1->v2:
> * remove SMT ops
> * remove cpu == 0 checks
> * add XSM hooks
> * only implement for 64bit Arm
> ---
> xen/arch/arm/Kconfig | 4 ++++
> xen/arch/arm/sysctl.c | 32 ++++++++++++++++++++++++++++++++
> 2 files changed, 36 insertions(+)
>
> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> index cf6af68299..931ae51575 100644
> --- a/xen/arch/arm/Kconfig
> +++ b/xen/arch/arm/Kconfig
> @@ -274,6 +274,10 @@ config PCI_PASSTHROUGH
> help
> This option enables PCI device passthrough
>
> +config RUNTIME_CPU_CONTROL
> + def_bool y
> + depends on ARM_64 && !TEE && !FFA && !HAS_ITS
> +
Having it in arch code I think right as Arch can have own deps
(Linux does it this way), but name has to be fixed and documented (docs).
may be more common name HOTPLUG_CPU (like in linux)?
naming is up to you.
> endmenu
>
> menu "ARM errata workaround via the alternative framework"
> diff --git a/xen/arch/arm/sysctl.c b/xen/arch/arm/sysctl.c
> index 32cab4feff..3c4e29d82c 100644
> --- a/xen/arch/arm/sysctl.c
> +++ b/xen/arch/arm/sysctl.c
> @@ -12,6 +12,7 @@
> #include <xen/dt-overlay.h>
> #include <xen/errno.h>
> #include <xen/hypercall.h>
> +#include <xsm/xsm.h>
> #include <asm/arm64/sve.h>
> #include <public/sysctl.h>
>
> @@ -23,6 +24,33 @@ void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
> XEN_SYSCTL_PHYSCAP_ARM_SVE_MASK);
> }
>
> +static long cpu_hotplug_sysctl(struct xen_sysctl_cpu_hotplug *hotplug)
> +{
> +#ifdef CONFIG_RUNTIME_CPU_CONTROL
> + int ret;
> +
> + switch ( hotplug->op )
> + {
> + case XEN_SYSCTL_CPU_HOTPLUG_ONLINE:
> + ret = xsm_resource_plug_core(XSM_HOOK);
> + if ( ret )
> + return ret;
> + return continue_hypercall_on_cpu(0, cpu_up_helper, _p(hotplug->cpu));
> +
> + case XEN_SYSCTL_CPU_HOTPLUG_OFFLINE:
> + ret = xsm_resource_unplug_core(XSM_HOOK);
> + if ( ret )
> + return ret;
> + return continue_hypercall_on_cpu(0, cpu_down_helper, _p(hotplug->cpu));
> +
> + default:
> + return -EOPNOTSUPP;
> + }
> +#else
> + return -EOPNOTSUPP;
> +#endif
> +}
> +
> long arch_do_sysctl(struct xen_sysctl *sysctl,
> XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl)
> {
> @@ -34,6 +62,10 @@ long arch_do_sysctl(struct xen_sysctl *sysctl,
> ret = dt_overlay_sysctl(&sysctl->u.dt_overlay);
> break;
>
> + case XEN_SYSCTL_cpu_hotplug:
> + ret = cpu_hotplug_sysctl(&sysctl->u.cpu_hotplug);
> + break;
> +
> default:
> ret = -ENOSYS;
> break;
Common code need to be placed under same config guards.
--
Best regards,
-grygorii
^ permalink raw reply [flat|nested] 20+ messages in thread* Re: [PATCH v4 6/8] arm/sysctl: Implement cpu hotplug ops
2025-11-13 15:49 ` Grygorii Strashko
@ 2025-11-21 16:12 ` Mykyta Poturai
0 siblings, 0 replies; 20+ messages in thread
From: Mykyta Poturai @ 2025-11-21 16:12 UTC (permalink / raw)
To: Grygorii Strashko, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini, Julien Grall, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk, Daniel P. Smith,
Alejandro.GarciaVallejo@amd.com
On 13.11.25 17:49, Grygorii Strashko wrote:
>
>
> On 12.11.25 12:51, Mykyta Poturai wrote:
>> Implement XEN_SYSCTL_CPU_HOTPLUG_{ONLINE,OFFLINE} calls to allow for
>> enabling/disabling CPU cores in runtime.
>>
>> For now this operations only support Arm64. For proper Arm32 support,
>> there needs to be a mechanism to free per-cpu page tables, allocated in
>> init_domheap_mappings.
>> Also, hotplug is not supported if ITS, FFA, or TEE is enabled, as they
>> use non-static IRQ actions.
>>
>> Create a Kconfig option RUNTIME_CPU_CONTROL that reflects this
>> constraints.
>>
>> Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
>>
>> v3->v4:
>> * don't reimplement cpu_up/down helpers
>> * add Kconfig option
>> * fixup formatting
>>
>> v2->v3:
>> * no changes
>>
>> v1->v2:
>> * remove SMT ops
>> * remove cpu == 0 checks
>> * add XSM hooks
>> * only implement for 64bit Arm
>> ---
>> xen/arch/arm/Kconfig | 4 ++++
>> xen/arch/arm/sysctl.c | 32 ++++++++++++++++++++++++++++++++
>> 2 files changed, 36 insertions(+)
>>
>> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
>> index cf6af68299..931ae51575 100644
>> --- a/xen/arch/arm/Kconfig
>> +++ b/xen/arch/arm/Kconfig
>> @@ -274,6 +274,10 @@ config PCI_PASSTHROUGH
>> help
>> This option enables PCI device passthrough
>> +config RUNTIME_CPU_CONTROL
>> + def_bool y
>> + depends on ARM_64 && !TEE && !FFA && !HAS_ITS
>> +
>
> Having it in arch code I think right as Arch can have own deps
> (Linux does it this way), but name has to be fixed and documented (docs).
>
> may be more common name HOTPLUG_CPU (like in linux)?
> naming is up to you.
>
I will change the config name to "CPU_HOTPLUG" in the next version if
everyone is okay with that.
--
Mykyta
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH v4 6/8] arm/sysctl: Implement cpu hotplug ops
2025-11-12 10:51 ` [PATCH v4 6/8] arm/sysctl: Implement cpu hotplug ops Mykyta Poturai
2025-11-13 15:49 ` Grygorii Strashko
@ 2025-11-16 11:34 ` Julien Grall
1 sibling, 0 replies; 20+ messages in thread
From: Julien Grall @ 2025-11-16 11:34 UTC (permalink / raw)
To: Mykyta Poturai, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk, Daniel P. Smith
Hi,
On 12/11/2025 10:51, Mykyta Poturai wrote:
> Implement XEN_SYSCTL_CPU_HOTPLUG_{ONLINE,OFFLINE} calls to allow for
> enabling/disabling CPU cores in runtime.
>
> For now this operations only support Arm64. For proper Arm32 support,
> there needs to be a mechanism to free per-cpu page tables, allocated in
> init_domheap_mappings.
> Also, hotplug is not supported if ITS, FFA, or TEE is enabled, as they
> use non-static IRQ actions.
>
> Create a Kconfig option RUNTIME_CPU_CONTROL that reflects this
> constraints.
>
> Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
>
> v3->v4:
> * don't reimplement cpu_up/down helpers
> * add Kconfig option
> * fixup formatting
>
> v2->v3:
> * no changes
>
> v1->v2:
> * remove SMT ops
> * remove cpu == 0 checks
> * add XSM hooks
> * only implement for 64bit Arm
> ---
> xen/arch/arm/Kconfig | 4 ++++
> xen/arch/arm/sysctl.c | 32 ++++++++++++++++++++++++++++++++
> 2 files changed, 36 insertions(+)
>
> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> index cf6af68299..931ae51575 100644
> --- a/xen/arch/arm/Kconfig
> +++ b/xen/arch/arm/Kconfig
> @@ -274,6 +274,10 @@ config PCI_PASSTHROUGH
> help
> This option enables PCI device passthrough
>
> +config RUNTIME_CPU_CONTROL
> + def_bool y
> + depends on ARM_64 && !TEE && !FFA && !HAS_ITS
> +
> endmenu
>
> menu "ARM errata workaround via the alternative framework"
> diff --git a/xen/arch/arm/sysctl.c b/xen/arch/arm/sysctl.c
> index 32cab4feff..3c4e29d82c 100644
> --- a/xen/arch/arm/sysctl.c
> +++ b/xen/arch/arm/sysctl.c
> @@ -12,6 +12,7 @@
> #include <xen/dt-overlay.h>
> #include <xen/errno.h>
> #include <xen/hypercall.h>
> +#include <xsm/xsm.h>
> #include <asm/arm64/sve.h>
> #include <public/sysctl.h>
>
> @@ -23,6 +24,33 @@ void arch_do_physinfo(struct xen_sysctl_physinfo *pi)
> XEN_SYSCTL_PHYSCAP_ARM_SVE_MASK);
> }
>
> +static long cpu_hotplug_sysctl(struct xen_sysctl_cpu_hotplug *hotplug)
As you moved the helper in common code. I was expecting the logic to
handle CPU_HOTPLUG_ONLINE and CPU_HOTPLUG_OFFLINE to also move in
common. Can you explain why this wasn't done?
> +{
> +#ifdef CONFIG_RUNTIME_CPU_CONTROL
> + int ret;
> +
> + switch ( hotplug->op )
> + {
> + case XEN_SYSCTL_CPU_HOTPLUG_ONLINE:
> + ret = xsm_resource_plug_core(XSM_HOOK);
> + if ( ret )
> + return ret;
> + return continue_hypercall_on_cpu(0, cpu_up_helper, _p(hotplug->cpu));
> +
> + case XEN_SYSCTL_CPU_HOTPLUG_OFFLINE:
> + ret = xsm_resource_unplug_core(XSM_HOOK);
> + if ( ret )
> + return ret;
> + return continue_hypercall_on_cpu(0, cpu_down_helper, _p(hotplug->cpu));
> +
> + default:
> + return -EOPNOTSUPP;
> + }
> +#else
> + return -EOPNOTSUPP;
> +#endif
> +}
> +
> long arch_do_sysctl(struct xen_sysctl *sysctl,
> XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl)
> {
> @@ -34,6 +62,10 @@ long arch_do_sysctl(struct xen_sysctl *sysctl,
> ret = dt_overlay_sysctl(&sysctl->u.dt_overlay);
> break;
>
> + case XEN_SYSCTL_cpu_hotplug:
> + ret = cpu_hotplug_sysctl(&sysctl->u.cpu_hotplug);
> + break;
> +
> default:
> ret = -ENOSYS;
> break;
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v4 5/8] smp: Move cpu_up/down helpers to common code
2025-11-12 10:51 [PATCH v4 0/8] Implement CPU hotplug on Arm Mykyta Poturai
` (4 preceding siblings ...)
2025-11-12 10:51 ` [PATCH v4 6/8] arm/sysctl: Implement cpu hotplug ops Mykyta Poturai
@ 2025-11-12 10:51 ` Mykyta Poturai
2025-11-13 10:37 ` Jan Beulich
2025-11-16 11:29 ` Julien Grall
2025-11-12 10:51 ` [PATCH v4 8/8] docs: Document CPU hotplug Mykyta Poturai
2025-11-12 10:51 ` [PATCH v4 7/8] tools: Allow building xen-hptool without CONFIG_MIGRATE Mykyta Poturai
7 siblings, 2 replies; 20+ messages in thread
From: Mykyta Poturai @ 2025-11-12 10:51 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Mykyta Poturai, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk, Andrew Cooper,
Anthony PERARD, Jan Beulich, Roger Pau Monné,
Timothy Pearson, Alistair Francis, Bob Eshleman, Connor Davis,
Oleksii Kurochko
This will reduce code duplication for the upcoming cpu hotplug support
on Arm64 patch.
SMT-disable enforcement check is moved into a separate
architecture-specific function.
Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
v3->v4:
* patch introduced
---
xen/arch/arm/smp.c | 6 ++++++
xen/arch/ppc/stubs.c | 4 ++++
xen/arch/riscv/stubs.c | 5 +++++
xen/arch/x86/include/asm/smp.h | 3 ---
xen/arch/x86/smp.c | 33 +++------------------------------
xen/common/smp.c | 32 ++++++++++++++++++++++++++++++++
xen/include/xen/smp.h | 4 ++++
7 files changed, 54 insertions(+), 33 deletions(-)
diff --git a/xen/arch/arm/smp.c b/xen/arch/arm/smp.c
index b372472188..85815aeda0 100644
--- a/xen/arch/arm/smp.c
+++ b/xen/arch/arm/smp.c
@@ -44,6 +44,12 @@ void smp_send_call_function_mask(const cpumask_t *mask)
}
}
+/* ARM don't have SMT so we don't need any special logic for CPU disabling */
+bool arch_smt_cpu_disable(unsigned int cpu)
+{
+ return false;
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/arch/ppc/stubs.c b/xen/arch/ppc/stubs.c
index bdaf474c5c..ca97cec51f 100644
--- a/xen/arch/ppc/stubs.c
+++ b/xen/arch/ppc/stubs.c
@@ -101,6 +101,10 @@ void smp_send_call_function_mask(const cpumask_t *mask)
BUG_ON("unimplemented");
}
+bool arch_smt_cpu_disable(unsigned int cpu)
+{
+ BUG_ON("unimplemented");
+}
/* irq.c */
void irq_ack_none(struct irq_desc *desc)
diff --git a/xen/arch/riscv/stubs.c b/xen/arch/riscv/stubs.c
index 1a8c86cd8d..60610349cb 100644
--- a/xen/arch/riscv/stubs.c
+++ b/xen/arch/riscv/stubs.c
@@ -80,6 +80,11 @@ void smp_send_call_function_mask(const cpumask_t *mask)
BUG_ON("unimplemented");
}
+bool arch_smt_cpu_disable(unsigned int cpu)
+{
+ BUG_ON("unimplemented");
+}
+
/* irq.c */
void irq_ack_none(struct irq_desc *desc)
diff --git a/xen/arch/x86/include/asm/smp.h b/xen/arch/x86/include/asm/smp.h
index 60eb4ac254..b77fc0bc6d 100644
--- a/xen/arch/x86/include/asm/smp.h
+++ b/xen/arch/x86/include/asm/smp.h
@@ -50,9 +50,6 @@ int cpu_add(uint32_t apic_id, uint32_t acpi_id, uint32_t pxm);
void __stop_this_cpu(void);
-long cf_check cpu_up_helper(void *data);
-long cf_check cpu_down_helper(void *data);
-
long cf_check core_parking_helper(void *data);
bool core_parking_remove(unsigned int cpu);
uint32_t get_cur_idle_nums(void);
diff --git a/xen/arch/x86/smp.c b/xen/arch/x86/smp.c
index 7936294f5f..d64b533cc0 100644
--- a/xen/arch/x86/smp.c
+++ b/xen/arch/x86/smp.c
@@ -418,35 +418,8 @@ void cf_check call_function_interrupt(void)
smp_call_function_interrupt();
}
-long cf_check cpu_up_helper(void *data)
+bool arch_smt_cpu_disable(unsigned int cpu)
{
- unsigned int cpu = (unsigned long)data;
- int ret = cpu_up(cpu);
-
- /* Have one more go on EBUSY. */
- if ( ret == -EBUSY )
- ret = cpu_up(cpu);
-
- if ( !ret && !opt_smt &&
- cpu_data[cpu].compute_unit_id == INVALID_CUID &&
- cpumask_weight(per_cpu(cpu_sibling_mask, cpu)) > 1 )
- {
- ret = cpu_down_helper(data);
- if ( ret )
- printk("Could not re-offline CPU%u (%d)\n", cpu, ret);
- else
- ret = -EPERM;
- }
-
- return ret;
-}
-
-long cf_check cpu_down_helper(void *data)
-{
- int cpu = (unsigned long)data;
- int ret = cpu_down(cpu);
- /* Have one more go on EBUSY. */
- if ( ret == -EBUSY )
- ret = cpu_down(cpu);
- return ret;
+ return !opt_smt && cpu_data[cpu].compute_unit_id == INVALID_CUID &&
+ cpumask_weight(per_cpu(cpu_sibling_mask, cpu)) > 1;
}
diff --git a/xen/common/smp.c b/xen/common/smp.c
index a011f541f1..114c1da77d 100644
--- a/xen/common/smp.c
+++ b/xen/common/smp.c
@@ -16,6 +16,7 @@
* GNU General Public License for more details.
*/
+#include <xen/cpu.h>
#include <asm/hardirq.h>
#include <asm/processor.h>
#include <xen/spinlock.h>
@@ -104,6 +105,37 @@ void smp_call_function_interrupt(void)
irq_exit();
}
+long cf_check cpu_up_helper(void *data)
+{
+ unsigned int cpu = (unsigned long)data;
+ int ret = cpu_up(cpu);
+
+ /* Have one more go on EBUSY. */
+ if ( ret == -EBUSY )
+ ret = cpu_up(cpu);
+
+ if ( !ret && arch_smt_cpu_disable(cpu) )
+ {
+ ret = cpu_down_helper(data);
+ if ( ret )
+ printk("Could not re-offline CPU%u (%d)\n", cpu, ret);
+ else
+ ret = -EPERM;
+ }
+
+ return ret;
+}
+
+long cf_check cpu_down_helper(void *data)
+{
+ int cpu = (unsigned long)data;
+ int ret = cpu_down(cpu);
+ /* Have one more go on EBUSY. */
+ if ( ret == -EBUSY )
+ ret = cpu_down(cpu);
+ return ret;
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/include/xen/smp.h b/xen/include/xen/smp.h
index 2ca9ff1bfc..c734033bfb 100644
--- a/xen/include/xen/smp.h
+++ b/xen/include/xen/smp.h
@@ -76,4 +76,8 @@ extern void *stack_base[NR_CPUS];
void initialize_cpu_data(unsigned int cpu);
int setup_cpu_root_pgt(unsigned int cpu);
+bool arch_smt_cpu_disable(unsigned int cpu);
+long cf_check cpu_up_helper(void *data);
+long cf_check cpu_down_helper(void *data);
+
#endif /* __XEN_SMP_H__ */
--
2.51.2
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [PATCH v4 5/8] smp: Move cpu_up/down helpers to common code
2025-11-12 10:51 ` [PATCH v4 5/8] smp: Move cpu_up/down helpers to common code Mykyta Poturai
@ 2025-11-13 10:37 ` Jan Beulich
2025-11-16 11:29 ` Julien Grall
1 sibling, 0 replies; 20+ messages in thread
From: Jan Beulich @ 2025-11-13 10:37 UTC (permalink / raw)
To: Mykyta Poturai
Cc: Stefano Stabellini, Julien Grall, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk, Andrew Cooper, Anthony PERARD,
Roger Pau Monné, Timothy Pearson, Alistair Francis,
Bob Eshleman, Connor Davis, Oleksii Kurochko,
xen-devel@lists.xenproject.org
On 12.11.2025 11:51, Mykyta Poturai wrote:
> This will reduce code duplication for the upcoming cpu hotplug support
> on Arm64 patch.
>
> SMT-disable enforcement check is moved into a separate
> architecture-specific function.
>
> Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
Solely from an x86 perspective this looks okay to me, but on Arm you introduce
...
> --- a/xen/common/smp.c
> +++ b/xen/common/smp.c
> @@ -16,6 +16,7 @@
> * GNU General Public License for more details.
> */
>
> +#include <xen/cpu.h>
> #include <asm/hardirq.h>
> #include <asm/processor.h>
> #include <xen/spinlock.h>
> @@ -104,6 +105,37 @@ void smp_call_function_interrupt(void)
> irq_exit();
> }
>
> +long cf_check cpu_up_helper(void *data)
> +{
> + unsigned int cpu = (unsigned long)data;
> + int ret = cpu_up(cpu);
> +
> + /* Have one more go on EBUSY. */
> + if ( ret == -EBUSY )
> + ret = cpu_up(cpu);
> +
> + if ( !ret && arch_smt_cpu_disable(cpu) )
> + {
> + ret = cpu_down_helper(data);
> + if ( ret )
> + printk("Could not re-offline CPU%u (%d)\n", cpu, ret);
> + else
> + ret = -EPERM;
> + }
> +
> + return ret;
> +}
> +
> +long cf_check cpu_down_helper(void *data)
> +{
> + int cpu = (unsigned long)data;
> + int ret = cpu_down(cpu);
> + /* Have one more go on EBUSY. */
> + if ( ret == -EBUSY )
> + ret = cpu_down(cpu);
> + return ret;
> +}
...unreachable code, which - for the case when RUNTIME_CPU_CONTROL=n - won't
even be rectified by the next patch.
Jan
^ permalink raw reply [flat|nested] 20+ messages in thread* Re: [PATCH v4 5/8] smp: Move cpu_up/down helpers to common code
2025-11-12 10:51 ` [PATCH v4 5/8] smp: Move cpu_up/down helpers to common code Mykyta Poturai
2025-11-13 10:37 ` Jan Beulich
@ 2025-11-16 11:29 ` Julien Grall
1 sibling, 0 replies; 20+ messages in thread
From: Julien Grall @ 2025-11-16 11:29 UTC (permalink / raw)
To: Mykyta Poturai, xen-devel@lists.xenproject.org
Cc: Stefano Stabellini, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk, Andrew Cooper, Anthony PERARD, Jan Beulich,
Roger Pau Monné, Timothy Pearson, Alistair Francis,
Bob Eshleman, Connor Davis, Oleksii Kurochko
Hi,
On 12/11/2025 10:51, Mykyta Poturai wrote:
> This will reduce code duplication for the upcoming cpu hotplug support
> on Arm64 patch.
>
> SMT-disable enforcement check is moved into a separate
> architecture-specific function.
>
> Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
>
> v3->v4:
> * patch introduced
> ---
> xen/arch/arm/smp.c | 6 ++++++
> xen/arch/ppc/stubs.c | 4 ++++
> xen/arch/riscv/stubs.c | 5 +++++
> xen/arch/x86/include/asm/smp.h | 3 ---
> xen/arch/x86/smp.c | 33 +++------------------------------
> xen/common/smp.c | 32 ++++++++++++++++++++++++++++++++
> xen/include/xen/smp.h | 4 ++++
> 7 files changed, 54 insertions(+), 33 deletions(-)
>
> diff --git a/xen/arch/arm/smp.c b/xen/arch/arm/smp.c
> index b372472188..85815aeda0 100644
> --- a/xen/arch/arm/smp.c
> +++ b/xen/arch/arm/smp.c
> @@ -44,6 +44,12 @@ void smp_send_call_function_mask(const cpumask_t *mask)
> }
> }
>
> +/* ARM don't have SMT so we don't need any special logic for CPU disabling */
Xen doesn't support SMT on Arm. But some of the cores may support SMT.
So would reword this to:
"We currently don't support SMT"
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v4 8/8] docs: Document CPU hotplug
2025-11-12 10:51 [PATCH v4 0/8] Implement CPU hotplug on Arm Mykyta Poturai
` (5 preceding siblings ...)
2025-11-12 10:51 ` [PATCH v4 5/8] smp: Move cpu_up/down helpers to common code Mykyta Poturai
@ 2025-11-12 10:51 ` Mykyta Poturai
2025-11-16 11:43 ` Julien Grall
2025-11-12 10:51 ` [PATCH v4 7/8] tools: Allow building xen-hptool without CONFIG_MIGRATE Mykyta Poturai
7 siblings, 1 reply; 20+ messages in thread
From: Mykyta Poturai @ 2025-11-12 10:51 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Mykyta Poturai, Andrew Cooper, Anthony PERARD, Michal Orzel,
Jan Beulich, Julien Grall, Roger Pau Monné,
Stefano Stabellini
Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
v3->v4:
* update configuration section
v2->v3:
* patch introduced
---
docs/misc/cpu-hotplug.txt | 51 +++++++++++++++++++++++++++++++++++++++
1 file changed, 51 insertions(+)
create mode 100644 docs/misc/cpu-hotplug.txt
diff --git a/docs/misc/cpu-hotplug.txt b/docs/misc/cpu-hotplug.txt
new file mode 100644
index 0000000000..1fbad0ecf7
--- /dev/null
+++ b/docs/misc/cpu-hotplug.txt
@@ -0,0 +1,51 @@
+CPU Hotplug
+===========
+
+CPU hotplug is a feature that allows pCPU cores to be added to or removed from a
+running system without requiring a reboot. It is supported on x86 and Arm64
+architectures.
+
+Implementation Details
+----------------------
+
+CPU hotplug is implemented through the `XEN_SYSCTL_CPU_HOTPLUG_*` sysctl calls.
+The specific calls are:
+
+- `XEN_SYSCTL_CPU_HOTPLUG_ONLINE`: Brings a pCPU online
+- `XEN_SYSCTL_CPU_HOTPLUG_OFFLINE`: Takes a pCPU offline
+- `XEN_SYSCTL_CPU_HOTPLUG_SMT_ENABLE`: Enables SMT threads (x86 only)
+- `XEN_SYSCTL_CPU_HOTPLUG_SMT_DISABLE`: Disables SMT threads (x86 only)
+
+All cores can be disabled, assuming hardware support, except for core 0. Sysctl
+calls are routed to core 0 before doing any actual up/down operations on other
+cores.
+
+Configuration
+-------------
+
+Sysctl handlers are enabled unconditionally on x86 architecture. On Arm64,
+handlers are enabled by default when ITS, FFA, and TEE configs are disabled.
+Building of the userspace tool "hptool" is controlled by the "hptool" flag in
+the configure script. It is enabled by default and can be disabled with
+--disable-hptool command line option.
+
+Usage
+-----
+
+Disable core:
+
+$ xen-hptool cpu-offline 2
+Prepare to offline CPU 2
+(XEN) Removing cpu 2 from runqueue 0
+CPU 2 offlined successfully
+
+Enable core:
+
+$ xen-hptool cpu-online 2
+Prepare to online CPU 2
+(XEN) Bringing up CPU2
+(XEN) GICv3: CPU2: Found redistributor in region 0 @00000a004005c000
+(XEN) CPU2: Guest atomics will try 1 times before pausing the domain
+(XEN) CPU 2 booted.
+(XEN) Adding cpu 2 to runqueue 0
+CPU 2 onlined successfully
--
2.51.2
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [PATCH v4 8/8] docs: Document CPU hotplug
2025-11-12 10:51 ` [PATCH v4 8/8] docs: Document CPU hotplug Mykyta Poturai
@ 2025-11-16 11:43 ` Julien Grall
0 siblings, 0 replies; 20+ messages in thread
From: Julien Grall @ 2025-11-16 11:43 UTC (permalink / raw)
To: Mykyta Poturai, xen-devel@lists.xenproject.org
Cc: Andrew Cooper, Anthony PERARD, Michal Orzel, Jan Beulich,
Roger Pau Monné, Stefano Stabellini
Hi,
On 12/11/2025 10:51, Mykyta Poturai wrote:
> Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
>
> v3->v4:
> * update configuration section
>
> v2->v3:
> * patch introduced
> ---
> docs/misc/cpu-hotplug.txt | 51 +++++++++++++++++++++++++++++++++++++++
> 1 file changed, 51 insertions(+)
> create mode 100644 docs/misc/cpu-hotplug.txt
>
> diff --git a/docs/misc/cpu-hotplug.txt b/docs/misc/cpu-hotplug.txt
> new file mode 100644
> index 0000000000..1fbad0ecf7
> --- /dev/null
> +++ b/docs/misc/cpu-hotplug.txt
> @@ -0,0 +1,51 @@
> +CPU Hotplug
> +===========
> +
> +CPU hotplug is a feature that allows pCPU cores to be added to or removed from a
> +running system without requiring a reboot. It is supported on x86 and Arm64
> +architectures.
"supported" usually implies that we will backport fixes and issues XSA.
Is this what you intended to say?
Asking because SUPPORT.md has the following:
### ACPI CPU Hotplug
Status, x86: Experimental
There is also no mention for Arm in SUPPORT.md. As you enable the
feature, we should clarify what's its state.
> +
> +Implementation Details
> +----------------------
> +
> +CPU hotplug is implemented through the `XEN_SYSCTL_CPU_HOTPLUG_*` sysctl calls.
> +The specific calls are:
> +
> +- `XEN_SYSCTL_CPU_HOTPLUG_ONLINE`: Brings a pCPU online
> +- `XEN_SYSCTL_CPU_HOTPLUG_OFFLINE`: Takes a pCPU offline
> +- `XEN_SYSCTL_CPU_HOTPLUG_SMT_ENABLE`: Enables SMT threads (x86 only)
> +- `XEN_SYSCTL_CPU_HOTPLUG_SMT_DISABLE`: Disables SMT threads (x86 only)
> +
> +All cores can be disabled, assuming hardware support, except for core 0. Sysctl
> +calls are routed to core 0 before doing any actual up/down operations on other
> +cores.
> +
> +Configuration
> +-------------
> +
> +Sysctl handlers are enabled unconditionally on x86 architecture. On Arm64,
> +handlers are enabled by default when ITS, FFA, and TEE configs are disabled.
> +Building of the userspace tool "hptool" is controlled by the "hptool" flag in
> +the configure script. It is enabled by default and can be disabled with
> +--disable-hptool command line option.
> +
> +Usage
> +-----
> +
> +Disable core:
> +
> +$ xen-hptool cpu-offline 2
> +Prepare to offline CPU 2
> +(XEN) Removing cpu 2 from runqueue 0
> +CPU 2 offlined successfully
> +
> +Enable core:
> +
> +$ xen-hptool cpu-online 2
> +Prepare to online CPU 2
> +(XEN) Bringing up CPU2
> +(XEN) GICv3: CPU2: Found redistributor in region 0 @00000a004005c000
> +(XEN) CPU2: Guest atomics will try 1 times before pausing the domain
> +(XEN) CPU 2 booted.
> +(XEN) Adding cpu 2 to runqueue 0
> +CPU 2 onlined successfully
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH v4 7/8] tools: Allow building xen-hptool without CONFIG_MIGRATE
2025-11-12 10:51 [PATCH v4 0/8] Implement CPU hotplug on Arm Mykyta Poturai
` (6 preceding siblings ...)
2025-11-12 10:51 ` [PATCH v4 8/8] docs: Document CPU hotplug Mykyta Poturai
@ 2025-11-12 10:51 ` Mykyta Poturai
2025-11-28 17:42 ` Anthony PERARD
7 siblings, 1 reply; 20+ messages in thread
From: Mykyta Poturai @ 2025-11-12 10:51 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Mykyta Poturai, Anthony PERARD, Juergen Gross
With CPU hotplug sysctls implemented on Arm it becomes useful to have a
tool for calling them. Introduce a new congifure option "hptool" to
allow building hptool separately from other migration tools, and enable
it by default.
Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
v3->v4:
* no changes
v2->v3:
* no changes
v1->v2:
* switch to configure from legacy config
---
config/Tools.mk.in | 1 +
tools/configure | 30 ++++++++++++++++++++++++++++++
tools/configure.ac | 1 +
tools/libs/guest/Makefile.common | 4 ++++
tools/misc/Makefile | 2 +-
5 files changed, 37 insertions(+), 1 deletion(-)
mode change 100755 => 100644 tools/configure
diff --git a/config/Tools.mk.in b/config/Tools.mk.in
index 0037ad5a64..d5855ca090 100644
--- a/config/Tools.mk.in
+++ b/config/Tools.mk.in
@@ -49,6 +49,7 @@ CONFIG_LIBNL := @libnl@
CONFIG_GOLANG := @golang@
CONFIG_PYGRUB := @pygrub@
CONFIG_LIBFSIMAGE := @libfsimage@
+CONFIG_HPTOOL := @hptool@
CONFIG_SYSTEMD := @systemd@
XEN_SYSTEMD_DIR := @SYSTEMD_DIR@
diff --git a/tools/configure b/tools/configure
old mode 100755
new mode 100644
index 3111f5688c..8a86582ece
--- a/tools/configure
+++ b/tools/configure
@@ -731,6 +731,7 @@ LD86
AS86
ipxe
LINUX_BACKEND_MODULES
+hptool
pygrub
golang
seabios
@@ -837,6 +838,7 @@ enable_ovmf
enable_seabios
enable_golang
enable_pygrub
+enable_hptool
with_linux_backend_modules
enable_ipxe
with_system_ipxe
@@ -1524,6 +1526,7 @@ Optional Features:
--disable-seabios Disable SeaBIOS (default is ENABLED)
--disable-golang Disable Go tools (default is ENABLED)
--disable-pygrub Disable pygrub (default is ENABLED)
+ --disable-hptool Disable hptool (default is ENABLED)
--enable-ipxe Enable in-tree IPXE, (DEFAULT is off, see also
--with-system-ipxe)
--enable-rombios Enable ROMBIOS, (DEFAULT is on if ipxe is enabled,
@@ -4816,6 +4819,33 @@ pygrub=$ax_cv_pygrub
+# Check whether --enable-hptool was given.
+if test ${enable_hptool+y}
+then :
+ enableval=$enable_hptool;
+fi
+
+
+if test "x$enable_hptool" = "xno"
+then :
+
+ ax_cv_hptool="n"
+
+elif test "x$enable_hptool" = "xyes"
+then :
+
+ ax_cv_hptool="y"
+
+elif test -z $ax_cv_hptool
+then :
+
+ ax_cv_hptool="y"
+
+fi
+hptool=$ax_cv_hptool
+
+
+
# Check whether --with-linux-backend-modules was given.
if test ${with_linux_backend_modules+y}
diff --git a/tools/configure.ac b/tools/configure.ac
index 285b4ea128..28a0c095c2 100644
--- a/tools/configure.ac
+++ b/tools/configure.ac
@@ -90,6 +90,7 @@ AX_ARG_DEFAULT_DISABLE([ovmf], [Enable OVMF])
AX_ARG_DEFAULT_ENABLE([seabios], [Disable SeaBIOS])
AX_ARG_DEFAULT_ENABLE([golang], [Disable Go tools])
AX_ARG_DEFAULT_ENABLE([pygrub], [Disable pygrub])
+AX_ARG_DEFAULT_ENABLE([hptool], [Disable hptool])
AC_ARG_WITH([linux-backend-modules],
AS_HELP_STRING([--with-linux-backend-modules="mod1 mod2"],
diff --git a/tools/libs/guest/Makefile.common b/tools/libs/guest/Makefile.common
index a026a2f662..774b1d5392 100644
--- a/tools/libs/guest/Makefile.common
+++ b/tools/libs/guest/Makefile.common
@@ -25,6 +25,10 @@ OBJS-y += xg_core.o
OBJS-$(CONFIG_X86) += xg_core_x86.o
OBJS-$(CONFIG_ARM) += xg_core_arm.o
+ifneq (,$(filter y,$(CONFIG_MIGRATE)$(CONFIG_HPTOOL)))
+OBJS-y += xg_offline_page.o
+endif
+
vpath %.c ../../../xen/common/libelf
LIBELF_OBJS += libelf-tools.o libelf-loader.o
diff --git a/tools/misc/Makefile b/tools/misc/Makefile
index c26e544e83..f783f16ae6 100644
--- a/tools/misc/Makefile
+++ b/tools/misc/Makefile
@@ -16,7 +16,7 @@ INSTALL_BIN += xencov_split
INSTALL_BIN += $(INSTALL_BIN-y)
# Everything to be installed in regular sbin/
-INSTALL_SBIN-$(CONFIG_MIGRATE) += xen-hptool
+INSTALL_SBIN-$(CONFIG_HPTOOL) += xen-hptool
INSTALL_SBIN-$(CONFIG_X86) += xen-hvmcrash
INSTALL_SBIN-$(CONFIG_X86) += xen-hvmctx
INSTALL_SBIN-$(CONFIG_X86) += xen-lowmemd
--
2.51.2
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [PATCH v4 7/8] tools: Allow building xen-hptool without CONFIG_MIGRATE
2025-11-12 10:51 ` [PATCH v4 7/8] tools: Allow building xen-hptool without CONFIG_MIGRATE Mykyta Poturai
@ 2025-11-28 17:42 ` Anthony PERARD
0 siblings, 0 replies; 20+ messages in thread
From: Anthony PERARD @ 2025-11-28 17:42 UTC (permalink / raw)
To: Mykyta Poturai
Cc: xen-devel@lists.xenproject.org, Anthony PERARD, Juergen Gross
On Wed, Nov 12, 2025 at 10:51:49AM +0000, Mykyta Poturai wrote:
> With CPU hotplug sysctls implemented on Arm it becomes useful to have a
> tool for calling them. Introduce a new congifure option "hptool" to
> allow building hptool separately from other migration tools, and enable
> it by default.
>
> Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
>
You should had a line with just "---" around here, right after the
Signed-off-by, and before the changelog. That way, when we commit the
patch, git will automatically discard this patch changelog. Notice that
git do just that before adding stats about file changes.
> v3->v4:
> * no changes
>
> v2->v3:
> * no changes
>
> v1->v2:
> * switch to configure from legacy config
> ---
> config/Tools.mk.in | 1 +
> tools/configure | 30 ++++++++++++++++++++++++++++++
> tools/configure.ac | 1 +
> tools/libs/guest/Makefile.common | 4 ++++
> tools/misc/Makefile | 2 +-
> 5 files changed, 37 insertions(+), 1 deletion(-)
> mode change 100755 => 100644 tools/configure
>
> diff --git a/config/Tools.mk.in b/config/Tools.mk.in
> index 0037ad5a64..d5855ca090 100644
> --- a/config/Tools.mk.in
> +++ b/config/Tools.mk.in
> @@ -49,6 +49,7 @@ CONFIG_LIBNL := @libnl@
> CONFIG_GOLANG := @golang@
> CONFIG_PYGRUB := @pygrub@
> CONFIG_LIBFSIMAGE := @libfsimage@
> +CONFIG_HPTOOL := @hptool@
>
> CONFIG_SYSTEMD := @systemd@
> XEN_SYSTEMD_DIR := @SYSTEMD_DIR@
> diff --git a/tools/configure.ac b/tools/configure.ac
> index 285b4ea128..28a0c095c2 100644
> --- a/tools/configure.ac
> +++ b/tools/configure.ac
> @@ -90,6 +90,7 @@ AX_ARG_DEFAULT_DISABLE([ovmf], [Enable OVMF])
> AX_ARG_DEFAULT_ENABLE([seabios], [Disable SeaBIOS])
> AX_ARG_DEFAULT_ENABLE([golang], [Disable Go tools])
> AX_ARG_DEFAULT_ENABLE([pygrub], [Disable pygrub])
> +AX_ARG_DEFAULT_ENABLE([hptool], [Disable hptool])
>
> AC_ARG_WITH([linux-backend-modules],
> AS_HELP_STRING([--with-linux-backend-modules="mod1 mod2"],
> diff --git a/tools/libs/guest/Makefile.common b/tools/libs/guest/Makefile.common
> index a026a2f662..774b1d5392 100644
> --- a/tools/libs/guest/Makefile.common
> +++ b/tools/libs/guest/Makefile.common
> @@ -25,6 +25,10 @@ OBJS-y += xg_core.o
> OBJS-$(CONFIG_X86) += xg_core_x86.o
> OBJS-$(CONFIG_ARM) += xg_core_arm.o
>
> +ifneq (,$(filter y,$(CONFIG_MIGRATE)$(CONFIG_HPTOOL)))
How is this supposed to work?
> +OBJS-y += xg_offline_page.o
> +endif
> +
> vpath %.c ../../../xen/common/libelf
>
> LIBELF_OBJS += libelf-tools.o libelf-loader.o
> diff --git a/tools/misc/Makefile b/tools/misc/Makefile
> index c26e544e83..f783f16ae6 100644
> --- a/tools/misc/Makefile
> +++ b/tools/misc/Makefile
> @@ -16,7 +16,7 @@ INSTALL_BIN += xencov_split
> INSTALL_BIN += $(INSTALL_BIN-y)
>
> # Everything to be installed in regular sbin/
> -INSTALL_SBIN-$(CONFIG_MIGRATE) += xen-hptool
> +INSTALL_SBIN-$(CONFIG_HPTOOL) += xen-hptool
So, I've look at the history as to why "xen-hptool" was only compiled
for CONFIG_MIGRATE, and it was because "xc_offline_page.c" didn't build
on ia64, commit 6b0b6e01b967 ("tools: disable xen-hptool on ia64").
And hiding xc_offline_page.c behind CONFIG_MIGRATE was probably just a
short-cut, 310311cd8863 ("libxc: fix link error on ia64") because I
guess the needed functions where behind this value at the time.
("xc_offline_page.c" is now "xg_offline_page.c")
This patch now build xen-hptool, and "xg_offline_page.c" by default. Does
this actually build on every architecture? We don't really have
per-binary configuration option, and having a library (libxenguest)
which get different functionality depending on which tool we want to
build doesn't seems to be the right thing to do.
If xg_offline_page.c can now be built on every architecture, how about
building it by default? Then, we can always build xen-hptool.
Thanks,
--
Anthony PERARD
^ permalink raw reply [flat|nested] 20+ messages in thread