[Qemu-devel] [PATCH v2] spapr: add "stop-self" RTAS call required to support hot CPU unplug

[Qemu-devel] [PATCH v2] spapr: add "stop-self" RTAS call required to support hot CPU unplug

[Alexey Kardashevskiy]

PAPR+ requires two RTAS calls to be supported by the hypervisor in
order to allow hotplugging VCPUs from the guest. The "start-cpu" RTAS
call was already there but "stop-self" was not.

This adds the "stop-self" RTAS call.

Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
Changes:
v2:
* exit_request flag change replaced with more correct cpu_exit() call
* fixed commit message, "spapr: support CPU hotplug"
---
 hw/ppc/spapr_rtas.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
index 394ce05..b906294 100644
--- a/hw/ppc/spapr_rtas.c
+++ b/hw/ppc/spapr_rtas.c
@@ -202,6 +202,17 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPREnvironment *spapr,
     rtas_st(rets, 0, -3);
 }
 
+static void rtas_stop_self(PowerPCCPU *cpu, sPAPREnvironment *spapr,
+                           uint32_t token, uint32_t nargs,
+                           target_ulong args,
+                           uint32_t nret, target_ulong rets)
+{
+    CPUState *cs = CPU(cpu);
+
+    cs->halted = 1;
+    cpu_exit(cs);
+}
+
 static struct rtas_call {
     const char *name;
     spapr_rtas_fn fn;
@@ -322,6 +333,7 @@ static void core_rtas_register_types(void)
     spapr_rtas_register("query-cpu-stopped-state",
                         rtas_query_cpu_stopped_state);
     spapr_rtas_register("start-cpu", rtas_start_cpu);
+    spapr_rtas_register("stop-self", rtas_stop_self);
 }
 
 type_init(core_rtas_register_types)
-- 
1.8.4.rc4

Re: [Qemu-devel] [PATCH v2] spapr: add "stop-self" RTAS call required to support hot CPU unplug

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1966 bytes --]

On Wed, Aug 28, 2013 at 07:32:51PM +1000, Alexey Kardashevskiy wrote:
> PAPR+ requires two RTAS calls to be supported by the hypervisor in
> order to allow hotplugging VCPUs from the guest. The "start-cpu" RTAS
> call was already there but "stop-self" was not.
> 
> This adds the "stop-self" RTAS call.
> 
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> ---
> Changes:
> v2:
> * exit_request flag change replaced with more correct cpu_exit() call
> * fixed commit message, "spapr: support CPU hotplug"
> ---
>  hw/ppc/spapr_rtas.c | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
> index 394ce05..b906294 100644
> --- a/hw/ppc/spapr_rtas.c
> +++ b/hw/ppc/spapr_rtas.c
> @@ -202,6 +202,17 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPREnvironment *spapr,
>      rtas_st(rets, 0, -3);
>  }
>  
> +static void rtas_stop_self(PowerPCCPU *cpu, sPAPREnvironment *spapr,
> +                           uint32_t token, uint32_t nargs,
> +                           target_ulong args,
> +                           uint32_t nret, target_ulong rets)
> +{
> +    CPUState *cs = CPU(cpu);
> +
> +    cs->halted = 1;

Uh.. I think you still need the msr = 0, or an interrupt could wake
the cpu up again.

> +    cpu_exit(cs);
> +}
> +
>  static struct rtas_call {
>      const char *name;
>      spapr_rtas_fn fn;
> @@ -322,6 +333,7 @@ static void core_rtas_register_types(void)
>      spapr_rtas_register("query-cpu-stopped-state",
>                          rtas_query_cpu_stopped_state);
>      spapr_rtas_register("start-cpu", rtas_start_cpu);
> +    spapr_rtas_register("stop-self", rtas_stop_self);
>  }
>  
>  type_init(core_rtas_register_types)

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [Qemu-devel] [PATCH v2] spapr: add "stop-self" RTAS call required to support hot CPU unplug

[Alexey Kardashevskiy]

On 08/28/2013 09:42 PM, David Gibson wrote:
> On Wed, Aug 28, 2013 at 07:32:51PM +1000, Alexey Kardashevskiy wrote:
>> PAPR+ requires two RTAS calls to be supported by the hypervisor in
>> order to allow hotplugging VCPUs from the guest. The "start-cpu" RTAS
>> call was already there but "stop-self" was not.
>>
>> This adds the "stop-self" RTAS call.
>>
>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>> ---
>> Changes:
>> v2:
>> * exit_request flag change replaced with more correct cpu_exit() call
>> * fixed commit message, "spapr: support CPU hotplug"
>> ---
>>  hw/ppc/spapr_rtas.c | 12 ++++++++++++
>>  1 file changed, 12 insertions(+)
>>
>> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
>> index 394ce05..b906294 100644
>> --- a/hw/ppc/spapr_rtas.c
>> +++ b/hw/ppc/spapr_rtas.c
>> @@ -202,6 +202,17 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPREnvironment *spapr,
>>      rtas_st(rets, 0, -3);
>>  }
>>  
>> +static void rtas_stop_self(PowerPCCPU *cpu, sPAPREnvironment *spapr,
>> +                           uint32_t token, uint32_t nargs,
>> +                           target_ulong args,
>> +                           uint32_t nret, target_ulong rets)
>> +{
>> +    CPUState *cs = CPU(cpu);
>> +
>> +    cs->halted = 1;
> 
> Uh.. I think you still need the msr = 0, or an interrupt could wake
> the cpu up again.


Tried with KVM and TCG - nothing wakes up the stopped CPU :-/
After a CPU stopped, the guest calls H_CPPR and disables interrupts on the
specific CPU (or "server") on the XICS level. And decrementer interrupt
does not wake up the CPU.

What is strange is that with TCG (but not with KVM) I see a warning from
the guest at WARN_ON(get_preferred_offline_state(cpu) !=
CPU_STATE_OFFLINE); (pseries_mach_cpu_die() function) but only when I stop
a CPU (any!) first time in the guest.

Specifically, I can do "echo 0 > ...cpuX/online" and  "echo 1 >
...cpuX/online" many times during the same boot but only the first stop at
all generates a warning (the state is 2 which is CPU_STATE_ONLINE).



>> +    cpu_exit(cs);
>> +}
>> +
>>  static struct rtas_call {
>>      const char *name;
>>      spapr_rtas_fn fn;
>> @@ -322,6 +333,7 @@ static void core_rtas_register_types(void)
>>      spapr_rtas_register("query-cpu-stopped-state",
>>                          rtas_query_cpu_stopped_state);
>>      spapr_rtas_register("start-cpu", rtas_start_cpu);
>> +    spapr_rtas_register("stop-self", rtas_stop_self);
>>  }
>>  
>>  type_init(core_rtas_register_types)
> 


-- 
Alexey

Re: [Qemu-devel] [PATCH v2] spapr: add "stop-self" RTAS call required to support hot CPU unplug

[Alexander Graf]

On 29.08.2013, at 11:04, Alexey Kardashevskiy wrote:

> On 08/28/2013 09:42 PM, David Gibson wrote:
>> On Wed, Aug 28, 2013 at 07:32:51PM +1000, Alexey Kardashevskiy wrote:
>>> PAPR+ requires two RTAS calls to be supported by the hypervisor in
>>> order to allow hotplugging VCPUs from the guest. The "start-cpu" RTAS
>>> call was already there but "stop-self" was not.
>>> 
>>> This adds the "stop-self" RTAS call.
>>> 
>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>>> ---
>>> Changes:
>>> v2:
>>> * exit_request flag change replaced with more correct cpu_exit() call
>>> * fixed commit message, "spapr: support CPU hotplug"
>>> ---
>>> hw/ppc/spapr_rtas.c | 12 ++++++++++++
>>> 1 file changed, 12 insertions(+)
>>> 
>>> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
>>> index 394ce05..b906294 100644
>>> --- a/hw/ppc/spapr_rtas.c
>>> +++ b/hw/ppc/spapr_rtas.c
>>> @@ -202,6 +202,17 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPREnvironment *spapr,
>>>     rtas_st(rets, 0, -3);
>>> }
>>> 
>>> +static void rtas_stop_self(PowerPCCPU *cpu, sPAPREnvironment *spapr,
>>> +                           uint32_t token, uint32_t nargs,
>>> +                           target_ulong args,
>>> +                           uint32_t nret, target_ulong rets)
>>> +{
>>> +    CPUState *cs = CPU(cpu);
>>> +
>>> +    cs->halted = 1;
>> 
>> Uh.. I think you still need the msr = 0, or an interrupt could wake
>> the cpu up again.
> 
> 
> Tried with KVM and TCG - nothing wakes up the stopped CPU :-/
> After a CPU stopped, the guest calls H_CPPR and disables interrupts on the
> specific CPU (or "server") on the XICS level. And decrementer interrupt
> does not wake up the CPU.

Why not? With TCG it should :).


Alex