[Qemu-devel] [PATCH] spapr: support CPU hotplug

[Qemu-devel] [PATCH] spapr: support CPU hotplug

[Alexey Kardashevskiy]

PAPR+ requires two RTAS calls to be supported by the hypervisor in
order to allow hotplugging VCPUs from the guest. The "start-cpu" RTAS
call was already there but "stop-self" was not.

This adds the "stop-self" RTAS call.

Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
 hw/ppc/spapr_rtas.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
index 394ce05..8a4cfa0 100644
--- a/hw/ppc/spapr_rtas.c
+++ b/hw/ppc/spapr_rtas.c
@@ -202,6 +202,19 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPREnvironment *spapr,
     rtas_st(rets, 0, -3);
 }
 
+static void rtas_stop_self(PowerPCCPU *cpu, sPAPREnvironment *spapr,
+                           uint32_t token, uint32_t nargs,
+                           target_ulong args,
+                           uint32_t nret, target_ulong rets)
+{
+    CPUState *cs = CPU(cpu);
+    CPUPPCState *env = &cpu->env;
+
+    cs->halted = 1;
+    env->msr = 0;
+    cs->exit_request = 1;
+}
+
 static struct rtas_call {
     const char *name;
     spapr_rtas_fn fn;
@@ -322,6 +335,7 @@ static void core_rtas_register_types(void)
     spapr_rtas_register("query-cpu-stopped-state",
                         rtas_query_cpu_stopped_state);
     spapr_rtas_register("start-cpu", rtas_start_cpu);
+    spapr_rtas_register("stop-self", rtas_stop_self);
 }
 
 type_init(core_rtas_register_types)
-- 
1.8.4.rc4

Re: [Qemu-devel] [PATCH] spapr: support CPU hotplug

[Alexander Graf]

On 23.08.2013, at 13:30, Alexey Kardashevskiy wrote:

> PAPR+ requires two RTAS calls to be supported by the hypervisor in
> order to allow hotplugging VCPUs from the guest. The "start-cpu" RTAS
> call was already there but "stop-self" was not.
> 
> This adds the "stop-self" RTAS call.
> 
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> ---
> hw/ppc/spapr_rtas.c | 14 ++++++++++++++
> 1 file changed, 14 insertions(+)
> 
> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
> index 394ce05..8a4cfa0 100644
> --- a/hw/ppc/spapr_rtas.c
> +++ b/hw/ppc/spapr_rtas.c
> @@ -202,6 +202,19 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPREnvironment *spapr,
>     rtas_st(rets, 0, -3);
> }
> 
> +static void rtas_stop_self(PowerPCCPU *cpu, sPAPREnvironment *spapr,
> +                           uint32_t token, uint32_t nargs,
> +                           target_ulong args,
> +                           uint32_t nret, target_ulong rets)
> +{
> +    CPUState *cs = CPU(cpu);
> +    CPUPPCState *env = &cpu->env;
> +
> +    cs->halted = 1;
> +    env->msr = 0;

So this is here to make sure we don't accidentally get out of halted state by an interrupt on that vcpu. Could you please somehow make that part obvious? Either by adding a comment or by only explicitly masking DEC and EE and a comment :).

> +    cs->exit_request = 1;

This should probably be qemu_cpu_kick_self().


Alex

Re: [Qemu-devel] [PATCH] spapr: support CPU hotplug

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1968 bytes --]

On Mon, Aug 26, 2013 at 02:24:49PM +0200, Alexander Graf wrote:
> 
> On 23.08.2013, at 13:30, Alexey Kardashevskiy wrote:
> 
> > PAPR+ requires two RTAS calls to be supported by the hypervisor in
> > order to allow hotplugging VCPUs from the guest. The "start-cpu" RTAS
> > call was already there but "stop-self" was not.
> > 
> > This adds the "stop-self" RTAS call.
> > 
> > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> > ---
> > hw/ppc/spapr_rtas.c | 14 ++++++++++++++
> > 1 file changed, 14 insertions(+)
> > 
> > diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
> > index 394ce05..8a4cfa0 100644
> > --- a/hw/ppc/spapr_rtas.c
> > +++ b/hw/ppc/spapr_rtas.c
> > @@ -202,6 +202,19 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPREnvironment *spapr,
> >     rtas_st(rets, 0, -3);
> > }
> > 
> > +static void rtas_stop_self(PowerPCCPU *cpu, sPAPREnvironment *spapr,
> > +                           uint32_t token, uint32_t nargs,
> > +                           target_ulong args,
> > +                           uint32_t nret, target_ulong rets)
> > +{
> > +    CPUState *cs = CPU(cpu);
> > +    CPUPPCState *env = &cpu->env;
> > +
> > +    cs->halted = 1;
> > +    env->msr = 0;
> 
> So this is here to make sure we don't accidentally get out of halted state by an interrupt on that vcpu. Could you please somehow make that part obvious? Either by adding a comment or by only explicitly masking DEC and EE and a comment :).
> 
> > +    cs->exit_request = 1;
> 
> This should probably be qemu_cpu_kick_self().

Uh, no, I don't think so.  This is there purely to make sure we exit
the inner loop, and actually test cpu_can_run() which will test
halted.  AFAICT qemu_cpu_kick_self() won't do anything similar.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [Qemu-devel] [PATCH] spapr: support CPU hotplug

[Alexey Kardashevskiy]

On 08/27/2013 01:29 AM, David Gibson wrote:
> On Mon, Aug 26, 2013 at 02:24:49PM +0200, Alexander Graf wrote:
>>
>> On 23.08.2013, at 13:30, Alexey Kardashevskiy wrote:
>>
>>> PAPR+ requires two RTAS calls to be supported by the hypervisor in
>>> order to allow hotplugging VCPUs from the guest. The "start-cpu" RTAS
>>> call was already there but "stop-self" was not.
>>>
>>> This adds the "stop-self" RTAS call.
>>>
>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>>> ---
>>> hw/ppc/spapr_rtas.c | 14 ++++++++++++++
>>> 1 file changed, 14 insertions(+)
>>>
>>> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
>>> index 394ce05..8a4cfa0 100644
>>> --- a/hw/ppc/spapr_rtas.c
>>> +++ b/hw/ppc/spapr_rtas.c
>>> @@ -202,6 +202,19 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPREnvironment *spapr,
>>>     rtas_st(rets, 0, -3);
>>> }
>>>
>>> +static void rtas_stop_self(PowerPCCPU *cpu, sPAPREnvironment *spapr,
>>> +                           uint32_t token, uint32_t nargs,
>>> +                           target_ulong args,
>>> +                           uint32_t nret, target_ulong rets)
>>> +{
>>> +    CPUState *cs = CPU(cpu);
>>> +    CPUPPCState *env = &cpu->env;
>>> +
>>> +    cs->halted = 1;
>>> +    env->msr = 0;
>>
>> So this is here to make sure we don't accidentally get out of halted state by an interrupt on that vcpu. Could you please somehow make that part obvious? Either by adding a comment or by only explicitly masking DEC and EE and a comment :).
>>
>>> +    cs->exit_request = 1;
>>
>> This should probably be qemu_cpu_kick_self().
> 
> Uh, no, I don't think so.  This is there purely to make sure we exit
> the inner loop, and actually test cpu_can_run() which will test
> halted.  AFAICT qemu_cpu_kick_self() won't do anything similar.


rtas_stop_self() eventually returns to kvm_cpu_exec() which calls
qemu_cpu_kick_self() and resets cs->exit_request before return so I do not
really see the difference in behaviour. And actually both ways CPU stops in
exactly the same way. What do I miss?



-- 
Alexey

Re: [Qemu-devel] [PATCH] spapr: support CPU hotplug

[Paolo Bonzini]

Il 27/08/2013 08:37, Alexey Kardashevskiy ha scritto:
>>> So this is here to make sure we don't accidentally get out of halted state by an interrupt on that vcpu. Could you please somehow make that part obvious? Either by adding a comment or by only explicitly masking DEC and EE and a comment :).
>>>
>>>> +    cs->exit_request = 1;
>>>
>>> This should probably be qemu_cpu_kick_self().
>>
>> Uh, no, I don't think so.  This is there purely to make sure we exit
>> the inner loop, and actually test cpu_can_run() which will test
>> halted.  AFAICT qemu_cpu_kick_self() won't do anything similar.
> 
> rtas_stop_self() eventually returns to kvm_cpu_exec() which calls
> qemu_cpu_kick_self() and resets cs->exit_request before return so I do not
> really see the difference in behaviour. And actually both ways CPU stops in
> exactly the same way. What do I miss?

What about TCG?

Paolo

Re: [Qemu-devel] [PATCH] spapr: support CPU hotplug

[Alexey Kardashevskiy]

On 08/27/2013 05:02 PM, Paolo Bonzini wrote:
> Il 27/08/2013 08:37, Alexey Kardashevskiy ha scritto:
>>>> So this is here to make sure we don't accidentally get out of halted state by an interrupt on that vcpu. Could you please somehow make that part obvious? Either by adding a comment or by only explicitly masking DEC and EE and a comment :).
>>>>
>>>>> +    cs->exit_request = 1;
>>>>
>>>> This should probably be qemu_cpu_kick_self().
>>>
>>> Uh, no, I don't think so.  This is there purely to make sure we exit
>>> the inner loop, and actually test cpu_can_run() which will test
>>> halted.  AFAICT qemu_cpu_kick_self() won't do anything similar.
>>
>> rtas_stop_self() eventually returns to kvm_cpu_exec() which calls
>> qemu_cpu_kick_self() and resets cs->exit_request before return so I do not
>> really see the difference in behaviour. And actually both ways CPU stops in
>> exactly the same way. What do I miss?
> 
> What about TCG?

Oh. Right. TCG :(

qemu_cpu_kick_self() crashes the guest and cs->exit_request works fine.

Why? Both should work? What is the expected behavior here? Thanks.


-- 
Alexey

Re: [Qemu-devel] [PATCH] spapr: support CPU hotplug

[Alexander Graf]

On 27.08.2013, at 09:41, Alexey Kardashevskiy wrote:

> On 08/27/2013 05:02 PM, Paolo Bonzini wrote:
>> Il 27/08/2013 08:37, Alexey Kardashevskiy ha scritto:
>>>>> So this is here to make sure we don't accidentally get out of halted state by an interrupt on that vcpu. Could you please somehow make that part obvious? Either by adding a comment or by only explicitly masking DEC and EE and a comment :).
>>>>> 
>>>>>> +    cs->exit_request = 1;
>>>>> 
>>>>> This should probably be qemu_cpu_kick_self().
>>>> 
>>>> Uh, no, I don't think so.  This is there purely to make sure we exit
>>>> the inner loop, and actually test cpu_can_run() which will test
>>>> halted.  AFAICT qemu_cpu_kick_self() won't do anything similar.
>>> 
>>> rtas_stop_self() eventually returns to kvm_cpu_exec() which calls
>>> qemu_cpu_kick_self() and resets cs->exit_request before return so I do not
>>> really see the difference in behaviour. And actually both ways CPU stops in
>>> exactly the same way. What do I miss?
>> 
>> What about TCG?
> 
> Oh. Right. TCG :(
> 
> qemu_cpu_kick_self() crashes the guest and cs->exit_request works fine.
> 
> Why? Both should work? What is the expected behavior here? Thanks.

Hrm. To me exit_request always was an internal piece of state that the inner loop uses to find out whether to exit, but not something we should randomly set from a device (and hypercalls / rtas calls are very similar to devices). So I would like to not have any code in hw/ that modifies it.

However, we need the functionality of breaking out of the main loop, I agree.

Maybe what you are really looking for is cpu_interrupt(CPU_INTERRUPT_HALT). That sets halted = 1 and exits the main loop, because it's an interrupt.


Alex

Re: [Qemu-devel] [PATCH] spapr: support CPU hotplug

[Alexey Kardashevskiy]

On 08/27/2013 06:54 PM, Alexander Graf wrote:
> 
> On 27.08.2013, at 09:41, Alexey Kardashevskiy wrote:
> 
>> On 08/27/2013 05:02 PM, Paolo Bonzini wrote:
>>> Il 27/08/2013 08:37, Alexey Kardashevskiy ha scritto:
>>>>>> So this is here to make sure we don't accidentally get out of halted state by an interrupt on that vcpu. Could you please somehow make that part obvious? Either by adding a comment or by only explicitly masking DEC and EE and a comment :).
>>>>>>
>>>>>>> +    cs->exit_request = 1;
>>>>>>
>>>>>> This should probably be qemu_cpu_kick_self().
>>>>>
>>>>> Uh, no, I don't think so.  This is there purely to make sure we exit
>>>>> the inner loop, and actually test cpu_can_run() which will test
>>>>> halted.  AFAICT qemu_cpu_kick_self() won't do anything similar.
>>>>
>>>> rtas_stop_self() eventually returns to kvm_cpu_exec() which calls
>>>> qemu_cpu_kick_self() and resets cs->exit_request before return so I do not
>>>> really see the difference in behaviour. And actually both ways CPU stops in
>>>> exactly the same way. What do I miss?
>>>
>>> What about TCG?
>>
>> Oh. Right. TCG :(
>>
>> qemu_cpu_kick_self() crashes the guest and cs->exit_request works fine.
>>
>> Why? Both should work? What is the expected behavior here? Thanks.
> 
> Hrm. To me exit_request always was an internal piece of state that the inner loop uses to find out whether to exit, but not something we should randomly set from a device (and hypercalls / rtas calls are very similar to devices). So I would like to not have any code in hw/ that modifies it.
> 
> However, we need the functionality of breaking out of the main loop, I agree.

> Maybe what you are really looking for is
> cpu_interrupt(CPU_INTERRUPT_HALT). That sets halted = 1 and exits the
> main loop, because it's an interrupt.

cpu_interrupt(CPU_INTERRUPT_HALT) works fine for TCG but does not for KVM
(the rtas call returns to the guest and it reports BUG).


-- 
Alexey

Re: [Qemu-devel] [PATCH] spapr: support CPU hotplug

[Alexander Graf]

On 27.08.2013, at 11:03, Alexey Kardashevskiy wrote:

> On 08/27/2013 06:54 PM, Alexander Graf wrote:
>> 
>> On 27.08.2013, at 09:41, Alexey Kardashevskiy wrote:
>> 
>>> On 08/27/2013 05:02 PM, Paolo Bonzini wrote:
>>>> Il 27/08/2013 08:37, Alexey Kardashevskiy ha scritto:
>>>>>>> So this is here to make sure we don't accidentally get out of halted state by an interrupt on that vcpu. Could you please somehow make that part obvious? Either by adding a comment or by only explicitly masking DEC and EE and a comment :).
>>>>>>> 
>>>>>>>> +    cs->exit_request = 1;
>>>>>>> 
>>>>>>> This should probably be qemu_cpu_kick_self().
>>>>>> 
>>>>>> Uh, no, I don't think so.  This is there purely to make sure we exit
>>>>>> the inner loop, and actually test cpu_can_run() which will test
>>>>>> halted.  AFAICT qemu_cpu_kick_self() won't do anything similar.
>>>>> 
>>>>> rtas_stop_self() eventually returns to kvm_cpu_exec() which calls
>>>>> qemu_cpu_kick_self() and resets cs->exit_request before return so I do not
>>>>> really see the difference in behaviour. And actually both ways CPU stops in
>>>>> exactly the same way. What do I miss?
>>>> 
>>>> What about TCG?
>>> 
>>> Oh. Right. TCG :(
>>> 
>>> qemu_cpu_kick_self() crashes the guest and cs->exit_request works fine.
>>> 
>>> Why? Both should work? What is the expected behavior here? Thanks.
>> 
>> Hrm. To me exit_request always was an internal piece of state that the inner loop uses to find out whether to exit, but not something we should randomly set from a device (and hypercalls / rtas calls are very similar to devices). So I would like to not have any code in hw/ that modifies it.
>> 
>> However, we need the functionality of breaking out of the main loop, I agree.
> 
>> Maybe what you are really looking for is
>> cpu_interrupt(CPU_INTERRUPT_HALT). That sets halted = 1 and exits the
>> main loop, because it's an interrupt.
> 
> cpu_interrupt(CPU_INTERRUPT_HALT) works fine for TCG but does not for KVM
> (the rtas call returns to the guest and it reports BUG).

How about cpu_exit()? That looks exactly like what we need.


Alex