Re: [Qemu-devel] [Qemu-ppc] [PATCH 4/5] target-ppc: Handle ibm, nmi-register RTAS call

[Alexander Graf <agraf@suse.de>]

On 04.09.14 15:49, Aravinda Prasad wrote:
> 
> 
> On Thursday 04 September 2014 06:39 PM, Alexander Graf wrote:
>>
>>
>>> Am 04.09.2014 um 10:25 schrieb Aravinda Prasad <aravinda@linux.vnet.ibm.com>:
>>>
>>>
>>>
>>>> On Friday 29 August 2014 03:46 AM, Alexander Graf wrote:
>>>>
>>>>
>>>>> On 28.08.14 19:42, Aravinda Prasad wrote:
>>>>>
>>>>>
>>>>>> On Thursday 28 August 2014 02:07 PM, Alexander Graf wrote:
>>>>>>
>>>>>>
>>>>>>> On 28.08.14 08:38, Aravinda Prasad wrote:
>>>>>>>
>>>>>>>
>>>>>>>> On Wednesday 27 August 2014 04:07 PM, Alexander Graf wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 25.08.14 15:45, Aravinda Prasad wrote:
>>>>>>>>> This patch adds FWNMI support in qemu for powerKVM
>>>>>>>>> guests by handling the ibm,nmi-register rtas call.
>>>>>>>>> Whenever OS issues ibm,nmi-register RTAS call, the
>>>>>>>>> machine check notification address is saved and the
>>>>>>>>> machine check interrupt vector 0x200 is patched to
>>>>>>>>> issue a private hcall.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Aravinda Prasad <aravinda@linux.vnet.ibm.com>
>>>>>>>>> ---
>>>>>>>>> hw/ppc/spapr_rtas.c    |   91 ++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>>>> include/hw/ppc/spapr.h |    8 ++++
>>>>>>>>> 2 files changed, 98 insertions(+), 1 deletion(-)
>>>>>>>>>
>>>>>>>>> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
>>>>>>>>> index 02ddbf9..1135d2b 100644
>>>>>>>>> --- a/hw/ppc/spapr_rtas.c
>>>>>>>>> +++ b/hw/ppc/spapr_rtas.c
>>>>>>>>> @@ -277,6 +277,91 @@ static void rtas_ibm_set_system_parameter(PowerPCCPU *cpu,
>>>>>>>>>     rtas_st(rets, 0, ret);
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> +static void rtas_ibm_nmi_register(PowerPCCPU *cpu,
>>>>>>>>> +                                  sPAPREnvironment *spapr,
>>>>>>>>> +                                  uint32_t token, uint32_t nargs,
>>>>>>>>> +                                  target_ulong args,
>>>>>>>>> +                                  uint32_t nret, target_ulong rets)
>>>>>>>>> +{
>>>>>>>>> +    int i;
>>>>>>>>> +    uint32_t branch_inst = 0x48000002;
>>>>>>>>> +    target_ulong guest_machine_check_addr;
>>>>>>>>> +    PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cpu);
>>>>>>>>> +    /*
>>>>>>>>> +     * Trampoline saves r3 in sprg2 and issues private hcall
>>>>>>>>> +     * to request qemu to build error log. QEMU builds the
>>>>>>>>> +     * error log, copies to rtas-blob and returns the address.
>>>>>>>>> +     * The initial 16 bytes in rtas-blob consists of saved srr0
>>>>>>>>> +     * and srr1 which we restore and pass on the actual error
>>>>>>>>> +     * log address to OS handled mcachine check notification
>>>>>>>>> +     * routine
>>>>>>>>> +     */
>>>>>>>>> +    uint32_t trampoline[] = {
>>>>>>>>> +        0x7c7243a6,    /* mtspr   SPRN_SPRG2,r3 */
>>>>>>>>> +        0x38600000,    /* li      r3,0   */
>>>>>>>>> +        /* 0xf004 is the KVMPPC_H_REPORT_ERR private HCALL */
>>>>>>>>> +        0x6063f004,    /* ori     r3,r3,f004  */
>>>>>>>>> +        /* Issue H_CALL */
>>>>>>>>> +        0x44000022,    /*  sc      1     */
>>>>>>>>
>>>>>>>> So up to here we're saving r3 in SPRG2 (how do we know that we can
>>>>>>>> clobber it?) and call our special hypercall.
>>>>>>>>
>>>>>>>> But what does all the cruft below here do?
>>>>>>>
>>>>>>> The saved r3 in SPRG2 is consumed in KVMPPC_H_REPORT_ERR hcall, hence we
>>>>>>> can clobber SPRG2 after hcall returns. I have included a comment in
>>>>>>> patch 3/5 while building error log. I think better I add one here as well.
>>>>>>>
>>>>>>>>
>>>>>>>>> +        0x7c9243a6,    /* mtspr r4 sprg2 */
>>>>>>>>
>>>>>>>> Apart from th fact that your order is wrong, this destroys the value of
>>>>>>>> r3 that we saved above again.
>>>>>>>
>>>>>>> SPRG2 is saved inside hcall and hence we don't need SPRG2 further after
>>>>>>> KVMPPC_H_REPORT_ERR hcall returns.
>>>>>>>
>>>>>>>>
>>>>>>>>> +        0xe8830000,    /* ld r4, 0(r3) */
>>>>>>>>> +        0x7c9a03a6,    /* mtspr r4, srr0 */
>>>>>>>>> +        0xe8830008,    /* ld r4, 8(r3) */
>>>>>>>>> +        0x7c9b03a6,    /* mtspr r4, srr1 */
>>>>>>>>
>>>>>>>> Can't we just set srr0 and srr1 directly?
>>>>>>>
>>>>>>> I checked for instructions in ISA which set srr0/1 directly given an
>>>>>>> address, but could not find any such instructions.
>>>>>>
>>>>>> I mean from QEMU :).
>>>>>
>>>>> srr0 and srr1, which are properly set when 0x200 is invoked, are
>>>>> clobbered when we return from KVMPPC_H_REPORT_ERR hcall. I think they
>>>>> are modified before issuing rfid (I can see them getting clobbered from
>>>>> QEMU monitor). However when we jump to OS registered machine check
>>>>> routine srr0 and srr1 should reflect the value they had when 0x200 was
>>>>> invoked.
>>>>>
>>>>> Hence srr0 and srr1 are saved in hcall and restored when we return from
>>>>> hcall. Also we don't have enough scratch registers available to save
>>>>> these before invoking hcall from 0x200.
>>>>>
>>>>> Or am I missing other ways to do this from QEMU?
>>>>
>>>> If you just do
>>>>
>>>> cpu_synchronize_state() and then change env->spr[SPRN_SRR0/1] inside
>>>> your hypercall handler that should also change the value when you return
>>>> from the hcall.
>>>
>>> I tried cpu_synchronize_state(), however, srr0 and srr1 are still clobbered.
>>>
>>> Just before I issue hcall from 0x200 I see the following values from
>>> QEMU monitor:
>>>
>>> SRR0 d000000000f40264  SRR1 8000000000209033
>>>
>>> Inside hcall, I call cpu_synchronize_state(). As soon as I return from
>>> hcall I see:
>>>
>>> SRR0 0000000000000214  SRR1 8000000000001001
>>>
>>> I see SRR0 is now set to nip in 0x200 and SRR1 to msr value. I think it
>>> is reset during returning from hcall before executing rfid.

Ah, because the hypercall is an interrupt itself. Heh ;). True.


Alex