From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46442) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XPpFl-0002d2-4E for qemu-devel@nongnu.org; Fri, 05 Sep 2014 04:52:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XPpFb-0000Az-9R for qemu-devel@nongnu.org; Fri, 05 Sep 2014 04:52:21 -0400 Received: from e35.co.us.ibm.com ([32.97.110.153]:55477) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XPpFa-0000Aa-Qn for qemu-devel@nongnu.org; Fri, 05 Sep 2014 04:52:11 -0400 Received: from /spool/local by e35.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 5 Sep 2014 02:52:10 -0600 Message-ID: <540979B1.7010703@linux.vnet.ibm.com> Date: Fri, 05 Sep 2014 14:22:01 +0530 From: Aravinda Prasad MIME-Version: 1.0 References: <20140825134353.2361.52046.stgit@aravindap> <20140825134545.2361.66860.stgit@aravindap> <53FDB4EA.3080009@suse.de> <53FECE82.6040908@linux.vnet.ibm.com> <53FEEA60.1000201@suse.de> <53FF6A09.1050707@linux.vnet.ibm.com> <53FFAA52.80705@suse.de> <54082213.305@linux.vnet.ibm.com> <6AD4C807-F0DD-46F5-ADBC-A91D9D7B15E7@suse.de> <54086DDB.1060800@linux.vnet.ibm.com> <54097859.5060302@suse.de> In-Reply-To: <54097859.5060302@suse.de> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [Qemu-ppc] [PATCH 4/5] target-ppc: Handle ibm, nmi-register RTAS call List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alexander Graf Cc: "aik@au1.ibm.com" , "benh@au1.ibm.com" , "qemu-ppc@nongnu.org" , "qemu-devel@nongnu.org" , "paulus@samba.org" On Friday 05 September 2014 02:16 PM, Alexander Graf wrote: > > > On 04.09.14 15:49, Aravinda Prasad wrote: >> >> >> On Thursday 04 September 2014 06:39 PM, Alexander Graf wrote: >>> >>> >>>> Am 04.09.2014 um 10:25 schrieb Aravinda Prasad : >>>> >>>> >>>> >>>>> On Friday 29 August 2014 03:46 AM, Alexander Graf wrote: >>>>> >>>>> >>>>>> On 28.08.14 19:42, Aravinda Prasad wrote: >>>>>> >>>>>> >>>>>>> On Thursday 28 August 2014 02:07 PM, Alexander Graf wrote: >>>>>>> >>>>>>> >>>>>>>> On 28.08.14 08:38, Aravinda Prasad wrote: >>>>>>>> >>>>>>>> >>>>>>>>> On Wednesday 27 August 2014 04:07 PM, Alexander Graf wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>>> On 25.08.14 15:45, Aravinda Prasad wrote: >>>>>>>>>> This patch adds FWNMI support in qemu for powerKVM >>>>>>>>>> guests by handling the ibm,nmi-register rtas call. >>>>>>>>>> Whenever OS issues ibm,nmi-register RTAS call, the >>>>>>>>>> machine check notification address is saved and the >>>>>>>>>> machine check interrupt vector 0x200 is patched to >>>>>>>>>> issue a private hcall. >>>>>>>>>> >>>>>>>>>> Signed-off-by: Aravinda Prasad >>>>>>>>>> --- >>>>>>>>>> hw/ppc/spapr_rtas.c | 91 ++++++++++++++++++++++++++++++++++++++++++++++++ >>>>>>>>>> include/hw/ppc/spapr.h | 8 ++++ >>>>>>>>>> 2 files changed, 98 insertions(+), 1 deletion(-) >>>>>>>>>> >>>>>>>>>> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c >>>>>>>>>> index 02ddbf9..1135d2b 100644 >>>>>>>>>> --- a/hw/ppc/spapr_rtas.c >>>>>>>>>> +++ b/hw/ppc/spapr_rtas.c >>>>>>>>>> @@ -277,6 +277,91 @@ static void rtas_ibm_set_system_parameter(PowerPCCPU *cpu, >>>>>>>>>> rtas_st(rets, 0, ret); >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> +static void rtas_ibm_nmi_register(PowerPCCPU *cpu, >>>>>>>>>> + sPAPREnvironment *spapr, >>>>>>>>>> + uint32_t token, uint32_t nargs, >>>>>>>>>> + target_ulong args, >>>>>>>>>> + uint32_t nret, target_ulong rets) >>>>>>>>>> +{ >>>>>>>>>> + int i; >>>>>>>>>> + uint32_t branch_inst = 0x48000002; >>>>>>>>>> + target_ulong guest_machine_check_addr; >>>>>>>>>> + PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cpu); >>>>>>>>>> + /* >>>>>>>>>> + * Trampoline saves r3 in sprg2 and issues private hcall >>>>>>>>>> + * to request qemu to build error log. QEMU builds the >>>>>>>>>> + * error log, copies to rtas-blob and returns the address. >>>>>>>>>> + * The initial 16 bytes in rtas-blob consists of saved srr0 >>>>>>>>>> + * and srr1 which we restore and pass on the actual error >>>>>>>>>> + * log address to OS handled mcachine check notification >>>>>>>>>> + * routine >>>>>>>>>> + */ >>>>>>>>>> + uint32_t trampoline[] = { >>>>>>>>>> + 0x7c7243a6, /* mtspr SPRN_SPRG2,r3 */ >>>>>>>>>> + 0x38600000, /* li r3,0 */ >>>>>>>>>> + /* 0xf004 is the KVMPPC_H_REPORT_ERR private HCALL */ >>>>>>>>>> + 0x6063f004, /* ori r3,r3,f004 */ >>>>>>>>>> + /* Issue H_CALL */ >>>>>>>>>> + 0x44000022, /* sc 1 */ >>>>>>>>> >>>>>>>>> So up to here we're saving r3 in SPRG2 (how do we know that we can >>>>>>>>> clobber it?) and call our special hypercall. >>>>>>>>> >>>>>>>>> But what does all the cruft below here do? >>>>>>>> >>>>>>>> The saved r3 in SPRG2 is consumed in KVMPPC_H_REPORT_ERR hcall, hence we >>>>>>>> can clobber SPRG2 after hcall returns. I have included a comment in >>>>>>>> patch 3/5 while building error log. I think better I add one here as well. >>>>>>>> >>>>>>>>> >>>>>>>>>> + 0x7c9243a6, /* mtspr r4 sprg2 */ >>>>>>>>> >>>>>>>>> Apart from th fact that your order is wrong, this destroys the value of >>>>>>>>> r3 that we saved above again. >>>>>>>> >>>>>>>> SPRG2 is saved inside hcall and hence we don't need SPRG2 further after >>>>>>>> KVMPPC_H_REPORT_ERR hcall returns. >>>>>>>> >>>>>>>>> >>>>>>>>>> + 0xe8830000, /* ld r4, 0(r3) */ >>>>>>>>>> + 0x7c9a03a6, /* mtspr r4, srr0 */ >>>>>>>>>> + 0xe8830008, /* ld r4, 8(r3) */ >>>>>>>>>> + 0x7c9b03a6, /* mtspr r4, srr1 */ >>>>>>>>> >>>>>>>>> Can't we just set srr0 and srr1 directly? >>>>>>>> >>>>>>>> I checked for instructions in ISA which set srr0/1 directly given an >>>>>>>> address, but could not find any such instructions. >>>>>>> >>>>>>> I mean from QEMU :). >>>>>> >>>>>> srr0 and srr1, which are properly set when 0x200 is invoked, are >>>>>> clobbered when we return from KVMPPC_H_REPORT_ERR hcall. I think they >>>>>> are modified before issuing rfid (I can see them getting clobbered from >>>>>> QEMU monitor). However when we jump to OS registered machine check >>>>>> routine srr0 and srr1 should reflect the value they had when 0x200 was >>>>>> invoked. >>>>>> >>>>>> Hence srr0 and srr1 are saved in hcall and restored when we return from >>>>>> hcall. Also we don't have enough scratch registers available to save >>>>>> these before invoking hcall from 0x200. >>>>>> >>>>>> Or am I missing other ways to do this from QEMU? >>>>> >>>>> If you just do >>>>> >>>>> cpu_synchronize_state() and then change env->spr[SPRN_SRR0/1] inside >>>>> your hypercall handler that should also change the value when you return >>>>> from the hcall. >>>> >>>> I tried cpu_synchronize_state(), however, srr0 and srr1 are still clobbered. >>>> >>>> Just before I issue hcall from 0x200 I see the following values from >>>> QEMU monitor: >>>> >>>> SRR0 d000000000f40264 SRR1 8000000000209033 >>>> >>>> Inside hcall, I call cpu_synchronize_state(). As soon as I return from >>>> hcall I see: >>>> >>>> SRR0 0000000000000214 SRR1 8000000000001001 >>>> >>>> I see SRR0 is now set to nip in 0x200 and SRR1 to msr value. I think it >>>> is reset during returning from hcall before executing rfid. > > Ah, because the hypercall is an interrupt itself. Heh ;). True. Yes. However, as you mentioned, we still need cpu_synchronize_state() to avoid stale value when we read env->spr[SPR_SPRG2] Aravinda. > > > Alex > -- Regards, Aravinda