Re: Xen EFI Issue on Seattle

[Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>]

On 10/06/2014 04:27 AM, Ian Campbell wrote:
> On Sun, 2014-10-05 at 09:36 -0500, Suravee Suthikulpanit wrote:
>> Hi Roy,
>>
>> I finally get a chance to look into the Xen EFI booting issue on Seattle
>> that I showed you right at the end of the Linaro Connect.
>>
>> So, I was tracing the code below with DS-5 and Dstream:
>>
>> ENTRY(efi_xen_start)
>>           /*
>>            * Turn off cache and MMU as Xen expects. EFI enables them, but
>> also
>>            * mandates a 1:1 (unity) VA->PA mapping, so we can turn off the
>>            * MMU while executing EFI code before entering Xen.
>>            * The EFI loader calls this to start Xen.
>>            * Preserve x0 (fdf pointer) across call to __flush_dcache_all,
>>            * restore for entry into Xen.
>>            */
>> 	b     . /*SURAVEE DEBUG*/
>>           mov   x20, x0
>>           bl    __flush_dcache_all
>>           ic    ialluis
>>
>>           /* Turn off Dcache and MMU */
>>           mrs   x0, sctlr_el2
>>           bic   x0, x0, #1 << 0        /* clear SCTLR.M */
>>           bic   x0, x0, #1 << 2        /* clear SCTLR.C */
>>           msr   sctlr_el2, x0
>>           isb		<---- HERE
>>
>>           /* Jump to Xen entry point */
>>           mov   x0, x20
>>           mov   x1, xzr
>>           mov   x2, xzr
>>           mov   x3, xzr
>>           b     real_start
>> ENDPROC(efi_xen_start)
>>
>> For some reason, as soon as it execute "isb", the instructions
>> afterwards (i.e. move x0, x20; mov x1, xzr;  .... ) become all zeros.
>
> By any chance is the symptom of this a never ending spew of Synchronous
> abort exceptions?

Yep, that's the one.

> I've seen those symptoms with both xen.efi stub, grub.efi (when it
> launches the kernel) and I think once or twice (but not frequently) with
> the Linux stub. Not every time and not on every platform, but enough
> that I'm suspicious of something being wrong on the Tianocore side, or
> else the folks implementing this stuff are all the same folks and are
> copying the same mistake around, but I think that less likely.

I doubt that this is the same issue across all different EFI boot path 
though.  Usually, this is quite random. However, the one we see here is 
consistent.

>  From your description it almost sounds like we aren't running on 1:1
> page tables, like we think we should be. Are you able to manually (or
> automatically?) walk the PTs with a DS-5 prior to disabling them?

I checked the PA:VA and it seems to be 1:1 mapping except certain ranges 
are appear all zeros
PA:

AXI:0x83FC46D770 : DCI      0x00000000 ; ? Undefined
AXI:0x83FC46D774 : DCI      0x00000000 ; ? Undefined
AXI:0x83FC46D778 : DCI      0x00000000 ; ? Undefined
AXI:0x83FC46D77C : DCI      0x00000000 ; ? Undefined
AXI:0x83FC46D780 : RET
AXI:0x83FC46D784 : DCI      0x33323130 ; ? Undefined
AXI:0x83FC46D788 : TBNZ     w20,#6,{pc}-0x395c ; 0x83fc469e2c
AXI:0x83FC46D78C : DCI      0x62613938 ; ? Undefined
AXI:0x83FC46D790 : DCI      0x66656463 ; ? Undefined
AXI:0x83FC46D794 : MOV      x0,#0
AXI:0x83FC46D798 : RET
AXI:0x83FC46D79C : B        {pc} ; 0x83fc46d79c
AXI:0x83FC46D7A0 : MOV      x20,x0
AXI:0x83FC46D7A4 : BL       {pc}+0x55648 ; 0x83fc4c2dec
AXI:0x83FC46D7A8 : IC       IALLUIS
AXI:0x83FC46D7AC : MRS      x0,SCTLR_EL2
AXI:0x83FC46D7B0 : AND      x0,x0,#0xfffffffffffffffe
AXI:0x83FC46D7B4 : AND      x0,x0,#0xfffffffffffffffb
AXI:0x83FC46D7B8 : HLT      #1
AXI:0x83FC46D7BC : ISB
AXI:0x83FC46D7C0 : DCI      0x00000000 ; ? Undefined
AXI:0x83FC46D7C4 : DCI      0x00000000 ; ? Undefined
AXI:0x83FC46D7C8 : DCI      0x00000000 ; ? Undefined
AXI:0x83FC46D7CC : DCI      0x00000000 ; ? Undefined

VA:

EL2:0x00000083FC46D770 : STRB     w2,[x23,#0]
EL2:0x00000083FC46D774 : LSL      x0,x0,#4
EL2:0x00000083FC46D778 : SUBS     x3,x3,#1
EL2:0x00000083FC46D77C : B.NE     {pc}-0x24 ; 0x83fc46d758
EL2:0x00000083FC46D780 : RET
EL2:0x00000083FC46D784 : DCI      0x33323130 ; ? Undefined
EL2:0x00000083FC46D788 : TBNZ     w20,#6,{pc}-0x395c ; 0x83fc469e2c
EL2:0x00000083FC46D78C : DCI      0x62613938 ; ? Undefined
EL2:0x00000083FC46D790 : DCI      0x66656463 ; ? Undefined
EL2:0x00000083FC46D794 : MOV      x0,#0
EL2:0x00000083FC46D798 : RET
EL2:0x00000083FC46D79C : B        {pc} ; 0x83fc46d79c
EL2:0x00000083FC46D7A0 : MOV      x20,x0
EL2:0x00000083FC46D7A4 : BL       {pc}+0x55648 ; 0x83fc4c2dec
EL2:0x00000083FC46D7A8 : IC       IALLUIS
EL2:0x00000083FC46D7AC : MRS      x0,SCTLR_EL2
EL2:0x00000083FC46D7B0 : AND      x0,x0,#0xfffffffffffffffe
EL2:0x00000083FC46D7B4 : AND      x0,x0,#0xfffffffffffffffb
EL2:0x00000083FC46D7B8 : MSR      SCTLR_EL2,x0
EL2:0x00000083FC46D7BC : ISB
EL2:0x00000083FC46D7C0 : MOV      x0,x20
EL2:0x00000083FC46D7C4 : MOV      x1,xzr
EL2:0x00000083FC46D7C8 : MOV      x2,xzr
EL2:0x00000083FC46D7CC : MOV      x3,xzr

Suravee


> Ian.
>