undefined instruction error during HYP mode switch for arndale-octa board

undefined instruction error during HYP mode switch for arndale-octa board

[HyonYoung Choi]

[-- Attachment #1.1: Type: text/plain, Size: 1237 bytes --]

Hello Andre,

I am trying to port xen to arndale-octa board.

I am using u-boot modified for arndale-octa
from tracking-arndale_octa branch of
http://landing-teams/working/samsung/u-boot.git

Because there is no HYP mode switch code in the u-boot,
I tried to port your code in
http://git.linaro.org/git/people/andre.przywara/u-boot.git.
I also put debug code.

In booting, it reboot infinitely with following error message:
======================
   switch_hyp: GIC dist enabled
  switch_hyp: 224 interrupts supportd
  switch_hyp: all interrupts switched to non-secure
  switch_hyp: setting SP start start address (0x9f86d6b4)
  switch_hyp: kicked all CPUs
  VBAR before: 0x9f86c000
undefined instruction
pc : [<9f86d748>]          lr : [<9f86d600>]
sp : 9f7638c0  ip : 0000000f     fp : 9f86e4b4
r10: 9f8ad294  r9 : 00000000     r8 : 9f763f10
r7 : 00000000  r6 : ffffffff     r5 : 00000007  r4 : 9f86c000
r3 : 10482000  r2 : 10480000     r1 : 016e3600  r0 : 00010000
Flags: nZCv  IRQs off  FIQs off  Mode SVC_32
Resetting CPU ...
========================

The problematic code is "mcr     p15, 0, r1, c1, c1, 2"
in arch/arm/cpu/armv7/nonsec_virt.S , nonsec_init function

Do you have any idea about this error?

Regards,
Hyonyoung Choi

[-- Attachment #1.2: Type: text/html, Size: 1910 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: undefined instruction error during HYP mode switch for arndale-octa board

[Julien Grall]

Hello HyonYoung,

On 18/04/14 22:26, HyonYoung Choi wrote:
> I am trying to port xen to arndale-octa board.
>
> I am using u-boot modified for arndale-octa
> from tracking-arndale_octa branch of
> http://landing-teams/working/samsung/u-boot.git
>
> Because there is no HYP mode switch code in the u-boot,
> I tried to port your code in
> http://git.linaro.org/git/people/andre.przywara/u-boot.git.
> I also put debug code.
>
> In booting, it reboot infinitely with following error message:
> ======================
>     switch_hyp: GIC dist enabled
>    switch_hyp: 224 interrupts supportd
>    switch_hyp: all interrupts switched to non-secure
>    switch_hyp: setting SP start start address (0x9f86d6b4)
>    switch_hyp: kicked all CPUs
>    VBAR before: 0x9f86c000
> undefined instruction
> pc : [<9f86d748>]          lr : [<9f86d600>]
> sp : 9f7638c0  ip : 0000000f     fp : 9f86e4b4
> r10: 9f8ad294  r9 : 00000000     r8 : 9f763f10
> r7 : 00000000  r6 : ffffffff     r5 : 00000007  r4 : 9f86c000
> r3 : 10482000  r2 : 10480000     r1 : 016e3600  r0 : 00010000
> Flags: nZCv  IRQs off  FIQs off  Mode SVC_32
> Resetting CPU ...
> ========================
>
> The problematic code is "mcr     p15, 0, r1, c1, c1, 2"
> in arch/arm/cpu/armv7/nonsec_virt.S , nonsec_init function
> Do you have any idea about this error?

U-boot is trying to write into NSCAR. This register is only writeable in 
Secure PL1 mode.

I suspect U-boot is running in non-secure mode. You may have the same 
problem as the odroid-xu where the SPL BL2 code brings up CPU in NS mode 
(see http://www.gossamer-threads.com/lists/xen/devel/325364#325364).

Regards,

-- 
Julien Grall

Re: undefined instruction error during HYP mode switch for arndale-octa board

[HyonYoung Choi]

[-- Attachment #1.1: Type: text/plain, Size: 4797 bytes --]

Hello Julien,

At first, I want to know who is the maintainer for this U-Boot or
Hypervisor mode support section.

>From previous email, I found that U-boot is running already in non-secure
mode.

>From the comment of the erroneous code in arch/arm/cpu/armv7/nonsec_vert.S,
it is guessed that it tries to switch non-secure mode assuming the CPU is
in secure mode.
========================================
/*
 * Switch a core to non-secure state.
 *
 *  1. initialize the GIC per-core interface
 *  2. allow coprocessor access in non-secure modes
 *  3. switch the cpu mode (by calling "smc #0")
 *
 * Called from smp_pen by secondary cores and directly by the BSP.
 * Do not assume that the stack is available and only use registers
 * r0-r3 and r12.
 *
 * PERIPHBASE is used to get the GIC address. This could be 40 bits long,
 * though, but we check this in C before calling this function.
 */
ENTRY(_nonsec_init)
=========================================

But I have known CPU is already in non-secure mode,
so, we commented out the calling _nonsec_init() and
just call hypervisor mode switch function only as below:

=========================================
static void do_nonsec_virt_switch(void)
{
#if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
//      if (armv7_switch_nonsec() == 0)
#ifdef CONFIG_ARMV7_VIRT
                if (armv7_switch_hyp() == 0)
                        debug("entered HYP mode\n");
#else
                debug("entered non-secure state\n");
#endif
#endif
}
=========================================

The code of function armv7_switch_hyp() is simple:
=========================================
ENTRY(_switch_to_hyp)
        mov     r0, lr
        mov     r1, sp                          @ save SVC copy of LR and SP
        isb
        hvc #0                   @ for older asm: .byte 0x70, 0x00, 0x40,
0xe1
        mov     sp, r1
        mov     lr, r0                          @ restore SVC copy of LR
and SP
        bx      lr
ENDPROC(_switch_to_hyp)
===========================================

But it hangs while booting in "hvc #0" instruction.

----------------------------------------------------------------------
## Booting kernel from Legacy Image at 20007000 ...
   Image Name:
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    656144 Bytes = 640.8 KiB
   Load Address: 20008000
   Entry Point:  20008000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 21f00000
   Booting using the fdt blob at 0x21f00000
   Loading Kernel Image ... OK
OK
   reserving fdt memory region: addr=21f00000 size=8000
   Using Device Tree in place at 21f00000, end 21f0afff
  switch_hyp: CPU supports virt ext
----------------------------------------------------------------------

>From the original sequence of function calls,
nonsec_init() is for turning to non-secure mode,
then it calls switch_to_hyp () in non-secure mode.

Is there any preparation for executing "hvc" instruction or any privilege
to be executed?

Best,
Hyonyoung

2014-04-21 8:39 GMT-04:00 Julien Grall <julien.grall@linaro.org>:

> Hello HyonYoung,
>
>
> On 18/04/14 22:26, HyonYoung Choi wrote:
>
>> I am trying to port xen to arndale-octa board.
>>
>> I am using u-boot modified for arndale-octa
>> from tracking-arndale_octa branch of
>> http://landing-teams/working/samsung/u-boot.git
>>
>> Because there is no HYP mode switch code in the u-boot,
>> I tried to port your code in
>> http://git.linaro.org/git/people/andre.przywara/u-boot.git.
>> I also put debug code.
>>
>> In booting, it reboot infinitely with following error message:
>> ======================
>>     switch_hyp: GIC dist enabled
>>    switch_hyp: 224 interrupts supportd
>>    switch_hyp: all interrupts switched to non-secure
>>    switch_hyp: setting SP start start address (0x9f86d6b4)
>>    switch_hyp: kicked all CPUs
>>    VBAR before: 0x9f86c000
>> undefined instruction
>> pc : [<9f86d748>]          lr : [<9f86d600>]
>> sp : 9f7638c0  ip : 0000000f     fp : 9f86e4b4
>> r10: 9f8ad294  r9 : 00000000     r8 : 9f763f10
>> r7 : 00000000  r6 : ffffffff     r5 : 00000007  r4 : 9f86c000
>> r3 : 10482000  r2 : 10480000     r1 : 016e3600  r0 : 00010000
>> Flags: nZCv  IRQs off  FIQs off  Mode SVC_32
>> Resetting CPU ...
>> ========================
>>
>> The problematic code is "mcr     p15, 0, r1, c1, c1, 2"
>> in arch/arm/cpu/armv7/nonsec_virt.S , nonsec_init function
>> Do you have any idea about this error?
>>
>
> U-boot is trying to write into NSCAR. This register is only writeable in
> Secure PL1 mode.
>
> I suspect U-boot is running in non-secure mode. You may have the same
> problem as the odroid-xu where the SPL BL2 code brings up CPU in NS mode
> (see http://www.gossamer-threads.com/lists/xen/devel/325364#325364).
>
> Regards,
>
> --
> Julien Grall
>

[-- Attachment #1.2: Type: text/html, Size: 13521 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: undefined instruction error during HYP mode switch for arndale-octa board

[Julien Grall]

On 04/22/2014 12:11 AM, HyonYoung Choi wrote:
> Hello Julien,

Hello HyonYoung,

> At first, I want to know who is the maintainer for this U-Boot or
> Hypervisor mode support section.

I'm not sure who is the maintainer of this U-boot. I've cced Tushar
which seems to be in most of the commit related to the octa.

> From previous email, I found that U-boot is running already in
> non-secure mode.
> 
> From the comment of the erroneous code in arch/arm/cpu/armv7/nonsec_vert.S, 
> it is guessed that it tries to switch non-secure mode assuming the CPU
> is in secure mode.
> ========================================
> /*
>  * Switch a core to non-secure state.
>  *
>  *  1. initialize the GIC per-core interface
>  *  2. allow coprocessor access in non-secure modes
>  *  3. switch the cpu mode (by calling "smc #0")
>  *
>  * Called from smp_pen by secondary cores and directly by the BSP.
>  * Do not assume that the stack is available and only use registers
>  * r0-r3 and r12.
>  *
>  * PERIPHBASE is used to get the GIC address. This could be 40 bits long,
>  * though, but we check this in C before calling this function.
>  */
> ENTRY(_nonsec_init)
> =========================================
> 
> But I have known CPU is already in non-secure mode, 
> so, we commented out the calling _nonsec_init() and
> just call hypervisor mode switch function only as below:
> 
> =========================================
> static void do_nonsec_virt_switch(void)
> {
> #if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
> //      if (armv7_switch_nonsec() == 0)
> #ifdef CONFIG_ARMV7_VIRT
>                 if (armv7_switch_hyp() == 0)
>                         debug("entered HYP mode\n");
> #else
>                 debug("entered non-secure state\n");
> #endif
> #endif
> }
> =========================================
> 
> The code of function armv7_switch_hyp() is simple:
> =========================================
> ENTRY(_switch_to_hyp)
>         mov     r0, lr
>         mov     r1, sp                          @ save SVC copy of LR and SP
>         isb
>         hvc #0                   @ for older asm: .byte 0x70, 0x00,
> 0x40, 0xe1
>         mov     sp, r1
>         mov     lr, r0                          @ restore SVC copy of LR
> and SP
>         bx      lr
> ENDPROC(_switch_to_hyp)
> ===========================================
> 
> But it hangs while booting in "hvc #0" instruction.
> 
> ----------------------------------------------------------------------
> ## Booting kernel from Legacy Image at 20007000 ...
>    Image Name:   
>    Image Type:   ARM Linux Kernel Image (uncompressed)
>    Data Size:    656144 Bytes = 640.8 KiB
>    Load Address: 20008000
>    Entry Point:  20008000
>    Verifying Checksum ... OK
> ## Flattened Device Tree blob at 21f00000
>    Booting using the fdt blob at 0x21f00000
>    Loading Kernel Image ... OK
> OK
>    reserving fdt memory region: addr=21f00000 size=8000
>    Using Device Tree in place at 21f00000, end 21f0afff
>   switch_hyp: CPU supports virt ext
> ----------------------------------------------------------------------
> 
> From the original sequence of function calls,
> nonsec_init() is for turning to non-secure mode,
> then it calls switch_to_hyp () in non-secure mode.
> 
> Is there any preparation for executing "hvc" instruction or any
> privilege to be executed?

You've commented the function armv7_switch_nonsec that set up the
monitors traps (and therefore the hyp trap). The "hvc #0" instruction
will jump to an unknown memory.

Regards,

-- 
Julien Grall

Re: undefined instruction error during HYP mode switch for arndale-octa board

[Tushar Behera]

On 04/22/2014 07:39 PM, Julien Grall wrote:
> On 04/22/2014 12:11 AM, HyonYoung Choi wrote:
>> Hello Julien,
> 
> Hello HyonYoung,
> 
>> At first, I want to know who is the maintainer for this U-Boot or
>> Hypervisor mode support section.
> 
> I'm not sure who is the maintainer of this U-boot. I've cced Tushar
> which seems to be in most of the commit related to the octa.
> 

The u-boot code was referenced from following git tree[1]. I have little
knowledge regarding Hypervisor mode support, so won't be of much help :(

[1] http://git.insignal.co.kr/insignal/arndale_octa-jb_mr1.1/u-boot

>> From previous email, I found that U-boot is running already in
>> non-secure mode.
>>
>> From the comment of the erroneous code in arch/arm/cpu/armv7/nonsec_vert.S, 
>> it is guessed that it tries to switch non-secure mode assuming the CPU
>> is in secure mode.
>> ========================================
>> /*
>>  * Switch a core to non-secure state.
>>  *
>>  *  1. initialize the GIC per-core interface
>>  *  2. allow coprocessor access in non-secure modes
>>  *  3. switch the cpu mode (by calling "smc #0")
>>  *
>>  * Called from smp_pen by secondary cores and directly by the BSP.
>>  * Do not assume that the stack is available and only use registers
>>  * r0-r3 and r12.
>>  *
>>  * PERIPHBASE is used to get the GIC address. This could be 40 bits long,
>>  * though, but we check this in C before calling this function.
>>  */
>> ENTRY(_nonsec_init)
>> =========================================
>>
>> But I have known CPU is already in non-secure mode, 
>> so, we commented out the calling _nonsec_init() and
>> just call hypervisor mode switch function only as below:
>>
>> =========================================
>> static void do_nonsec_virt_switch(void)
>> {
>> #if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
>> //      if (armv7_switch_nonsec() == 0)
>> #ifdef CONFIG_ARMV7_VIRT
>>                 if (armv7_switch_hyp() == 0)
>>                         debug("entered HYP mode\n");
>> #else
>>                 debug("entered non-secure state\n");
>> #endif
>> #endif
>> }
>> =========================================
>>
>> The code of function armv7_switch_hyp() is simple:
>> =========================================
>> ENTRY(_switch_to_hyp)
>>         mov     r0, lr
>>         mov     r1, sp                          @ save SVC copy of LR and SP
>>         isb
>>         hvc #0                   @ for older asm: .byte 0x70, 0x00,
>> 0x40, 0xe1
>>         mov     sp, r1
>>         mov     lr, r0                          @ restore SVC copy of LR
>> and SP
>>         bx      lr
>> ENDPROC(_switch_to_hyp)
>> ===========================================
>>
>> But it hangs while booting in "hvc #0" instruction.
>>
>> ----------------------------------------------------------------------
>> ## Booting kernel from Legacy Image at 20007000 ...
>>    Image Name:   
>>    Image Type:   ARM Linux Kernel Image (uncompressed)
>>    Data Size:    656144 Bytes = 640.8 KiB
>>    Load Address: 20008000
>>    Entry Point:  20008000
>>    Verifying Checksum ... OK
>> ## Flattened Device Tree blob at 21f00000
>>    Booting using the fdt blob at 0x21f00000
>>    Loading Kernel Image ... OK
>> OK
>>    reserving fdt memory region: addr=21f00000 size=8000
>>    Using Device Tree in place at 21f00000, end 21f0afff
>>   switch_hyp: CPU supports virt ext
>> ----------------------------------------------------------------------
>>
>> From the original sequence of function calls,
>> nonsec_init() is for turning to non-secure mode,
>> then it calls switch_to_hyp () in non-secure mode.
>>
>> Is there any preparation for executing "hvc" instruction or any
>> privilege to be executed?
> 
> You've commented the function armv7_switch_nonsec that set up the
> monitors traps (and therefore the hyp trap). The "hvc #0" instruction
> will jump to an unknown memory.
> 
> Regards,
> 


-- 
Tushar Behera

Re: undefined instruction error during HYP mode switch for arndale-octa board

[Tushar Behera]

On 04/22/2014 07:39 PM, Julien Grall wrote:
> On 04/22/2014 12:11 AM, HyonYoung Choi wrote:
>> Hello Julien,
> 
> Hello HyonYoung,
> 
>> At first, I want to know who is the maintainer for this U-Boot or
>> Hypervisor mode support section.
> 
> I'm not sure who is the maintainer of this U-boot. I've cced Tushar
> which seems to be in most of the commit related to the octa.
> 

The u-boot code was referenced from following git tree[1]. I have little
knowledge regarding Hypervisor mode support, so won't be of much help :(

[1] http://git.insignal.co.kr/insignal/arndale_octa-jb_mr1.1/u-boot

>> From previous email, I found that U-boot is running already in
>> non-secure mode.
>>
>> From the comment of the erroneous code in arch/arm/cpu/armv7/nonsec_vert.S, 
>> it is guessed that it tries to switch non-secure mode assuming the CPU
>> is in secure mode.
>> ========================================
>> /*
>>  * Switch a core to non-secure state.
>>  *
>>  *  1. initialize the GIC per-core interface
>>  *  2. allow coprocessor access in non-secure modes
>>  *  3. switch the cpu mode (by calling "smc #0")
>>  *
>>  * Called from smp_pen by secondary cores and directly by the BSP.
>>  * Do not assume that the stack is available and only use registers
>>  * r0-r3 and r12.
>>  *
>>  * PERIPHBASE is used to get the GIC address. This could be 40 bits long,
>>  * though, but we check this in C before calling this function.
>>  */
>> ENTRY(_nonsec_init)
>> =========================================
>>
>> But I have known CPU is already in non-secure mode, 
>> so, we commented out the calling _nonsec_init() and
>> just call hypervisor mode switch function only as below:
>>
>> =========================================
>> static void do_nonsec_virt_switch(void)
>> {
>> #if defined(CONFIG_ARMV7_NONSEC) || defined(CONFIG_ARMV7_VIRT)
>> //      if (armv7_switch_nonsec() == 0)
>> #ifdef CONFIG_ARMV7_VIRT
>>                 if (armv7_switch_hyp() == 0)
>>                         debug("entered HYP mode\n");
>> #else
>>                 debug("entered non-secure state\n");
>> #endif
>> #endif
>> }
>> =========================================
>>
>> The code of function armv7_switch_hyp() is simple:
>> =========================================
>> ENTRY(_switch_to_hyp)
>>         mov     r0, lr
>>         mov     r1, sp                          @ save SVC copy of LR and SP
>>         isb
>>         hvc #0                   @ for older asm: .byte 0x70, 0x00,
>> 0x40, 0xe1
>>         mov     sp, r1
>>         mov     lr, r0                          @ restore SVC copy of LR
>> and SP
>>         bx      lr
>> ENDPROC(_switch_to_hyp)
>> ===========================================
>>
>> But it hangs while booting in "hvc #0" instruction.
>>
>> ----------------------------------------------------------------------
>> ## Booting kernel from Legacy Image at 20007000 ...
>>    Image Name:   
>>    Image Type:   ARM Linux Kernel Image (uncompressed)
>>    Data Size:    656144 Bytes = 640.8 KiB
>>    Load Address: 20008000
>>    Entry Point:  20008000
>>    Verifying Checksum ... OK
>> ## Flattened Device Tree blob at 21f00000
>>    Booting using the fdt blob at 0x21f00000
>>    Loading Kernel Image ... OK
>> OK
>>    reserving fdt memory region: addr=21f00000 size=8000
>>    Using Device Tree in place at 21f00000, end 21f0afff
>>   switch_hyp: CPU supports virt ext
>> ----------------------------------------------------------------------
>>
>> From the original sequence of function calls,
>> nonsec_init() is for turning to non-secure mode,
>> then it calls switch_to_hyp () in non-secure mode.
>>
>> Is there any preparation for executing "hvc" instruction or any
>> privilege to be executed?
> 
> You've commented the function armv7_switch_nonsec that set up the
> monitors traps (and therefore the hyp trap). The "hvc #0" instruction
> will jump to an unknown memory.
> 
> Regards,
> 


-- 
Tushar Behera