Questions about XEN/ARM toolchain, XEN 4.5, Exynos 5420 and Arndale Octaboard.

Questions about XEN/ARM toolchain, XEN 4.5, Exynos 5420 and Arndale Octaboard.

[Fabio Canigliula]

[-- Attachment #1.1: Type: text/plain, Size: 4637 bytes --]

Hello,

I am a Computer Engineering student, currently working on building a
toolchain to have Xen running on ARM embedded platforms.

I have an Arndale Octa Board (Samsung Exynos 5420 MPSoC) and I am trying to
run Xen 4.5 on it.



I followed all the steps written in the Wiki (
http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions/Arndale) as
guideline and the following discussions about the Octaboard:
[1] - http://xen.markmail.org/thread/3shxyiipnq2ewj2k
[2] - http://markmail.org/thread/dhiknm25xfbsrlo6

I have successfully enabled the early_printk feature of Xen for the Exynos
5420 [1], so that I could have the following debug messages on Xen booting:

----------------------------------------------------
Starting kernel ...

- UART enabled -
- CPU 00000000 booting -
- Xen must be entered in NS Hyp mode -
- Boot failed -
----------------------------------------------------

The problem is that Xen does not match the correct CPU mode for its boot.
For this reason I started working on the booting process, trying to deeply
understand how it works for this board.



The board utilizes a secure multi stages booting process (chain of trust),
having 4 different boot levels (BL#):

- BL0 runs on iROM, vendor fused.
- BL1 runs on iRAM, vendor provided.
- BL2 runs on memory, signed.
- BL3 is U-Boot and the only one working seems to be a signed 2012.07
version.

As far as I know, the booting processor loads Xen while in Non-Secure state
and Supervisor mode (NS SVC), failing its boot and generating the above
debug messages. To bring the processor in Hypervisor mode (NS HYP), I
patched U-Boot by adding all the virtualization modules needed by the ARMv7
U-Boot subtree (u-boot/arch/arm/cpu/armv7) (
http://lists.denx.de/pipermail/u-boot/2013-September/163019.html).



The patch executes two functions at the end of the boot_prep_linux( … )
function in bootm.c file (../arch/arm/lib):

- armv7_switch_nonsec(), which brings the processor from secure to
non-secure state.
- armv7_switch_hyp(), which brings the processor from a non-secure mode to
the HYP mode.

However, CPU does not switch properly to HYP mode as the execution hangs.
The problem is that the processor is already in NS SVC mode when the
armv7_switch_nonsec() is invoked: the assembly routine _nonsec_init in
nonsec_virt.S file fails when it tries to write the MVBAR register, only
accessible from a secure mode. Unfortunately the function (or some of its
instructions) cannot be simply bypassed, as it holds all the set-up needed
to be done in the secure state before actually switching to HYP mode: it
sets up vector table and registers needed to enable the hypervisor call
(hvc #0) instruction (else, it would hang just as in [2]).



Following the discussion (
http://lists.denx.de/pipermail/u-boot/2013-April/150718.html), I patched
the start.S file, where the switch to NS SVC mode seems to happen. The
patch just tests if the processor is already in HYP mode and does not
switch back to SVC mode. Unfortunately neither this attempt was successful,
driving me to think that the problem is even before the U-Boot (BL3) stage.



The solution would be having the BL2 to set the processor in HYP mode and
deliver it to U-Boot already in that mode, so that Xen would boot easily.
Changing the BL2 requires the signature of the manufacturer, as it happens
for the Odroid-XU3 board (Exynos 5422) (
http://forum.odroid.com/viewtopic.php?f=98&t=6147) but Unfortunately acting
on BL2 is not possible for the Arndale Octaboard, as there is no support
from the Insignal.



Is there a solution to overcome these problems, or anyone still working on
it?



When Xen 4.5 was released I read in the notes that support for the Exynos
5420 MPSoC was added (
http://wiki.xenproject.org/wiki/Xen_Project_4.5_Feature_List#ARM_Hypervisor-Specific_Updates).
Which is the platform where it was implemented? Is it open or is there
private software just like in the Arndale Octaboard case?



Downline of everything, this experience was needed to understand that I
cannot use boards equipped with private software for my work, as overcoming
it would be “hacking” more than building a good toolchain. As my goal is to
have the full control over the hardware I am going to use, I cannot have
private stages that requires intervention of a third part help.


Could you suggest me some boards based on ARMv7 (A7 and/or A15) or ARMv8
fully controllable and on which Xen 4.5 works? Any help would be greatly
appreciated.





Regards,

F.C.

[-- Attachment #1.2: Type: text/html, Size: 10084 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Questions about XEN/ARM toolchain, XEN 4.5, Exynos 5420 and Arndale Octaboard.

[Ian Campbell]

On Thu, 2015-03-05 at 16:45 +0100, Fabio Canigliula wrote:
> The solution would be having the BL2 to set the processor in HYP mode
> and deliver it to U-Boot already in that mode, so that Xen would boot
> easily. Changing the BL2 requires the signature of the manufacturer,
> as it happens for the Odroid-XU3 board (Exynos 5422)
> (http://forum.odroid.com/viewtopic.php?f=98&t=6147) but Unfortunately
> acting on BL2 is not possible for the Arndale Octaboard, as there is
> no support from the Insignal.

There might be an SMC call which you can make to ask Secure world to
switch you up to NS HYP, or even an HVC to do it, but discovering it
without support from the manufacturer would probably be quite tricky (if
not impossible)

I'm afraid that once you are in NS.SVC without the earlier code having
left behind a handler for such a call there is no way to get back into
HYP mode.

> Is there a solution to overcome these problems, or anyone still
> working on it?

Not that I am aware of. You might have more luck in a forum dedicated to
hacking on these devices and/or SoCs, if there is one.

> When Xen 4.5 was released I read in the notes that support for the
> Exynos 5420 MPSoC was added
> (http://wiki.xenproject.org/wiki/Xen_Project_4.5_Feature_List#ARM_Hypervisor-Specific_Updates). Which is the platform where it was implemented? Is it open or is there private software just like in the Arndale Octaboard case?

I'm afraid that think this might have been a typo and should actually
refer to the 5422 in the Odroid-XU. At least neither my memory nor my
mailbox say anything about the Arndale Octa.

Sorry about this, I'm going to go fix the wiki right now.

> Could you suggest me some boards based on ARMv7 (A7 and/or A15) or
> ARMv8 fully controllable and on which Xen 4.5 works? Any help would be
> greatly appreciated.

My preferred option is the Cubietruck (AKA cubieboard 3).

Ian.

Re: Questions about XEN/ARM toolchain, XEN 4.5, Exynos 5420 and Arndale Octaboard.

[Ian Campbell]

(I'm putting the list back since I think the below may be of more
general interest)
On Sat, 2015-03-07 at 12:19 +0100, Fabio Canigliula wrote:
> >> Could you suggest me some boards based on ARMv7 (A7 and/or A15) or
> >> ARMv8 fully controllable and on which Xen 4.5 works? Any help would be
> >> greatly appreciated.
> >
> > My preferred option is the Cubietruck (AKA cubieboard 3).
> 
> I am very thankful for your suggestion. It is indeed a very
> interesting board with open source hardware and with an active
> community.

BTW there are also various Allwinner based platforms which are fully
Open Source HW (i.e schematics, layouts etc). linux-sunxi.org has pages
for a variety of then. The various Olimex boards are the main example.
I've not tried on myself but IMHO anything with an A20 (sun7i) processor
(same a Cubietruck) should work with Xen OK.

I don't think we've tried anything other sunxi, but I'd expect anything
with a Cortex-A7 or -A15 to work OK, although none of the newer ones
seem to have onboard SATA which is a bit sad.

The older Cortex-A8 based stuff won't work with Xen (no virt
extensions).

Ian.