Request for Documentation on Bringing Up Xen on R-Car H3e (H3ULCB)

Request for Documentation on Bringing Up Xen on R-Car H3e (H3ULCB)

[L, John Preetham (893)]

[-- Attachment #1: Type: text/plain, Size: 769 bytes --]

Dear Xen Community,

I hope this message finds you well.

I am currently working on a project that involves bringing up Xen on the Renesas R-Car H3e (H3ULCB) platform. I am seeking guidance on where I can find the proper documentation for this process. Specifically, I am looking for comprehensive documentation that covers everything from scratch to the end, including the required versions and any specific configurations.

Could anyone point me to the relevant resources or share any documentation that might be helpful?

Thank you in advance for your assistance.

Best regards,
John Preetham L


If you are not the addressee, please inform us immediately that you have received this e-mail by mistake, and delete it. We thank you for your support.


[-- Attachment #2: Type: text/html, Size: 4717 bytes --]

Re: Request for Documentation on Bringing Up Xen on R-Car H3e (H3ULCB)

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 1883 bytes --]

Hi John,

Let me add a few people that might have more information for you on this
topic.

Typically, the minimal set of operations to get started is the
following. Build Xen from sources (hypervisor only):

export CROSS_COMPILE=/path/to/cross-compiler
export XEN_TARGET_ARCH=arm64
cd xen.git/xen; make

Use ImageBuilder to generate a U-boot boot script to start
Xen and one ore more virtual machines:
https://gitlab.com/xen-project/imagebuilder/

Here are a few links:
https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions#Building_Xen_on_ARM
https://wiki.xenproject.org/wiki/ImageBuilder
https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Salvator-XS

I know this is for a different board (Xilinx), but it might help you
get concrete examples on how Xen is typically used:
https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/2451243106/Building+Xen+Hypervisor+with+PetaLinux+2022.1

People in CC might be able to give you additional material specific to
R-Car H3e.

Cheers,

Stefano


On Wed, 22 Jan 2025, L, John Preetham (893) wrote:
> Dear Xen Community,
> 
> I hope this message finds you well.
> 
> I am currently working on a project that involves bringing up Xen on the Renesas R-Car H3e (H3ULCB) platform. I am seeking guidance on
> where I can find the proper documentation for this process. Specifically, I am looking for comprehensive documentation that covers
> everything from scratch to the end, including the required versions and any specific configurations.
> 
> Could anyone point me to the relevant resources or share any documentation that might be helpful?
> 
> Thank you in advance for your assistance.
> 
> Best regards,
> John Preetham L
> 
>  
> 
> 
> If you are not the addressee, please inform us immediately that you have received this e-mail by mistake, and delete it. We thank you for
> your support.
> 
> 
> 

Re: Request for Documentation on Bringing Up Xen on R-Car H3e (H3ULCB)

[Volodymyr Babchuk]

Hi John,

Sorry, I hit wrong "reply" button last time. So I'm re-sending this mail
with xen-devel ML included.

"L, John Preetham (893)" <john_preetham.l@daimlertruck.com> writes:

> Dear Xen Community,
>
> I hope this message finds you well.
>
> I am currently working on a project that involves bringing up Xen on the Renesas R-Car H3e (H3ULCB) platform. I am seeking guidance on
> where I can find the proper documentation for this process. Specifically, I am looking for comprehensive documentation that covers everything
> from scratch to the end, including the required versions and any specific configurations.
>
> Could anyone point me to the relevant resources or share any documentation that might be helpful?

Well, you can build all from the scratch. I can guide you through all
the steps if you wish. But I recommend you to try this approach first:

https://github.com/xen-troops/meta-xt-prod-devel-rcar

This will build you a full system with Xen and (multiple domains of you
want) on SD card image. Just follow the instruction in the
README.md. You can use this as a starting point and/or reference for
your own setup.

Just be aware that this approach will build you highly customized setup
with "thin" Dom0 and separate driver domain. But anyways, this is a good
start.

Alternatively, you start with official Renesas manual on building BSP:

https://elinux.org/R-Car/Boards/H3SK#Quick_Start_How_To

and then build on top of. But this will require many manual steps. On
other hand, this will give your much better understanding on what goes
on. There are two ways:

1. You can build Xen hypervisor separately, but you will have issues
with building Xen Tools for your dom0, as you will need to create and
install Yocto SDK and then resolve incompatibilities between Yocto SDK
and Xen's own build system.

2. Easier approach is to add meta-virtualization meta-layer to your BSP
and build Xen hypervisor together will toolstack using this meta-layer.

Apart from building Xen Hypervisor and Xen Tools you also need to:

1. Ensure that ARM-TF boots the system in EL2. There is a
Renesas-specific makefile option for that.

2a. Alter BSP device tree, so it can be used by Xen. You need to provide
IO-MMU mappings.  Sadly, Renesas does not provide them by default. We
have custom dtsi just for that:
https://github.com/xen-troops/meta-xt-prod-devel-rcar/blob/master/layers/meta-xt-domd-gen3/recipes-kernel/linux/files/r8a77951-h3ulcb-4x2g-xen.dts

2b. Alter BSP device tree, so it can be used by Xen. Apart from IO-MMU
configuration you need to tell Xen where to look for module. You can use
ImageBuilder, mentioned by Stefano or do this by hand, like this:
https://github.com/xen-troops/meta-xt-rcar/blob/master/meta-xt-rcar-driver-domain/recipes-kernel/linux/files/pvr/xen-chosen.dtsi

3. Hide SCIF2 clock from Linux (or use "clk_ignore_unused" dom0 kernel
argument). Otherwise Dom0 will disable console during the boot.


-- 
WBR, Volodymyr

Re: Request for Documentation on Bringing Up Xen on R-Car H3e (H3ULCB)

[Volodymyr Babchuk]

"L, John Preetham (893)" <john_preetham.l@daimlertruck.com> writes:

Hi John,

> Hi Volodymyr,
>
> Thank you for the detailed suggestions.
> Since I'm new to XEN hypervisor.
>
> I will approach the recommended method.

Yeah, I think this is the best approach if you want get something
working ASAP.

> Could you please let me know which Yocto version is stable and tested from your end?

As of now we are using Kirkstone. I believe, this is the latest Yocto
version supported by Renesas BSP. And we of course can't jump over
their head. Anyways, if you'll follow instruction in README.md, moulin
tool will fetch all required meta-layers and configure Yocto for you.

If you are interested in exact commit ids, you can check the YAML file
used by moulin tool:

https://github.com/xen-troops/meta-xt-prod-devel-rcar/blob/master/prod-devel-rcar.yaml#L32

-- 
WBR, Volodymyr

RE: Request for Documentation on Bringing Up Xen on R-Car H3e (H3ULCB)

[L, John Preetham (893)]

Hi Volodymyr,

Thank you once again for the detailed explanation and the helpful resources.

With your guidance, I was able to bring up the XEN hypervisor on the R-Car H3e board successfully. I really appreciate your support.

Now, I'm looking to move forward with bringing up a QNX guest on the XEN hypervisor. Could you please share the procedure or steps you recommend for this? It would also be very helpful if you could provide an example Device Tree Source (DTS) and XEN domain configuration (CFG) file for the QNX guest, if available.

Looking forward to your input.

Best regards,
John Preetham L

-----Original Message-----
From: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Sent: 24 January 2025 10:33
To: L, John Preetham (893) <john_preetham.l@daimlertruck.com>
Cc: xen-devel@lists.xenproject.org; Ruslan Shymkevych <Ruslan_Shymkevych@epam.com>
Subject: Re: Request for Documentation on Bringing Up Xen on R-Car H3e (H3ULCB)

[You don't often get email from volodymyr_babchuk@epam.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

"L, John Preetham (893)" <john_preetham.l@daimlertruck.com> writes:

Hi John,

> Hi Volodymyr,
>
> Thank you for the detailed suggestions.
> Since I'm new to XEN hypervisor.
>
> I will approach the recommended method.

Yeah, I think this is the best approach if you want get something working ASAP.

> Could you please let me know which Yocto version is stable and tested from your end?

As of now we are using Kirkstone. I believe, this is the latest Yocto version supported by Renesas BSP. And we of course can't jump over their head. Anyways, if you'll follow instruction in README.md, moulin tool will fetch all required meta-layers and configure Yocto for you.

If you are interested in exact commit ids, you can check the YAML file used by moulin tool:

https://github.com/xen-troops/meta-xt-prod-devel-rcar/blob/master/prod-devel-rcar.yaml#L32

--
WBR, Volodymyr

If you are not the addressee, please inform us immediately that you have received this e-mail by mistake, and delete it. We thank you for your support.

Re: Request for Documentation on Bringing Up Xen on R-Car H3e (H3ULCB)

[Volodymyr Babchuk]

Hi John,

"L, John Preetham (893)" <john_preetham.l@daimlertruck.com> writes:

> Hi Volodymyr,
>
> Thank you once again for the detailed explanation and the helpful resources.
>
> With your guidance, I was able to bring up the XEN hypervisor on the R-Car H3e board successfully. I really appreciate your support.
>

I glad that it worked.

> Now, I'm looking to move forward with bringing up a QNX guest on the
> XEN hypervisor.

If your QNX guest supports arm64 boot protocol (which is defined at [1])
and supports either Xen HVC console or SBSA (pl011-ish) UART - you can just
boot it by providing kernel= option in your domain configuration file.

This at least should give you access to guest's serial console, which is
a good starting point to further bring-up.

> Could you please share the procedure or steps you
> recommend for this?

If I'd did this, I'd done this in the following way:

1. Check that QNX kernel image is compatible with aarch64 boot
protocol. Most of projects (Linux kernel, Xen, U-boot, etc) use it and
it become de-facto standard, so probably QNX should support it too. But
if not - you'll need another loader to boot QNX. We have Xen-enabled
U-Boot, by the way.

2. Boot QNX kernel and get serial console working. As I said, - either via
HVC or SBSA. This is crucial, as you need some feedback from your guest
to understand what going on.

3. Boot rest of QNX environment. See what is working, fix things that
don't :) It would be great if QNX supports Xen PV drivers for block
and network devices, otherwise, you'll have to use virtio or
pass-through, which is somewhat more difficult to configure.

> It would also be very helpful if you could provide
> an example Device Tree Source (DTS) and

If you are not doing real device passthrough (and I recommend to
postpone this until you'll get your guest running) - you don't need any
DTS files. Xen toolstack will generate all the required DTS entries
automatically for the given configuration.

> XEN domain configuration (CFG)
> file for the QNX guest, if available.

No, we don't have any. We had experience with QNX guests, but that was
really long time ago and I don't think that any data survived. Anyways,
I recommend checking manual on xl.cfg ([2]).

Please note, that if you want to use SBSA uart, you need to enable it in
your configuration file. It should be vuart = "sbsa_uart", I
believe. Also, you'll have to run xl console with additional arguments
to make it access SBSA uart console, not default HVC one.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/arch/arm64/booting.rst?h=v6.15-rc5
[2] https://xenbits.xen.org/docs/unstable/man/xl.cfg.5.html

-- 
WBR, Volodymyr