Re: [Qemu-devel] Jailhouse on Ultrascale+ (ZCU102)

["Edgar E. Iglesias" <edgar.iglesias@xilinx.com>]

On Fri, Apr 21, 2017 at 10:04:56AM +0300, Constantin Petra wrote:
> OK,
> 
> Changed configs for inmate to UART0, and the messages appeared on the
> console (on the ZCU102 board, not QEMU), so it is indeed functional.
> root@plnx_aarch64:~# jailhouse/usr/local/sbin/jailhouse cell start
> ZynqMP-linux-demo
> Started cell "ZynqMP-linux-demo"
> Hello 1 from cell!
> Hello 2 from cell!
> Hello 3 from cell!
> ---
> etc
> 
> Edgar, are the images I have sent of any help?

Thanks Constantin, yes, I can reproduce a lockup.
This is what I see:
root@plnx_aarch64:~# jailhouse/usr/local/sbin/jailhouse enable /home/root/jailhouse/configs/zynqmp-zcu102.cell

Initializing Jailhouse hypervisor v0.6 (47-ge02403c-dirty) on CPU 2
Code location: 0x0000ffffc0200040
Page pool usage after early setup: mem 33/996, remap 64/131072
Initializing processors:
 CPU 2... OK
 CPU 3... OK
 CPU 0... OK
 CPU 1... OK
Adding virtual PCI device 00:00.0 to cell "ZynqMP-ZCU102"
Adding virtual PCI device 00:01.0 to cell "ZynqMP-ZCU102"
Page pool usage after late setup: mem 44/996, remap 69/131072
Activating hypervisor
[   73.743265] OF: PCI: host bridge //vpci@0 ranges:
[   73.744161] OF: PCI:   MEM 0xfc100000..0xfc103fff -> 0xfc100000
[   73.747033] pci-host-generic fc000000.vpci: ECAM at [mem 0xfc000000-0xfc0fffff] for [bus 00]
[   73.748659] pci-host-generic fc000000.vpci: PCI host bridge to bus 0001:00
[   73.748972] pci_bus 0001:00: root bus resource [bus 00]
[   73.749644] pci_bus 0001:00: root bus resource [mem 0xfc100000-0xfc103fff]
[   73.758116] pci 0001:00:00.0: BAR 0: assigned [mem 0xfc100000-0xfc1000ff 64bit]
[   73.758928] pci 0001:00:01.0: BAR 0: assigned [mem 0xfc100100-0xfc1001ff 64bit]
[   73.761465] The Jailhouse is opening.
root@plnx_aarch64:~# [   74.710264] ivshmem-net 0001:00:00.0: enabling device (0000 -> 0002)
[   74.713514] ivshmem-net: probe of 0001:00:00.0 failed with error -16
[   74.714971] ivshmem-net 0001:00:01.0: enabling device (0000 -> 0002)
[   74.715904] ivshmem-net: probe of 0001:00:01.0 failed with error -16


After that, it all deadlocks.
I'll see what I can find.

Cheers,
Edgar

> Let me know if you need any other information about the setup.
> 
> Best Regards,
> Constantin
> P.S. Sent the above previously by chance only to Edgar, reposted again for
> the larger audience :)
> 
> On Thu, Apr 20, 2017 at 11:19 AM, Constantin Petra <
> constantin.petra@gmail.com> wrote:
> 
> > Hi,
> >
> > Sure, I have added the archive containing the image files and jailhouse
> > build here (the archive contains the /tftpboot folder directly)
> > https://drive.google.com/file/d/0B2bhbcx8eg-caTBNdnZ4cWhJcWc/view?usp=
> > sharing
> >
> > I use the following sequence to start qemu / kernel / jailhouse, provided
> > the /tftpboot folder has the structure in the previously archive and, of
> > course, there is tftp access to the host)
> > petalinux-boot --qemu --uboot --qemu-args "-redir tcp:10022:10.0.2.15:22"
> >
> > In QEMU:
> > setenv bootargs 'console=ttyPS0,115200'; tftpboot 0x10000000  image.ub;
> > bootm
> >
> > tftp -g 10.0.2.2 jailhouse.tar.gz -r jailhouse.tar.gz
> >
> > tar zvxf jailhouse.tar.gz
> >
> > mv tftpboot/jailhouse jailhouse
> >
> > mkdir -p  /lib/firmware
> >
> > cp ./jailhouse/lib/firmware/jailhouse.bin /lib/firmware
> >
> > insmod ./jailhouse/lib/modules/4.11.0-rc4/extra/driver/jailhouse.ko
> >
> > sleep 2
> >
> > jailhouse/usr/local/sbin/jailhouse enable /home/root/jailhouse/configs/
> > zynqmp-zcu102.cell
> >
> >
> > Thank you,
> >
> > Constantin
> >
> >
> > On Apr 19, 2017 23:30, "Edgar E. Iglesias" <edgar.iglesias@xilinx.com>
> > wrote:
> >
> >> On Wed, Apr 19, 2017 at 11:02:21AM +0200, Jan Kiszka wrote:
> >> > On 2017-04-19 10:45, Peter Maydell wrote:
> >> > > On 19 April 2017 at 05:37, Jan Kiszka <jan.kiszka@web.de> wrote:
> >> > >> On 2017-04-19 05:48, Constantin Petra wrote:
> >> > >>> So from some reason, under QEMU something is not in place, but what
> >> can
> >> > >>> that be? It would be convenient from our perspective, if it would
> >> work...
> >> > >>
> >> > >> I fully agree, and I'm waiting for official EL2 support in QEMU in
> >> order
> >> > >> to introduce a virtual target for ARM[64], just like we have on x86
> >> already.
> >> > >
> >> > > This has been in upstream QEMU for a little while now,
> >> > > and will be in the upcoming 2.9 release. (Applies only for
> >> > > AArch64 hosts using GICv3, needs some board support changes
> >> > > which have currently only been done for 'virt'.)
> >> >
> >> > Ah, v3 - oversaw this "minor" detail. Too bad that there is still no
> >> > GICv3 silicon in our hands that we could test against (do you know some
> >> > recommendable board?). I'm pretty sure Jailhouse's GICv3 support is
> >> > broken by now (due to lacking tests), plus it is not yet enabled for our
> >> > ARM64 port. But I'm a bit reluctant to enable this only over QEMU...
> >>
> >> Hi,
> >>
> >> Xilinx QEMU has support for the virtualization extensions in the GICv2 but
> >> we're seeing issues with running hypervisors in multi-core setups.
> >> It seems like guest migration between cores is causing trouble but we
> >> haven't pinpointed the exact cause.
> >>
> >> I'd be interested in debugging the problem seen with Jailhouse.
> >> Can the images and instructions on howto run be shared with me?
> >>
> >> Thanks,
> >> Edgar
> >>
> >