Re: KVM/ARM64: Booting KVM on FVP/fastmodels

[Andre Przywara <andre.przywara@arm.com>]

On Tue, 22 Jun 2021 02:33:44 +0000
胤桦 陈 <yinhua_chen@outlook.com> wrote:

Hi,

> I follow the document https://git.linaro.org/landing-teams/working/arm/arm-reference-platforms.git/about/docs/basefvp/user-guide.rst,

sorry for that ;-)

> and I build a Linux-on-FVP environment successfully. However, I still cannot boot KVM, the details are as follows:
> 1.I make sure that two configuration files enable KVM (I guess these are the right configuration files), they are
> layers/meta-arm/meta-arm-bsp/recipes-kernel/linux/files/tc0/defconfig:CONFIG_KVM=y
> build-poky/tmp-poky/work-shared/fvp-base/kernel-source/arch/arm64/configs/defconfig:CONFIG_KVM=y

> 2.I follow the document https://git.linaro.org/landing-teams/working/arm/arm-reference-platforms.git/about/docs/basefvp/user-guide.rst. After building the whole software stack, I try several commands but the results show that KVM does not work.

Have you tried actually running a guest? Using kvmtool, for instance?
kvmtool can be statically linked, so you end up with a single
executable file, easy to copy into whatever userland you use.
If you clone it from
git://git.kernel.org/pub/scm/linux/kernel/git/will/kvmtool.git
and do "make lkvm-static" on any arm64 system, you can copy the
resulting binary "lkvm-static" into your rootfs, and then run it on the
model:
# lkvm-static run -k <some_kernel_image>
You might need to create the device file first (see below).

> The screenshots are shown as follows[commands]I try three commands,
> "kvm-ok"

This does not do much on ARM/arm64, it just merely checks for the
existence of /dev/kvm. Depending on your userland, nobody might have
created that - but still KVM might be configured fine.
A simple "mknod /dev/kvm c 10 232" fixes that, but the test passing now
doesn't mean much (as is just checks for the existence of this file,
not actual KVM functionality).

> "dmesg | grep KVM"

Please always use "grep -i" when grep'ing in dmesg, capitalisation of
those names may vary. On my kernel there is indeed not a single hit for
"KVM", but several for "kvm".
Keep in mind that this is not a bullet-proof method: the boot messages
from KVM might have been flushed from the dmesg buffer already - on one
of my systems running for some weeks with some debug messages they
certainly are.
But right after boot there is a high chance of success, though, and
it's the easiest check for KVM.

> "grep -E -o 'vmx|svm' /proc/cpuinfo"

This is x86 specific and does not work at all on ARM/arm64.
There is not much to learn about KVM in cpuinfo on arm64.

> I guess maybe the reason is that FVP configuration uses a KVM-unsupport CPU, but I do not know the right configuration, and the FVP documents do not mention anything about KVM.

I doubt that, the normal models should all come with EL2 enabled. The
RevC model definitely works.

Cheers,
Andre

> And this is the FVP booting command in the scripts:
> 
> ===
> 
> /FVP_Base_RevC-2xAEMv8A
> 
>  -C pctl.startup=0.0.0.0
> 
>  -C bp.secure_memory=0
> 
>  -C cluster0.NUM_CORES=1
> 
>  -C cluster1.NUM_CORES=1
> 
>  -C cache_state_modelled=0
> 
>  -C bp.pl011_uart0.untimed_fifos=1
> 
>  -C bp.pl011_uart0.unbuffered_output=1
> 
>  -C bp.pl011_uart0.out_file=uart0-1624328932274439497.log
> 
>  -C bp.pl011_uart1.out_file=uart1-1624328932274439497.log
> 
>  -C bp.secureflashloader.fname=<workspace>/build-poky/tmp-poky/deploy/images/fvp-base/bl1-fvp.bin
> 
>  -C bp.flashloader0.fname=<workspace>/build-poky/tmp-poky/deploy/images/fvp-base/fip-fvp.bin
> 
>  -C bp.hostbridge.interfaceName=tap0
> 
>  -C bp.smsc_91c111.enabled=1
> 
>  --data cluster0.cpu0=<workspace>/build-poky/tmp-poky/deploy/images/fvp-base/Image@0x80080000
> 
>  --data cluster0.cpu0=<workspace>/build-poky/tmp-poky/deploy/images/fvp-base/fvp-base-gicv3-psci-custom.dtb@0x83000000
> 
>  -C bp.ve_sysregs.mmbSiteDefault=0
> 
>  -C bp.ve_sysregs.exit_on_shutdown=1
> 
>  -C bp.virtioblockdevice.image_path=<workspace>/build-poky/tmp-poky/deploy/images/fvp-base/core-image-minimal-fvp-base.disk.img
> 
> ===
> 
> 
> 
> If you can provide any kind of help, I appreciate it very much.
> 
> ________________________________
> From: Andre Przywara <andre.przywara@arm.com>
> Sent: Wednesday, April 14, 2021 16:50
> To: 胤桦 陈 <yinhua_chen@outlook.com>
> Cc: kvmarm@lists.cs.columbia.edu <kvmarm@lists.cs.columbia.edu>
> Subject: Re: KVM/ARM64: Booting KVM on FVP/fastmodels
> 
> On Wed, 14 Apr 2021 02:37:07 +0000
> 胤桦 陈 <yinhua_chen@outlook.com> wrote:
> 
> > Hi, Andre
> >
> > Do you mean that if I boot Linux on FVP, then I can run KVM on FVP? Since KVM is just a module in Linux?  
> 
> Well, on ARM it's not a kernel module, but built-in to the kernel
> image, but yes: You can run any normal KVM enabled kernel on
> it. So just use the same recipe you would use on any other system to
> run KVM guests.
> 
> Cheers,
> Andre
> 
> > ________________________________
> > From: André Przywara <andre.przywara@arm.com>
> > Sent: Tuesday, March 30, 2021 18:22
> > To: yinhua_chen@outlook.com <yinhua_chen@outlook.com>; kvmarm@lists.cs.columbia.edu <kvmarm@lists.cs.columbia.edu>
> > Subject: Re: KVM/ARM64: Booting KVM on FVP/fastmodels
> >
> > On 29/03/2021 13:13, yinhua_chen@outlook.com wrote:
> >
> > Hi,
> >  
> > > I want to boot KVM on simulators.
> > >
> > > I don't want to use QEMU since some features are only supported on
> > > FVP/fastmodels now.
> > >
> > > However, I cannot find documents/blogs/wiki about booting KVM on
> > > FVP/fastmodels.
> > >
> > > Does KVM support running on FVP?  
> >
> > The FVP emulates the full architecture, including all exception levels
> > (so also EL2), and the virtualization support of the GIC and the Generic
> > Timer.
> > So yes, the models fully support KVM (or any other hypervisor, for that
> > matter), and are in fact often used for developing KVM support of new
> > hardware features.
> >
> > So what are the problems that you are facing? Just running a kernel like
> > you would normally do on the model should work out of the box.
> > If you describe your setup and the error messages, we can probably help
> > you better.
> >
> > Cheers,
> > Andre
> >  
> > >
> > > Appreciate for your help!
> > >
> > > Sincerely,
> > > Yinhua
> > >
> > > _______________________________________________
> > > kvmarm mailing list
> > > kvmarm@lists.cs.columbia.edu
> > > https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
> > >  
> >  
> 

_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm