[Qemu-devel] Cubietruck: cannot create KVM guests: "kvm_init_vcpu failed: Invalid argument"

[Qemu-devel] Cubietruck: cannot create KVM guests: "kvm_init_vcpu failed: Invalid argument"

[Kashyap Chamarthy]

Booting a minimal KVM guest throws the below error on Cubietruck:

  "kvm_init_vcpu failed: Invalid argument"

More context and an easy reproducer in this QEMU bug[1] for Fedora.

Context quoting Rich Jones from comment #2:

    "For some reason I thought this had been fixed upstream, but
    now that I've finally got my CT working again, I see that I
    am still carrying that patch in my custom qemu.
    
    diff --git a/target-arm/cpu.c b/target-arm/cpu.c
    index 5ce7350..04d69d1 100644
    --- a/target-arm/cpu.c
    +++ b/target-arm/cpu.c
    @@ -858,7 +858,7 @@ static void cortex_a15_initfn(Object *obj)
         set_feature(&cpu->env, ARM_FEATURE_DUMMY_C15_REGS);
         set_feature(&cpu->env, ARM_FEATURE_CBAR_RO);
         set_feature(&cpu->env, ARM_FEATURE_LPAE);
    -    cpu->kvm_target = QEMU_KVM_ARM_TARGET_CORTEX_A15;
    +    cpu->kvm_target = QEMU_KVM_ARM_TARGET_CORTEX_A7;
         cpu->midr = 0x412fc0f1;
         cpu->reset_fpsid = 0x410430f0;
         cpu->mvfr0 = 0x10110222;
    
    So that's the answer really, it's a qemu bug.  Actually it looks as
    if qemu contains some code to try to get the host CPU type, but it
    doesn't work, or maybe we need to pass a -cpu option ..."



  [1] https://bugzilla.redhat.com/show_bug.cgi?id=1171501 -- Cubietruck:
      cannot create KVM guests: "kvm_init_vcpu failed: Invalid argument"

-- 
/kashyap

Re: [Qemu-devel] Cubietruck: cannot create KVM guests: "kvm_init_vcpu failed: Invalid argument"

[Peter Maydell]

On 9 December 2014 at 10:50, Kashyap Chamarthy <kchamart@redhat.com> wrote:
> Booting a minimal KVM guest throws the below error on Cubietruck:
>
>   "kvm_init_vcpu failed: Invalid argument"
>
> More context and an easy reproducer in this QEMU bug[1] for Fedora.
>
> Context quoting Rich Jones from comment #2:
>
>     "For some reason I thought this had been fixed upstream, but
>     now that I've finally got my CT working again, I see that I
>     am still carrying that patch in my custom qemu.
>
>     diff --git a/target-arm/cpu.c b/target-arm/cpu.c
>     index 5ce7350..04d69d1 100644
>     --- a/target-arm/cpu.c
>     +++ b/target-arm/cpu.c
>     @@ -858,7 +858,7 @@ static void cortex_a15_initfn(Object *obj)
>          set_feature(&cpu->env, ARM_FEATURE_DUMMY_C15_REGS);
>          set_feature(&cpu->env, ARM_FEATURE_CBAR_RO);
>          set_feature(&cpu->env, ARM_FEATURE_LPAE);
>     -    cpu->kvm_target = QEMU_KVM_ARM_TARGET_CORTEX_A15;
>     +    cpu->kvm_target = QEMU_KVM_ARM_TARGET_CORTEX_A7;
>          cpu->midr = 0x412fc0f1;
>          cpu->reset_fpsid = 0x410430f0;
>          cpu->mvfr0 = 0x10110222;

This is obviously a bogus patch -- this is the initfn
for Cortex-A15 so telling it to be a Cortex-A7 is wrong
(and would break working setups on A15 hosts).

>     So that's the answer really, it's a qemu bug.  Actually it looks as
>     if qemu contains some code to try to get the host CPU type, but it
>     doesn't work, or maybe we need to pass a -cpu option ..."

Yes, you need to pass a -cpu option. For KVM on ARM, you
always need to either:
 * pass a -cpu option matching the host CPU
 * pass "-cpu host"

The CubieTruck is a Cortex-A7, which we don't have specific
support for in QEMU, so you will need "-cpu host".

thanks
-- PMM

Re: [Qemu-devel] Cubietruck: cannot create KVM guests: "kvm_init_vcpu failed: Invalid argument"

[Richard W.M. Jones]

On Tue, Dec 09, 2014 at 11:50:31AM +0100, Kashyap Chamarthy wrote:
> Booting a minimal KVM guest throws the below error on Cubietruck:
> 
>   "kvm_init_vcpu failed: Invalid argument"
> 
> More context and an easy reproducer in this QEMU bug[1] for Fedora.
> 
> Context quoting Rich Jones from comment #2:
> 
>     "For some reason I thought this had been fixed upstream, but
>     now that I've finally got my CT working again, I see that I
>     am still carrying that patch in my custom qemu.
>     
>     diff --git a/target-arm/cpu.c b/target-arm/cpu.c
>     index 5ce7350..04d69d1 100644
>     --- a/target-arm/cpu.c
>     +++ b/target-arm/cpu.c
>     @@ -858,7 +858,7 @@ static void cortex_a15_initfn(Object *obj)
>          set_feature(&cpu->env, ARM_FEATURE_DUMMY_C15_REGS);
>          set_feature(&cpu->env, ARM_FEATURE_CBAR_RO);
>          set_feature(&cpu->env, ARM_FEATURE_LPAE);
>     -    cpu->kvm_target = QEMU_KVM_ARM_TARGET_CORTEX_A15;
>     +    cpu->kvm_target = QEMU_KVM_ARM_TARGET_CORTEX_A7;
>          cpu->midr = 0x412fc0f1;
>          cpu->reset_fpsid = 0x410430f0;
>          cpu->mvfr0 = 0x10110222;
>     
>     So that's the answer really, it's a qemu bug.  Actually it looks as
>     if qemu contains some code to try to get the host CPU type, but it
>     doesn't work, or maybe we need to pass a -cpu option ..."

Kashyap,

Can you try modifying src/launch-direct.c to see if we can pass

  -cpu cortex-a7

and if that fixes the problem?

If that does solve the problem, the question becomes how to detect the
right CPU (either cortex-a7, cortex-a15, cortex-a57, ...)  This
information isn't easily available to libguestfs AFAIK.

I really think that qemu should just "do the right thing" though.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

Re: [Qemu-devel] Cubietruck: cannot create KVM guests: "kvm_init_vcpu failed: Invalid argument"

[Richard W.M. Jones]

On Tue, Dec 09, 2014 at 10:53:41AM +0000, Peter Maydell wrote:
> On 9 December 2014 at 10:50, Kashyap Chamarthy <kchamart@redhat.com> wrote:
> > Booting a minimal KVM guest throws the below error on Cubietruck:
> >
> >   "kvm_init_vcpu failed: Invalid argument"
> >
> > More context and an easy reproducer in this QEMU bug[1] for Fedora.
> >
> > Context quoting Rich Jones from comment #2:
> >
> >     "For some reason I thought this had been fixed upstream, but
> >     now that I've finally got my CT working again, I see that I
> >     am still carrying that patch in my custom qemu.
> >
> >     diff --git a/target-arm/cpu.c b/target-arm/cpu.c
> >     index 5ce7350..04d69d1 100644
> >     --- a/target-arm/cpu.c
> >     +++ b/target-arm/cpu.c
> >     @@ -858,7 +858,7 @@ static void cortex_a15_initfn(Object *obj)
> >          set_feature(&cpu->env, ARM_FEATURE_DUMMY_C15_REGS);
> >          set_feature(&cpu->env, ARM_FEATURE_CBAR_RO);
> >          set_feature(&cpu->env, ARM_FEATURE_LPAE);
> >     -    cpu->kvm_target = QEMU_KVM_ARM_TARGET_CORTEX_A15;
> >     +    cpu->kvm_target = QEMU_KVM_ARM_TARGET_CORTEX_A7;
> >          cpu->midr = 0x412fc0f1;
> >          cpu->reset_fpsid = 0x410430f0;
> >          cpu->mvfr0 = 0x10110222;
> 
> This is obviously a bogus patch -- this is the initfn
> for Cortex-A15 so telling it to be a Cortex-A7 is wrong
> (and would break working setups on A15 hosts).

Yes, I definitely was not proposing this as a serious patch!  In my
copy of qemu it has "HACK" in all capital letters in the summary ...

> >     So that's the answer really, it's a qemu bug.  Actually it looks as
> >     if qemu contains some code to try to get the host CPU type, but it
> >     doesn't work, or maybe we need to pass a -cpu option ..."
> 
> Yes, you need to pass a -cpu option. For KVM on ARM, you
> always need to either:
>  * pass a -cpu option matching the host CPU
>  * pass "-cpu host"
> 
> The CubieTruck is a Cortex-A7, which we don't have specific
> support for in QEMU, so you will need "-cpu host".

Kashyap ^^ can you try this?  Should be a trivial one-liner change
in src/launch-direct.c.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html

Re: [Qemu-devel] Cubietruck: cannot create KVM guests: "kvm_init_vcpu failed: Invalid argument"

[Peter Maydell]

On 9 December 2014 at 11:25, Richard W.M. Jones <rjones@redhat.com> wrote:
> I really think that qemu should just "do the right thing" though.

Tricky, because you have to use the same CPU as the host if
you want to use KVM, and that conflicts with wanting to do the
same thing on all host platforms for a given set of command line
options. If you have some concrete suggestions for improving
things that don't break backwards-compatibility I'm happy to
hear them, though.

-- PMM

Re: [Qemu-devel] Cubietruck: cannot create KVM guests: "kvm_init_vcpu failed: Invalid argument"

[Kashyap Chamarthy]

On Tue, Dec 09, 2014 at 11:27:02AM +0000, Richard W.M. Jones wrote:
> On Tue, Dec 09, 2014 at 10:53:41AM +0000, Peter Maydell wrote:
> > On 9 December 2014 at 10:50, Kashyap Chamarthy <kchamart@redhat.com> wrote:
> > > Booting a minimal KVM guest throws the below error on Cubietruck:
> > >
> > >   "kvm_init_vcpu failed: Invalid argument"
> > >
> > > More context and an easy reproducer in this QEMU bug[1] for Fedora.
> > >
> > > Context quoting Rich Jones from comment #2:
> > >
> > >     "For some reason I thought this had been fixed upstream, but
> > >     now that I've finally got my CT working again, I see that I
> > >     am still carrying that patch in my custom qemu.
> > >
> > >     diff --git a/target-arm/cpu.c b/target-arm/cpu.c
> > >     index 5ce7350..04d69d1 100644
> > >     --- a/target-arm/cpu.c
> > >     +++ b/target-arm/cpu.c
> > >     @@ -858,7 +858,7 @@ static void cortex_a15_initfn(Object *obj)
> > >          set_feature(&cpu->env, ARM_FEATURE_DUMMY_C15_REGS);
> > >          set_feature(&cpu->env, ARM_FEATURE_CBAR_RO);
> > >          set_feature(&cpu->env, ARM_FEATURE_LPAE);
> > >     -    cpu->kvm_target = QEMU_KVM_ARM_TARGET_CORTEX_A15;
> > >     +    cpu->kvm_target = QEMU_KVM_ARM_TARGET_CORTEX_A7;
> > >          cpu->midr = 0x412fc0f1;
> > >          cpu->reset_fpsid = 0x410430f0;
> > >          cpu->mvfr0 = 0x10110222;
> > 
> > This is obviously a bogus patch -- this is the initfn
> > for Cortex-A15 so telling it to be a Cortex-A7 is wrong
> > (and would break working setups on A15 hosts).
> 
> Yes, I definitely was not proposing this as a serious patch!  In my
> copy of qemu it has "HACK" in all capital letters in the summary ...
> 
> > >     So that's the answer really, it's a qemu bug.  Actually it looks as
> > >     if qemu contains some code to try to get the host CPU type, but it
> > >     doesn't work, or maybe we need to pass a -cpu option ..."
> > 
> > Yes, you need to pass a -cpu option. For KVM on ARM, you
> > always need to either:
> >  * pass a -cpu option matching the host CPU
> >  * pass "-cpu host"
> > 
> > The CubieTruck is a Cortex-A7, which we don't have specific
> > support for in QEMU, so you will need "-cpu host".
> 
> Kashyap ^^ can you try this?  Should be a trivial one-liner change
> in src/launch-direct.c.

Yeah, libguestfs build in progress on Cubietruck (it's slow), will test.

-- 
/kashyap

Re: [Qemu-devel] Cubietruck: cannot create KVM guests: "kvm_init_vcpu failed: Invalid argument"

[Dr. David Alan Gilbert]

* Peter Maydell (peter.maydell@linaro.org) wrote:
> On 9 December 2014 at 11:25, Richard W.M. Jones <rjones@redhat.com> wrote:
> > I really think that qemu should just "do the right thing" though.
> 
> Tricky, because you have to use the same CPU as the host if
> you want to use KVM

That's an unusally strict requirement; is there no concept
of a common subset that will work on multiple CPUs?

Is that still true for v8? I can see it making it tricky for users
with piles of newer/older hosts.

Dave

> and that conflicts with wanting to do the
> same thing on all host platforms for a given set of command line
> options. If you have some concrete suggestions for improving
> things that don't break backwards-compatibility I'm happy to
> hear them, though.
> 
> -- PMM
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] Cubietruck: cannot create KVM guests: "kvm_init_vcpu failed: Invalid argument"

[Peter Maydell]

On 9 December 2014 at 12:33, Dr. David Alan Gilbert <dgilbert@redhat.com> wrote:
> * Peter Maydell (peter.maydell@linaro.org) wrote:
>> On 9 December 2014 at 11:25, Richard W.M. Jones <rjones@redhat.com> wrote:
>> > I really think that qemu should just "do the right thing" though.
>>
>> Tricky, because you have to use the same CPU as the host if
>> you want to use KVM
>
> That's an unusally strict requirement; is there no concept
> of a common subset that will work on multiple CPUs?
>
> Is that still true for v8? I can see it making it tricky for users
> with piles of newer/older hosts.

It's a limitation of the kernel implementation -- we don't
attempt to trap things like ID registers, so what the guest
sees is just the host CPU. The architecture provides the means
to trap these and the KVM API allows userspace to ask the
kernel to provide an emulation specifically of a particular
CPU. But the kernel will currently always refuse to emulate
anything other than the host CPU type it's running on.

thanks
-- PMM

Re: [Qemu-devel] Cubietruck: cannot create KVM guests: "kvm_init_vcpu failed: Invalid argument"

[Kashyap Chamarthy]

On Tue, Dec 09, 2014 at 11:27:02AM +0000, Richard W.M. Jones wrote:
> On Tue, Dec 09, 2014 at 10:53:41AM +0000, Peter Maydell wrote:
> > On 9 December 2014 at 10:50, Kashyap Chamarthy <kchamart@redhat.com> wrote:
> > > Booting a minimal KVM guest throws the below error on Cubietruck:
> > >
> > >   "kvm_init_vcpu failed: Invalid argument"

[. . .]

> > >     So that's the answer really, it's a qemu bug.  Actually it looks as
> > >     if qemu contains some code to try to get the host CPU type, but it
> > >     doesn't work, or maybe we need to pass a -cpu option ..."
> > 
> > Yes, you need to pass a -cpu option. For KVM on ARM, you
> > always need to either:
> >  * pass a -cpu option matching the host CPU
> >  * pass "-cpu host"
> > 
> > The CubieTruck is a Cortex-A7, which we don't have specific
> > support for in QEMU, so you will need "-cpu host".
> 
> Kashyap ^^ can you try this?  Should be a trivial one-liner change
> in src/launch-direct.c.

Yes it works with it :-)

With the below diff in `libguestfs`:

    $ git diff src/launch.c
    diff --git a/src/launch.c b/src/launch.c
    index 9fadce8..ce71a8e 100644
    --- a/src/launch.c
    +++ b/src/launch.c
    @@ -400,7 +400,7 @@ const char *
     guestfs___get_cpu_model (int kvm)
     {
     #if defined(__arm__)            /* 32 bit ARM. */
    -  return NULL;
    +  return "host";
     
     #elif defined(__aarch64__)
       /* With -M virt, the default -cpu is cortex-a15.  Stupid. */


I can boot a KVM guest successfully on Cubietruck, invoked via
libguestfs appliance:

. . .
[01433ms] /bin/qemu-system-arm \
    -global virtio-blk-device.scsi=off \
    -nodefconfig \
    -enable-fips \
    -nodefaults \
    -display none \
    -M vexpress-a15 \
    -cpu host \
    -machine accel=kvm:tcg \
    -m 500 \
    -no-reboot \
    -rtc driftfix=slew \
    -global kvm-pit.lost_tick_policy=discard \
    -kernel /home/kashyapc/src/libguestfs/tmp/.guestfs-1000/appliance.d/kernel \
    -dtb /home/kashyapc/src/libguestfs/tmp/.guestfs-1000/appliance.d/dtb \
    -initrd /home/kashyapc/src/libguestfs/tmp/.guestfs-1000/appliance.d/initrd \
    -device virtio-scsi-device,id=scsi \
    -drive file=/home/kashyapc/src/libguestfs/tmp/libguestfsRMfHzO/scratch.1,cache=unsafe,format=raw,id=hd0,if=none \
    -device scsi-hd,drive=hd0 \
    -drive file=/home/kashyapc/src/libguestfs/tmp/.guestfs-1000/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none \
    -device scsi-hd,drive=appliance \
    -device virtio-serial-device \
    -serial stdio \
    -chardev socket,path=/home/kashyapc/src/libguestfs/tmp/libguestfsRMfHzO/guestfsd.sock,id=channel0 \
    -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
    -append 'panic=1 mem=500M console=ttyAMA0 udevtimeout=6000 udev.event-timeout=6000 no_timer_check acpi=off printk.time=1 cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=vt102'
. . .


Thanks, Rich & Peter.

-- 
/kashyap