Re: Xen unstable fails to boot from cold on i386 core duo

["Daniel P. Berrange" <berrange@redhat.com>]

On Mon, Dec 17, 2007 at 03:42:21PM +0000, Daniel P. Berrange wrote:
> Just testing xen-unstable on some of my machines here. I've got a failure
> to boot on a Mac Mini with a i386 Core Duo cpu.
> 
> If I try to boot xen-unstable changeset 16606 from cold it will immediately
> crash with this:
> 
>   http://people.redhat.com/berrange/xen-3.2.0-fail.jpg
> 
> Sorry for lame camera pic - this being a mac mini there's no serial card,
> or even ability to add one in :-(
> 
> If I boot into Xen 3.1.0 it works fine. If I warm-reboot from Xen 3.1.0
> into Xen-unstable changeset 16606 it also works fine. So it seems only the
> boot from cold is problematic.
> 
> Attaching the 'xm dmesg' output from a succesfull warm-reboot into 
> xen-unstable, and the CPU info. 
> 
> Any ideas / suggestions ?

Converting ff16420d / vmx_cpu_up+0x25d address into a line number with 
eu-addr2line, it appears as if Xen is crashing & burning in the 'wrmsr' 
call in the 2nd half of this conditional from arch/x86/hvm/vmx/vmcs.c:

    if ( eax & IA32_FEATURE_CONTROL_MSR_LOCK )
    {
        if ( !(eax & (IA32_FEATURE_CONTROL_MSR_ENABLE_VMXON_OUTSIDE_SMX |
                      IA32_FEATURE_CONTROL_MSR_ENABLE_VMXON_INSIDE_SMX)) )
        {
            printk("CPU%d: VMX disabled by BIOS.\n", cpu);
            return 0;
        }
    }
    else
    {
        eax = (IA32_FEATURE_CONTROL_MSR_LOCK |
               IA32_FEATURE_CONTROL_MSR_ENABLE_VMXON_OUTSIDE_SMX |
               IA32_FEATURE_CONTROL_MSR_ENABLE_VMXON_INSIDE_SMX);
        wrmsr(IA32_FEATURE_CONTROL_MSR, eax, 0);
    }


In Xen 3.1.0 (which works for me), the 'vmx_cpu_up' method does not exist,
but the 'start_vmx' function which does more of less the same thing
writes a slightly different msr:

    if ( eax & IA32_FEATURE_CONTROL_MSR_LOCK )
    {
        if ( (eax & IA32_FEATURE_CONTROL_MSR_ENABLE_VMXON) == 0x0 )
        {
            printk("VMX disabled by Feature Control MSR.\n");
            return 0;
        }
    }
    else
    {
        wrmsr(IA32_FEATURE_CONTROL_MSR,
              IA32_FEATURE_CONTROL_MSR_LOCK |
              IA32_FEATURE_CONTROL_MSR_ENABLE_VMXON, 0);
    }


I guess this explains why Xen-unstable works if I warm re-boot from xen-3.1.0,
because the 3.1.0 hypervisor will have already enabled and lock VMX msr.

Is it possible that older CPUs / BIOS don't play nice with the extra bit

   IA32_FEATURE_CONTROL_MSR_ENABLE_VMXON_INSIDE_SMX

...if they don't have SMX support ?

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=|