confused with invept

confused with invept

[Bill Rieske]

I get the following extracted from dmesg when essentially modprobing kvm_intel.  Note: distro using kvm 78

loaded kvm module (kvm-78-9.3)
------------[ cut here ]------------
kernel BUG at /usr/src/packages/BUILD/kvm-78/suse-kernel/default/x86/kvm_main.c:1920!
invalid opcode: 0000 [1] SMP
last sysfs file: /sys/devices/pci0000:00/0000:00:1a.7/modalias
CPU 3
Modules linked in: kvm_intel(N+) kvm(N) sg usbhid hid ff_memless uhci_hcd ehci_hcd sd_mod crc_t10dif usbcore edd ext3 mbcache jbd fan thermal processor thermal_sys hwmon ide_pci_generic ide_core ata_generic pata_marvell ahci libata scsi_mod dock
Supported: No
Pid: 1405, comm: modprobe Tainted: G          2.6.27.11-1-default #1
RIP: 0010:[<ffffffffa016a1c2>]  [<ffffffffa016a1c2>] kvm_handle_fault_on_reboot+0xb/0xf [kvm]
RSP: 0018:ffff88012bc65ea0  EFLAGS: 00010246
RAX: ffff88012bc65ec8 RBX: ffffe20004b80940 RCX: 0000000000000002
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: 0000000000000004 R09: 0000000000000030
R10: ffff88012b94f000 R11: ffffffff80318014 R12: 00007fe82b5f0000
R13: 0000000000000000 R14: 000000000061a0e0 R15: 00007fe82b5f0000
FS:  00007fe82b5d66f0(0000) GS:ffff88012fba0ec0(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007fe82b602000 CR3: 000000012dcaa000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process modprobe (pid: 1405, threadinfo ffff88012bc64000, task ffff88012e17a2c0)
Stack:  ffffffffa011d4da 0000000000000001 ffff88012bc65ed0 ffffffff8022d4a9
 ffff88012bc65ef0 0000000000000000 0000000000000000 ffff880028088700
 ffffffffa019fe00 ffffffffa011d2ee ffffffff80209041 ffffffff802363e2
Call Trace:
 [<ffffffffa011d4da>] vmx_init+0x1ec/0x22d [kvm_intel]
 [<ffffffff80209041>] _stext+0x41/0x110
 [<ffffffff80261495>] sys_init_module+0xa0/0x1ba
 [<ffffffff8020bfbb>] system_call_fastpath+0x16/0x1b
 [<00007fe82b16276a>] 0x7fe82b16276a


Code: 08 74 d5 c3 48 c7 86 80 00 00 00 50 26 19 a0 31 c0 c3 48 c7 86 80 00 00 00 00 26 19 a0 31 c0 c3 80 3d ba 91 02 00 00 74 02 eb fe <0f> 0b eb fe 55 48 89 fd 53 31 db 48 83 ec 08 eb 15 48 63 c3 48
RIP  [<ffffffffa016a1c2>] kvm_handle_fault_on_reboot+0xb/0xf [kvm]
 RSP <ffff88012bc65ea0>
---[ end trace 9f1c8e8ecace5658 ]---

This is what I have derived from objdump -M x86_64,intel -Sl vmx.o
vmx_init is at 0x2ee  + 0x1ec is 4da
extracted from the objdump 
 4b5:   f6 05 00 00 00 00 04    test   BYTE PTR ds:0x0,0x4
 4bc:   74 4d                   je     50b <init_module+0x21d>
 4be:   48 c7 44 24 20 00 00    mov    QWORD PTR [rsp+32],0x0
 4c5:   00 00
 4c7:   48 c7 44 24 28 00 00    mov    QWORD PTR [rsp+40],0x0
 4ce:   00 00
 4d0:   48 8d 44 24 20          lea    rax,[rsp+32]
 4d5:   b9 02 00 00 00          mov    ecx,0x2
 4da:   66                      data16
 4db:   0f 38 80                (bad)
 4de:   08 77 02                or     BYTE PTR [rdi+2],dh
 4e1:   0f 0b                   ud2a
 4e3:   eb 26                   jmp    50b <init_module+0x21d>

which from what I gather is the instruction for INVEPT opcode 660f3880
from what I can tell from the code I shouldn't invoke this instruction if the bit isn't valid from 
       ept_sync_global();
:code snippets follow

static inline void ept_sync_global(void)
{
        if (cpu_has_vmx_invept_global())
                __invept(VMX_EPT_EXTENT_GLOBAL, 0, 0);
}

static inline int cpu_has_vmx_invept_global(void)
{
        return (!!(vmx_capability.ept & VMX_EPT_EXTENT_GLOBAL_BIT));
}

from what I gather it's faulting in the __invept or am I totally down the wrong path.
Any Ideas on whats going on would be helpful or how I should proceed.

Thanks

Re: confused with invept

[Sheng Yang]

On Fri, Jan 30, 2009 at 12:10:18PM -0700, Bill Rieske wrote:
> I get the following extracted from dmesg when essentially modprobing kvm_intel.  Note: distro using kvm 78

Hi Bill

Are you using Core i7 processor? Only it support EPT. What's your host
kernel version? And I think we can try the latest KVM module first.

> This is what I have derived from objdump -M x86_64,intel -Sl vmx.o
> vmx_init is at 0x2ee  + 0x1ec is 4da
> extracted from the objdump 
>  4b5:   f6 05 00 00 00 00 04    test   BYTE PTR ds:0x0,0x4
>  4bc:   74 4d                   je     50b <init_module+0x21d>
>  4be:   48 c7 44 24 20 00 00    mov    QWORD PTR [rsp+32],0x0
>  4c5:   00 00
>  4c7:   48 c7 44 24 28 00 00    mov    QWORD PTR [rsp+40],0x0
>  4ce:   00 00
>  4d0:   48 8d 44 24 20          lea    rax,[rsp+32]
>  4d5:   b9 02 00 00 00          mov    ecx,0x2
>  4da:   66                      data16
>  4db:   0f 38 80                (bad)
>  4de:   08 77 02                or     BYTE PTR [rdi+2],dh
>  4e1:   0f 0b                   ud2a
>  4e3:   eb 26                   jmp    50b <init_module+0x21d>
> 
> which from what I gather is the instruction for INVEPT opcode 660f3880
> from what I can tell from the code I shouldn't invoke this instruction if the bit isn't valid from 
>        ept_sync_global();

Sure. The bit should be filled by setup_vmcs_config() in vmx.c. I hope the
vmx_capability haven't been corrupted...

> from what I gather it's faulting in the __invept or am I totally down the wrong path.
> Any Ideas on whats going on would be helpful or how I should proceed.
> 

Please help to identify your processor, the value of vmx_capability.ept and
try latest KVM.

Thanks.

-- 
regards
Yang, Sheng	|Intel Opensource Technology Center

Re: confused with invept

[Bill Rieske]

>>> On 1/30/2009 at  8:18 PM, Sheng Yang <yasker@gmail.com> wrote: 
> On Fri, Jan 30, 2009 at 12:10:18PM -0700, Bill Rieske wrote:
>> I get the following extracted from dmesg when essentially modprobing 
> kvm_intel.  Note: distro using kvm 78
> 
> Hi Bill
> 
> Are you using Core i7 processor? Only it support EPT. What's your host
> kernel version? And I think we can try the latest KVM module first.
> 
I believe the processor is the Core i7.  
from cat /proc/cpuinfo
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Genuine Intel(R) CPU             000  @ 3.20GHz
stepping        : 4
cpuid level     : 11
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm ida

Kernel,
Linux 2.6.27.13-1-default #1 SMP 2009-01-27 13:41:16 +0100 x86_64 x86_64 x86_64 GNU/Linux

I will try the latest KVM.  

> Sure. The bit should be filled by setup_vmcs_config() in vmx.c. I hope the
> vmx_capability haven't been corrupted...
> 
>> from what I gather it's faulting in the __invept or am I totally down the 
> wrong path.
>> Any Ideas on whats going on would be helpful or how I should proceed.
>> 
> 
> Please help to identify your processor, the value of vmx_capability.ept and
> try latest KVM.
> 
> Thanks.
I will see if I can get some debug print in there to print the value out.

Thanks
Bill R.

Re: confused with invept

[Sheng Yang]

On Tue, Feb 3, 2009 at 2:09 AM, Bill Rieske <brieske@novell.com> wrote:
>
>>>> On 1/30/2009 at  8:18 PM, Sheng Yang <yasker@gmail.com> wrote:
>> On Fri, Jan 30, 2009 at 12:10:18PM -0700, Bill Rieske wrote:
>>> I get the following extracted from dmesg when essentially modprobing
>> kvm_intel.  Note: distro using kvm 78
>>
>> Hi Bill
>>
>> Are you using Core i7 processor? Only it support EPT. What's your host
>> kernel version? And I think we can try the latest KVM module first.
>>
> I believe the processor is the Core i7.
> from cat /proc/cpuinfo
> vendor_id       : GenuineIntel
> cpu family      : 6
> model           : 26
> model name      : Genuine Intel(R) CPU             000  @ 3.20GHz
> stepping        : 4
> cpuid level     : 11
> flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm ida
>
> Kernel,
> Linux 2.6.27.13-1-default #1 SMP 2009-01-27 13:41:16 +0100 x86_64 x86_64 x86_64 GNU/Linux
>
> I will try the latest KVM.

Strange. All i7 processor should support this instruction. And our all
nightly test based on i7, didn't encounter this before... Maybe you
can also try 2.6.28 host kernel, which provide EPT showing in
/proc/cpuinfo.

>> Sure. The bit should be filled by setup_vmcs_config() in vmx.c. I hope the
>> vmx_capability haven't been corrupted...
>>
>>> from what I gather it's faulting in the __invept or am I totally down the
>> wrong path.
>>> Any Ideas on whats going on would be helpful or how I should proceed.
>>>
>>
>> Please help to identify your processor, the value of vmx_capability.ept and
>> try latest KVM.
>>
>> Thanks.
> I will see if I can get some debug print in there to print the value out.

Thanks. Sorry for no more information, since I am still in Spring
Festival vacation now...

-- 
regards,
Yang, Sheng | Intel Opensource Technology Center