efi Grub2 of Ubuntu 10.10 x64 fails to execute on qemu with -enable-kvm

efi Grub2 of Ubuntu 10.10 x64 fails to execute on qemu with -enable-kvm

[Adhyas Avasthi]

[-- Attachment #1: Type: text/plain, Size: 2508 bytes --]

I am trying to run Ubuntu 10.10 x64 on qemu with OVMF EFI BIOS. Everything
is fresh off the tree. While I can boot into the Bootloader and the
bootloader can boot into OS if I have disabled kvm using -no-kvm option when
I start the VM.
The qemu system crashes with the following error if I enable kvm in the VM.

!!!! X64 Exception Type - 000000000000000D !!!!
ExceptionData - 0000000000000000
RIP - 000000001FFA937A, RFL - 0000000000010206
RAX - 000000001FF351C0, RCX - 0000000000000000, RDX - 000000001FFBB1B0
RBX - 000000001FF35400, RSP - 000000001FF97540, RBP - 000000001FFBB1B0
RSI - 000000001FFBB1F0, RDI - 000000001FF35200
R8  - 0000000000000000, R9  - 000000001FFBB1EF, R10 - 000000001E5E1728
R11 - 000000001FF973D8, R12 - 000000001FFB6810, R13 - 0000000000000070
R14 - 000000001DDBAFFF, R15 - 0000000000000060
CS  - 0028, DS  - 0008, ES  - 0008, FS  - 0008, GS  - 0008, SS  - 0008
GDT - 000000001FF1CE98; 003F,                   IDT - 000000001FE88BC0; 0FFF
LDT - 0000000000000000, TR  - 0000000000000000
CR0 - 0000000080000023, CR2 - 0000000000000000, CR3 - 000000001FF36000
CR4 - 0000000000000668, CR8 - 0000000000000000
DR0 - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000000000000
DR3 - 0000000000000000, DR6 - 00000000FFFF0FF0, DR7 - 0000000000000400

This is a pain, because without the kvm the installation and booting is
really really slow. I tried to debug it from the EFI BIOS a little bit and
it seems that the boot loader crashed with this exception immediately after
it was started.
Other developers on OVMF and EDK2 development (EFI BIOS) have also pointed
out that in order to use Ubuntu 10.10 with EFI, you have to disable kvm.
This seems to be an issue.

If there are any pointers (or any other debug help you want from me) in
order to help root cause the issue, that would be helpful. Also, I can drop
into the EFI Shell and start the bootloader efi image manually, which
results in the same crash. Can I pass some parameters to the boot loader
image to get some more information about the crash or the boot loader
progress before the crash?

I have downloaded the Grub2 sources and am trying to look into this issue.
If someone has been able to reproduce the issue, do let me know.

-- 
Adhyas
********************************************************************
Two types have compatible type if their types are the same.
    — ANSI C Standard, 3.1.2.6.
********************************************************************

[-- Attachment #2: Type: text/html, Size: 2596 bytes --]

Re: efi Grub2 of Ubuntu 10.10 x64 fails to execute on qemu with -enable-kvm

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 3880 bytes --]

On 11/11/2010 04:33 AM, Adhyas Avasthi wrote:
> I am trying to run Ubuntu 10.10 x64 on qemu with OVMF EFI BIOS.
> Everything is fresh off the tree. While I can boot into the Bootloader
> and the bootloader can boot into OS if I have disabled kvm using
> -no-kvm option when I start the VM.
> The qemu system crashes with the following error if I enable kvm in
> the VM.
KVM like any kind of acceleration is imprecise. Because of it results of
testing under kvm not always reflect the real state of things. For
testing bootloader capabilities speed doesn't matter. When testing
userspace firmware shouldn't matter and if you boot using
coreboot-on-qemu+grub or grub-as-qemu-firmware you can check for
firmware independance (you need to boot in graphics mode).
EFI is only a pain. While it's important to check that GRUB works with
it on real machines and faithful emulators, there is really no point in
using it when you don't need to.
It's better to just use SeaBIOS with qemu if you need to use Ubuntu in VM.
This being said I was able to load Linux amd64 on kvm 0.12.5 using
64-bit tianocore and 1.99~beta0 GRUB on Intel(R) Core(TM) i5 CPU       M
540  @ 2.53GHz.
While if someone figures it out and offers a sane workaround (not a
"fix", it's not our bug, but inherent disadvantage of kvm) it can be
accepted, there is no point in spending time on it

>
> !!!! X64 Exception Type - 000000000000000D !!!!
> ExceptionData - 0000000000000000
> RIP - 000000001FFA937A, RFL - 0000000000010206
> RAX - 000000001FF351C0, RCX - 0000000000000000, RDX - 000000001FFBB1B0
> RBX - 000000001FF35400, RSP - 000000001FF97540, RBP - 000000001FFBB1B0
> RSI - 000000001FFBB1F0, RDI - 000000001FF35200
> R8  - 0000000000000000, R9  - 000000001FFBB1EF, R10 - 000000001E5E1728
> R11 - 000000001FF973D8, R12 - 000000001FFB6810, R13 - 0000000000000070
> R14 - 000000001DDBAFFF, R15 - 0000000000000060
> CS  - 0028, DS  - 0008, ES  - 0008, FS  - 0008, GS  - 0008, SS  - 0008
> GDT - 000000001FF1CE98; 003F,                   IDT -
> 000000001FE88BC0; 0FFF
> LDT - 0000000000000000, TR  - 0000000000000000
> CR0 - 0000000080000023, CR2 - 0000000000000000, CR3 - 000000001FF36000
> CR4 - 0000000000000668, CR8 - 0000000000000000
> DR0 - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000000000000
> DR3 - 0000000000000000, DR6 - 00000000FFFF0FF0, DR7 - 0000000000000400
>
> This is a pain, because without the kvm the installation and booting
> is really really slow. I tried to debug it from the EFI BIOS a little
> bit and it seems that the boot loader crashed with this exception
> immediately after it was started.
> Other developers on OVMF and EDK2 development (EFI BIOS) have also
> pointed out that in order to use Ubuntu 10.10 with EFI, you have to
> disable kvm. This seems to be an issue.
>
> If there are any pointers (or any other debug help you want from me)
> in order to help root cause the issue, that would be helpful. Also, I
> can drop into the EFI Shell and start the bootloader efi image
> manually, which results in the same crash. Can I pass some parameters
> to the boot loader image to get some more information about the crash
> or the boot loader progress before the crash?
>
> I have downloaded the Grub2 sources and am trying to look into this
> issue. If someone has been able to reproduce the issue, do let me know.
>
> -- 
> Adhyas
> ********************************************************************
> Two types have compatible type if their types are the same.
>     — ANSI C Standard, 3.1.2.6.
> ********************************************************************
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel
>   


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: efi Grub2 of Ubuntu 10.10 x64 fails to execute on qemu with -enable-kvm

[Adhyas Avasthi]

I get your point, but that just stops "some" people from using Grub2
for EFI Development work and to look for other options. Unfortunately,
people who also want to boot regular OSes on their EFI BIOSes (besides
their own OSes which they want to convert to pure EFI) suffer because
regular OSes (at least Ubuntu) have chosen Grub 2 as their platform,
and a lot of companies out their do wish to run Ubuntu in a VM. Some
of my clients wish to develop certain EFI applications or run time OS
applications using EFI, for which they have chosen Virtual Machines as
their development platforms and wish to boot to Ubuntu on EFI on qemu.
Without KVM, the guest is really not usable as it is very very slow.
They do have their own BIOS ports and expose certain vendor specific
Run Time services to use and expose to the users through the custom
apps.

Let me see if I can root cause the bug (err, workaround/issue) myself.
I do need to make it work for the client as it is their requirement to
boot a Linux OS in a VM on pure EFI world, so practically it is an
issue whether we as a community choose to acknowledge it or not. As of
now, KVM does execute all other x64 bit code successfully (including
the entire 64-bit kernel), so I would put a little bit more faith on
KVM. Let me see why I crash using kvm on qemu-kvm-0.13.0. My host
kernel is 2.6.32 but that should not matter because I have updated my
kvm-kmod for that kernel anyway. Am compiling a new kernel to see if
the issue still exists for 2.6.36 kernel.

BTW, when you say you loaded Grub2, what does that mean? Does it mean
you were able to boot to the OS as well? Or just see the Grub menu
screen or command prompt? I am just curious what works with 0.12.5 and
I can try to put my Ubuntu DVD image as well to see if I can match
what you have. Are there any command line parameters I can pass to the
EFI Grub2 boot loader when I invoke it manually from the EFI Shell, to
see any more information about the crash? Also, I haven't yet fully
understood how to enable more debug (EFI BIOS dumps all debug I want
on the serial port) from within the boot loader if I build it myself,
any pointers on that will be useful.

PS: I would still appreciate if I can ask questions on this mailing
list while I debug this issue, and I can get some help where I am
blocked.

Thanks,
Adhyas

> KVM like any kind of acceleration is imprecise. Because of it results of
> testing under kvm not always reflect the real state of things. For
> testing bootloader capabilities speed doesn't matter. When testing
> userspace firmware shouldn't matter and if you boot using
> coreboot-on-qemu+grub or grub-as-qemu-firmware you can check for
> firmware independance (you need to boot in graphics mode).
> EFI is only a pain. While it's important to check that GRUB works with
> it on real machines and faithful emulators, there is really no point in
> using it when you don't need to.
> It's better to just use SeaBIOS with qemu if you need to use Ubuntu in VM.
> This being said I was able to load Linux amd64 on kvm 0.12.5 using
> 64-bit tianocore and 1.99~beta0 GRUB on Intel(R) Core(TM) i5 CPU       M
> 540  @ 2.53GHz.
> While if someone figures it out and offers a sane workaround (not a
> "fix", it's not our bug, but inherent disadvantage of kvm) it can be
> accepted, there is no point in spending time on it
>

Re: efi Grub2 of Ubuntu 10.10 x64 fails to execute on qemu with -enable-kvm

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 4834 bytes --]

On 11/11/2010 07:54 PM, Adhyas Avasthi wrote:
> I get your point, but that just stops "some" people from using Grub2
> for EFI Development work and to look for other options.
If they are developpers and they care about GRUB supporting EFI+kvm,
it's a free software world, and patches are welcome. I just stated that
from where I sit (tram stop, and it rains) it has little point.
>  Unfortunately,
> people who also want to boot regular OSes on their EFI BIOSes
EFI isn't BIOS. EFI and BIOS are different kind of firmware.
What's the real point in running OS under X or Y firmware if once kernel
kicks in the difference is pretty small?
>  (besides
> their own OSes which they want to convert to pure EFI) suffer because
> regular OSes (at least Ubuntu) have chosen Grub 2 as their platform,
> and a lot of companies out their do wish to run Ubuntu in a VM. Some
> of my clients wish to develop certain EFI applications or run time OS
> applications using EFI,
What's the advantage over old good kernel modules? I fail to see any.
>  for which they have chosen Virtual Machines as
> their development platforms and wish to boot to Ubuntu on EFI on qemu.
> Without KVM, the guest is really not usable as it is very very slow.
> They do have their own BIOS ports and expose certain vendor specific
> Run Time services to use and expose to the users through the custom
> apps.
>
>   
Why not simply move the relevant code from firmware (EFI, BIOS and so
on) to the custom app itself? It will make the handling and upgrade of
relevant functions way easier.
> Let me see if I can root cause the bug (err, workaround/issue) myself.
> I do need to make it work for the client as it is their requirement to
> boot a Linux OS in a VM on pure EFI world, so practically it is an
> issue whether we as a community choose to acknowledge it or not. 
The issue itself is obviously acknowledged but
a) The root cause is probably burried deep into the way CPU handles Blue
Pill.
b) Priority is low (your priorities don't have to be the same as ours)
> As of
> now, KVM does execute all other x64 bit code successfully (including
> the entire 64-bit kernel), so I would put a little bit more faith on
> KVM. 
Only because it's optimised for this code to function. Qemu (w/o kqemu
or kvm) is the most accurate free emulator and second-best if you
consider non-free as well. But even qemu is completely accurate. As an
example I'm pretty sure that our video_cirrus which is coded for qemu
won't work with real Cirrus card as it is now. Also I spoke with
Carl-Daniel (coreboot dev and flashrom maintainer) and he agreed with it
and added that qemu even being very good has a number of flaws.
> Let me see why I crash using kvm on qemu-kvm-0.13.0. My host
> kernel is 2.6.32 but that should not matter because I have updated my
> kvm-kmod for that kernel anyway. Am compiling a new kernel to see if
> the issue still exists for 2.6.36 kernel.
>
> BTW, when you say you loaded Grub2, what does that mean? 
It means that I was able to load GRUB, have its shell, load Linux which
panic'ed because it couldn't find root (understandable since I had no
root= option or viable root partition at all)
> Does it mean
> you were able to boot to the OS as well? Or just see the Grub menu
> screen or command prompt? I am just curious what works with 0.12.5 and
>   
You can have my ROM images at
http://download-mirror.savannah.gnu.org/releases/grub/phcoder/rom.tar.lzma
Linux used comes from some debian package, can't say which one.
I suggest to first compare the CPUs, especially the extended page table
capability
> I can try to put my Ubuntu DVD image as well to see if I can match
> what you have. Are there any command line parameters I can pass to the
> EFI Grub2 boot loader when I invoke it manually from the EFI Shell, to
> see any more information about the crash?
No but you can embed the config consisting of "debug=all"
>  Also, I haven't yet fully
> understood how to enable more debug (EFI BIOS dumps all debug I want
> on the serial port) from within the boot loader if I build it myself,
> any pointers on that will be useful.
>
> PS: I would still appreciate if I can ask questions on this mailing
> list while I debug this issue, and I can get some help where I am
> blocked.
>
>   
You can ask the questions but a questions of kind "I'm stuck, can you
spend 5 hours on issue that is very low-priv for you?" isn't likely to
be answered. Be sure to get to the point, ideally producing a minimal
example which will give a motivation and make it easy to answer. For
more info refer:
http://all-nettools.com/toolbox/smart-whois.php

Vladimir, sitting at tram stop, but tram should come any second now

-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]