Re: KVM git hangs with if=virtio (works under kvm 0.12.3)

[ewheeler <kvm@ew.ewheeler.org>]

On Wed, 2010-07-07 at 10:03 +0100, Stefan Hajnoczi wrote:
> 2010/7/5 Gleb Natapov <gleb@redhat.com>:
> > On Mon, Jul 05, 2010 at 01:36:08PM +0100, Stefan Hajnoczi wrote:
> >> 2010/7/5 Gleb Natapov <gleb@redhat.com>:
> >> > On Mon, Jul 05, 2010 at 01:11:25PM +0100, Stefan Hajnoczi wrote:
> >> >> On Fri, Jul 2, 2010 at 9:07 AM, Stefan Hajnoczi <stefanha@gmail.com> wrote:
> >> >> > On Fri, Jul 2, 2010 at 4:31 AM, ewheeler <kvm@ew.ewheeler.org> wrote:
> >> >> >> Hello all,
> >> >> >>
> >> >> >> I'm booting a CentOS kernel under today's KVM git and it hangs after
> >> >> >> initializing the serial port when the drive if=virtio, but not when
> >> >> >> drive if=ide.  Look close---this is not a "forgot to add virtio_blk"
> >> >> >> problem.  If I use 0.12.3 from Ubuntu 10.04 it works properly.
> >> >> >>
> >> >> >> Reproduction:
> >> >> >>
> >> >> >> Using kvm 0.12.3 on ubuntu 10.04 (1:84+dfsg-0ubuntu16+0.12.3+noroms
> >> >> >> +0ubuntu9) it will work properly:
> >> >> >>
> >> >> >>  qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \
> >> >> >>                        -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus
> >> >> >>
> >> >> >> As expected, the kernel panics unable to mount root (good-boot.png).
> >> >> >> This makes sense, as "dummy-disk-image" is 1MB of 0x00 bytes.
> >> >> >>
> >> >> >> ---However---if I use today's git (2010-07-01) of kvm:
> >> >> >>
> >> >> >>   /usr/local/kvm-git/bin/qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \
> >> >> >>        -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus
> >> >> >>
> >> >> >> This hangs just after initializing the Serial device (obtained by adding
> >> >> >> -serial stdio -append console=ttyS0):
> >> >> >>
> >> >> >> Note that this only happens with the disk interface set to virtio
> >> >> >> (if=virtio).  It works fine for ide (if=ide).
> >> >> >>
> >> >> >>
> >> >> >> Am I doing something wrong here?
> >> >> >> Is anyone else having this problem?
> >> >> >
> >> >> > I have seen this issue with a RHEL 5.5 guest running under
> >> >> > qemu-kvm.git.  It boots a new guest fine but hangs as you described
> >> >> > with the RHEL 5.5 kernel.  I have not investigated.
> >> >>
> >> >> This issue is affected by extboot, a feature that enables booting from
> >> >> virtio-blk devices.  I have just sent a patch to the KVM mailing list
> >> >> to restore extboot functionality which has been broken in
> >> >> qemu-kvm.git.  That patch can be used to work around this issue by
> >> >> using "-drive ...,boot=on" but it doesn't explain why the RHEL 5.5
> >> >> kernel hangs during serial initialization when extboot is not present.
> >> >>
> >> > Hang that happens during guest boot (after bootloader started the
> >> > kernel) cannot be worked around by extboot. extboot is also not needed
> >> > with latest qemu git to boot from virtio disks since the support for
> >> > that is in the bios now.
> >>
> >> I agree that something else is going on here and needs to be
> >> investigated, but I do think that extboot can indirectly affect the
> >> guest boot.
> >>
> >> With extboot the virtio-blk PCI adapter is not touched by the
> >> firmware/bootloader.  Is it possible that a virtio-blk interrupt is
> >> raised and not acknowledged before entering Linux.  When Linux brings
> >> up the serial port it gets swamped with interrupts?  That's just a
> >> guess.
> >>
> > That is possible when bios is actually used to boot the guest, but bug
> > reporter uses -kernel option so no bios boot code should run at all.
> > Virtio is initialized anyway, but this will happen with boot=on too.
> 
> Okay, I got to the bottom of this.  Here's the story, see bottom of the mail for
> the solution and workarounds:
> 
> It turns out that -kernel does involve the BIOS.  KVM pulls apart the bzImage
> and makes it available via the fw_cfg interface.  The linuxboot.bin option ROM
> is executed by the BIOS inside the VM to actually jump into the kernel.  This
> means the BIOS does POST and sets itself up; the kernel's real-mode boot
> code is going to use BIOS interrupts.
> 
> So now the VM has booted, BIOS finished POST and executed linuxboot.bin,
> linuxboot.bin transferred control to Linux.  Then, in Linux arch/x86/boot/edd.c
> a disk read to sector=0 bytes=512 is made using INT 13h.  Since this disk
> read comes from the Linux kernel, it happens regardless of -kernel or not.
> 
> The disk read is serviced by the BIOS.  Older versions of SeaBIOS leave the
> interrupt raised here, so then the kernel hangs in serial initialization later.
> 
> However, there is a simple workaround:
> 
>   x86_64-softmmu/qemu-system-x86_64 -m 512 -drive
> file=~/rhel5u5.img,if=virtio -kernel /boot/vmlinuz-2.6.32-5-amd64
> -append edd=skipmbr
> 
> When the edd=skipmbr kernel parameter is used, the kernel will not perform
> the disk read and the interrupt will never get stuck.  The VM boots
> successfully.

Wow, brilliant.  Thank you for jumping into this so quickly!

-Eric