[Qemu-devel] Boot order problem

[Qemu-devel] Boot order problem

[Minoru Usui]

Hi, everyone

I'm in trouble about boot order of VM.
If anyone know cause of this problem, please let me know.

On following environment, I tried to boot from IDE CD-ROM device 
without inserting any bootable media, which is expected to fail,
but VM was booting up from virtio HDD which was not specified as bootable device.

  * host : RHEL6.1(x86_64)
    guest: RHEL6.1(x86_64)
  * VM has IDE CD-ROM and virtio HDD.
  * There is no bootable media in IDE CD-ROM.
  * RHEL6.1 is installed in virtio HDD
  * Only IDE CD-ROM was spcified as bootable device.
  * XML configuration of libvirt is below.
    I tested <boot dev> and <boot order> setting,
    but both are booting up from virtio HDD.
    -------------------------------------------------------
    [<boot dev> setting version]
      <os>
        <type arch='x86_64' machine='rhel6.1.0'>hvm</type>
        <boot dev='cdrom'/>
        <bootmenu enable='no'/>
      </os>

    [<boot order> setting version]
       <disk type='file' device='cdrom'>
         <driver name='qemu' type='raw'/>
         <target dev='hdc' bus='ide'/>
         <boot order='1'/>
         <readonly/>
         <address type='drive' controller='0' bus='1' unit='0'/>
       </disk>
    -------------------------------------------------------

I installed latest qemu-kvm to /usr/local/qemu, and replaced
/usr/libexec/qemu-kvm to /user/local/qemu/bin/qemu-system-x86_64,
but it was booting up from virtio HDD.

On RHEL6.0 host, I tested <boot dev> setting version, 
VM didn't boot up from virtio HDD.
it cannot boot up from CD-ROM. (expected behaviour)

I checked qemu-kvm option by ps command,
but I think correct -boot or bootindex of -device option is specified.

[<boot dev> setting version(RHEL6.1 host)]
qemu      9871     1 74 16:22 ?        00:00:25 /usr/libexec/qemu-kvm -S -M rhel6.1.0 -enable-kvm -m 2048
 -smp 8,sockets=8,cores=1,threads=1 -name RHEL6.1-GA -uuid 49703959-f0b0-9864-5788-4bc07028bac4 -nodefconfig
 -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/RHEL6.1-GA.monitor,server,nowait 
 -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -boot order=d,menu=off
 -drive file=/work/libvirt/images/RHEL6.1GA_x86_64_full_file.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,aio=threads
 -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
 -drive file=/tmp/not_bootable_image.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0
 -device virtio-net-pci,vlan=0,id=net0,mac=52:54:00:0b:68:82,bus=pci.0,addr=0x3 -net tap,fd=21,vlan=0,name=hostnet0
 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0
 -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4
 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6

  -> "-boot order=d" is specified


[<boot order> setting version(RHEL6.1 host)]
qemu      4073     1 23 10:08 ?        00:00:37 /usr/libexec/qemu-kvm -S -M rhel6.1.0 -enable-kvm -m 2048
 -smp 8,sockets=8,cores=1,threads=1 -name RHEL6.1-GA -uuid 49703959-f0b0-9864-5788-4bc07028bac4 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/RHEL6.1-GA.monitor,server,nowait
 -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -drive file=/work/libvirt/images/RHEL6.1GA_x86_64_full_file.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,aio=threads
 -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw
 -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1
 -device virtio-net-pci,vlan=0,id=net0,mac=52:54:00:0b:68:82,bus=pci.0,addr=0x3 -net tap,fd=21,vlan=0,name=hostnet0
 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0
 -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4
 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6

  -> "bootindex=1" is specified for IDE CD-ROM device only.


Next, I suspected seabios, but it was same version of RHEL6.0 and RHEL6.1.

  # rpm -q seabios
  seabios-0.6.1.2-3.el6.x86_64

What causes this problem?
-- 
Minoru Usui <usui@mxm.nes.nec.co.jp>

Re: [Qemu-devel] Boot order problem

[Gleb Natapov]

On Fri, Jul 22, 2011 at 09:51:16AM +0900, Minoru Usui wrote:
> Hi, everyone
> 
> I'm in trouble about boot order of VM.
> If anyone know cause of this problem, please let me know.
> 
The cause of the "problem" is the design. booindex and -boot only
modifies the order in which bios will search for bootable device.
It does not exclude devices from a boot device list.
 
> On following environment, I tried to boot from IDE CD-ROM device 
> without inserting any bootable media, which is expected to fail,
> but VM was booting up from virtio HDD which was not specified as bootable device.
> 
>   * host : RHEL6.1(x86_64)
>     guest: RHEL6.1(x86_64)
>   * VM has IDE CD-ROM and virtio HDD.
>   * There is no bootable media in IDE CD-ROM.
>   * RHEL6.1 is installed in virtio HDD
>   * Only IDE CD-ROM was spcified as bootable device.
>   * XML configuration of libvirt is below.
>     I tested <boot dev> and <boot order> setting,
>     but both are booting up from virtio HDD.
>     -------------------------------------------------------
>     [<boot dev> setting version]
>       <os>
>         <type arch='x86_64' machine='rhel6.1.0'>hvm</type>
>         <boot dev='cdrom'/>
>         <bootmenu enable='no'/>
>       </os>
> 
>     [<boot order> setting version]
>        <disk type='file' device='cdrom'>
>          <driver name='qemu' type='raw'/>
>          <target dev='hdc' bus='ide'/>
>          <boot order='1'/>
>          <readonly/>
>          <address type='drive' controller='0' bus='1' unit='0'/>
>        </disk>
>     -------------------------------------------------------
> 
> I installed latest qemu-kvm to /usr/local/qemu, and replaced
> /usr/libexec/qemu-kvm to /user/local/qemu/bin/qemu-system-x86_64,
> but it was booting up from virtio HDD.
> 
> On RHEL6.0 host, I tested <boot dev> setting version, 
> VM didn't boot up from virtio HDD.
> it cannot boot up from CD-ROM. (expected behaviour)
> 
This is not expected behaviour. Expected behaviour is VM boots from HDD.
The only way I can explain behaviour you describe above is that the bios
you are using for RHEL6.0 rpm does not support booting from virtio HDD.
You can test this but making HDD to be ide and retry your test.

> I checked qemu-kvm option by ps command,
> but I think correct -boot or bootindex of -device option is specified.
> 
> [<boot dev> setting version(RHEL6.1 host)]
> qemu      9871     1 74 16:22 ?        00:00:25 /usr/libexec/qemu-kvm -S -M rhel6.1.0 -enable-kvm -m 2048
>  -smp 8,sockets=8,cores=1,threads=1 -name RHEL6.1-GA -uuid 49703959-f0b0-9864-5788-4bc07028bac4 -nodefconfig
>  -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/RHEL6.1-GA.monitor,server,nowait 
>  -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -boot order=d,menu=off
>  -drive file=/work/libvirt/images/RHEL6.1GA_x86_64_full_file.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,aio=threads
>  -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
>  -drive file=/tmp/not_bootable_image.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0
>  -device virtio-net-pci,vlan=0,id=net0,mac=52:54:00:0b:68:82,bus=pci.0,addr=0x3 -net tap,fd=21,vlan=0,name=hostnet0
>  -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0
>  -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4
>  -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
> 
>   -> "-boot order=d" is specified
> 
> 
> [<boot order> setting version(RHEL6.1 host)]
> qemu      4073     1 23 10:08 ?        00:00:37 /usr/libexec/qemu-kvm -S -M rhel6.1.0 -enable-kvm -m 2048
>  -smp 8,sockets=8,cores=1,threads=1 -name RHEL6.1-GA -uuid 49703959-f0b0-9864-5788-4bc07028bac4 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/RHEL6.1-GA.monitor,server,nowait
>  -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -drive file=/work/libvirt/images/RHEL6.1GA_x86_64_full_file.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,aio=threads
>  -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
>  -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw
>  -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1
>  -device virtio-net-pci,vlan=0,id=net0,mac=52:54:00:0b:68:82,bus=pci.0,addr=0x3 -net tap,fd=21,vlan=0,name=hostnet0
>  -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0
>  -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4
>  -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
> 
>   -> "bootindex=1" is specified for IDE CD-ROM device only.
> 
> 
> Next, I suspected seabios, but it was same version of RHEL6.0 and RHEL6.1.
> 
>   # rpm -q seabios
>   seabios-0.6.1.2-3.el6.x86_64
> 
This bios supports booting from virtio and when I do the same test as
you, qemu boots from virtio HDD. Try to reinstall seabios rpm.

> What causes this problem?
> -- 
> Minoru Usui <usui@mxm.nes.nec.co.jp>

--
			Gleb.

Re: [Qemu-devel] Boot order problem

[Minoru Usui]

Hi, Gleb

Thank you for your reply.

On Sun, 24 Jul 2011 09:30:49 +0300
Gleb Natapov <gleb@redhat.com> wrote:

> On Fri, Jul 22, 2011 at 09:51:16AM +0900, Minoru Usui wrote:
> > Hi, everyone
> > 
> > I'm in trouble about boot order of VM.
> > If anyone know cause of this problem, please let me know.
> > 
> The cause of the "problem" is the design. booindex and -boot only
> modifies the order in which bios will search for bootable device.
> It does not exclude devices from a boot device list.

Hmm, this design is little bit strange to me.
We can specify BIOS boot order on baremetal BIOS.
In this case, we won't expected boots up from not specified boot device, generally.

Isn't it intuitive that -boot and bootindex option can specify 
not only boot order, but also boot device list?

> > On following environment, I tried to boot from IDE CD-ROM device 
> > without inserting any bootable media, which is expected to fail,
> > but VM was booting up from virtio HDD which was not specified as bootable device.
> > 
> >   * host : RHEL6.1(x86_64)
> >     guest: RHEL6.1(x86_64)
> >   * VM has IDE CD-ROM and virtio HDD.
> >   * There is no bootable media in IDE CD-ROM.
> >   * RHEL6.1 is installed in virtio HDD
> >   * Only IDE CD-ROM was spcified as bootable device.
> >   * XML configuration of libvirt is below.
> >     I tested <boot dev> and <boot order> setting,
> >     but both are booting up from virtio HDD.
> >     -------------------------------------------------------
> >     [<boot dev> setting version]
> >       <os>
> >         <type arch='x86_64' machine='rhel6.1.0'>hvm</type>
> >         <boot dev='cdrom'/>
> >         <bootmenu enable='no'/>
> >       </os>
> > 
> >     [<boot order> setting version]
> >        <disk type='file' device='cdrom'>
> >          <driver name='qemu' type='raw'/>
> >          <target dev='hdc' bus='ide'/>
> >          <boot order='1'/>
> >          <readonly/>
> >          <address type='drive' controller='0' bus='1' unit='0'/>
> >        </disk>
> >     -------------------------------------------------------
> > 
> > I installed latest qemu-kvm to /usr/local/qemu, and replaced
> > /usr/libexec/qemu-kvm to /user/local/qemu/bin/qemu-system-x86_64,
> > but it was booting up from virtio HDD.
> > 
> > On RHEL6.0 host, I tested <boot dev> setting version, 
> > VM didn't boot up from virtio HDD.
> > it cannot boot up from CD-ROM. (expected behaviour)
> > 
> This is not expected behaviour. Expected behaviour is VM boots from HDD.
> The only way I can explain behaviour you describe above is that the bios
> you are using for RHEL6.0 rpm does not support booting from virtio HDD.
> You can test this but making HDD to be ide and retry your test.

I changed virtio HDD to IDE HDD, and retry my test on RHEL6.0(seabios-0.5.1-3.el6.x86_64),
but VM didn't boot up from IDE HDD.(same result)
Of cource, if I specified IDE HDD first (not IDE CD-ROM), it can boot up.
Am I somthing wrong?

[libvirt setting]
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source dev='/dev/sdb5'/>
      <target dev='hda' bus='ide'/>
      <address type='drive' controller='0' bus='1' unit='1'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <target dev='hdc' bus='ide'/>
      <readonly/>
      <alias name='ide0-1-0'/>
      <address type='drive' controller='0' bus='1' unit='0'/>
    </disk>
     <controller type='ide' index='0'>
      <alias name='ide0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>

[qemu-kvm option by ps command]
 qemu      6685     1 21 09:42 ?        00:00:03 /usr/libexec/qemu-kvm -S -M rhel6.0.0 -enable-kvm -m 2048
 -smp 2,sockets=2,cores=1,threads=1 -name RHEL5.5-x86_64-disk1 -uuid b05c3fa6-a52d-1f15-8412-00a876a0c672
 -nodefconfig -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/RHEL5.5-x86_64-disk1.monitor,server,nowait
 -mon chardev=monitor,mode=control -rtc base=utc -boot d -drive file=/dev/sdb5,if=none,id=drive-ide0-1-1,format=raw,cache=none
 -device ide-drive,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1
 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw
 -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=24,id=hostnet0
 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:47:2a:f9,bus=pci.0,addr=0x3 -chardev pty,id=serial0
 -device isa-serial,chardev=serial0 -usb -vnc 127.0.0.1:1 -vga cirrus -device AC97,id=sound0,bus=pci.0,addr=0x4
 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6


> > I checked qemu-kvm option by ps command,
> > but I think correct -boot or bootindex of -device option is specified.
> > 
> > [<boot dev> setting version(RHEL6.1 host)]
> > qemu      9871     1 74 16:22 ?        00:00:25 /usr/libexec/qemu-kvm -S -M rhel6.1.0 -enable-kvm -m 2048
> >  -smp 8,sockets=8,cores=1,threads=1 -name RHEL6.1-GA -uuid 49703959-f0b0-9864-5788-4bc07028bac4 -nodefconfig
> >  -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/RHEL6.1-GA.monitor,server,nowait 
> >  -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -boot order=d,menu=off
> >  -drive file=/work/libvirt/images/RHEL6.1GA_x86_64_full_file.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,aio=threads
> >  -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
> >  -drive file=/tmp/not_bootable_image.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0
> >  -device virtio-net-pci,vlan=0,id=net0,mac=52:54:00:0b:68:82,bus=pci.0,addr=0x3 -net tap,fd=21,vlan=0,name=hostnet0
> >  -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0
> >  -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4
> >  -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
> > 
> >   -> "-boot order=d" is specified
> > 
> > 
> > [<boot order> setting version(RHEL6.1 host)]
> > qemu      4073     1 23 10:08 ?        00:00:37 /usr/libexec/qemu-kvm -S -M rhel6.1.0 -enable-kvm -m 2048
> >  -smp 8,sockets=8,cores=1,threads=1 -name RHEL6.1-GA -uuid 49703959-f0b0-9864-5788-4bc07028bac4 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/RHEL6.1-GA.monitor,server,nowait
> >  -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -drive file=/work/libvirt/images/RHEL6.1GA_x86_64_full_file.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,aio=threads
> >  -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
> >  -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw
> >  -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1
> >  -device virtio-net-pci,vlan=0,id=net0,mac=52:54:00:0b:68:82,bus=pci.0,addr=0x3 -net tap,fd=21,vlan=0,name=hostnet0
> >  -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0
> >  -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4
> >  -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
> > 
> >   -> "bootindex=1" is specified for IDE CD-ROM device only.
> > 
> > 
> > Next, I suspected seabios, but it was same version of RHEL6.0 and RHEL6.1.
> > 
> >   # rpm -q seabios
> >   seabios-0.6.1.2-3.el6.x86_64
> > 
> This bios supports booting from virtio and when I do the same test as
> you, qemu boots from virtio HDD. Try to reinstall seabios rpm.

I recheck the version of seabios, it was different version of RHEL6.0 and RHEL6.1.
I'm sorry for misleading.

[RHEL6.0]
  seabios-0.5.1-3.el6.x86_64

[RHEL6.1]
  seabios-0.6.1.2-3.el6.x86_64

> > What causes this problem?
> > -- 
> > Minoru Usui <usui@mxm.nes.nec.co.jp>
> 
> --
> 			Gleb.


-- 
Minoru Usui <usui@mxm.nes.nec.co.jp>

Re: [Qemu-devel] Boot order problem

[Gleb Natapov]

On Mon, Jul 25, 2011 at 04:07:12PM +0900, Minoru Usui wrote:
> Hi, Gleb
> 
> Thank you for your reply.
> 
> On Sun, 24 Jul 2011 09:30:49 +0300
> Gleb Natapov <gleb@redhat.com> wrote:
> 
> > On Fri, Jul 22, 2011 at 09:51:16AM +0900, Minoru Usui wrote:
> > > Hi, everyone
> > > 
> > > I'm in trouble about boot order of VM.
> > > If anyone know cause of this problem, please let me know.
> > > 
> > The cause of the "problem" is the design. booindex and -boot only
> > modifies the order in which bios will search for bootable device.
> > It does not exclude devices from a boot device list.
> 
> Hmm, this design is little bit strange to me.
> We can specify BIOS boot order on baremetal BIOS.
> In this case, we won't expected boots up from not specified boot device, generally.
>
My bios has 4 boot device slots filled by factory defaults. If I modify
the first one (like -boot c does) it leaves other slots with their
previous values. If first slot is not bootable it will fall back to
next one.

> Isn't it intuitive that -boot and bootindex option can specify 
> not only boot order, but also boot device list?
What is the use case? If this is what it will do you will be required to
specify full boot order list each time you start a VM. Much more common
scenario is to modify boot order priority by moving some devices at the
top. It is possible to add an option that will exclude device from a
boot list. Need some work both in qemu and seabios.

> 
> > > On following environment, I tried to boot from IDE CD-ROM device 
> > > without inserting any bootable media, which is expected to fail,
> > > but VM was booting up from virtio HDD which was not specified as bootable device.
> > > 
> > >   * host : RHEL6.1(x86_64)
> > >     guest: RHEL6.1(x86_64)
> > >   * VM has IDE CD-ROM and virtio HDD.
> > >   * There is no bootable media in IDE CD-ROM.
> > >   * RHEL6.1 is installed in virtio HDD
> > >   * Only IDE CD-ROM was spcified as bootable device.
> > >   * XML configuration of libvirt is below.
> > >     I tested <boot dev> and <boot order> setting,
> > >     but both are booting up from virtio HDD.
> > >     -------------------------------------------------------
> > >     [<boot dev> setting version]
> > >       <os>
> > >         <type arch='x86_64' machine='rhel6.1.0'>hvm</type>
> > >         <boot dev='cdrom'/>
> > >         <bootmenu enable='no'/>
> > >       </os>
> > > 
> > >     [<boot order> setting version]
> > >        <disk type='file' device='cdrom'>
> > >          <driver name='qemu' type='raw'/>
> > >          <target dev='hdc' bus='ide'/>
> > >          <boot order='1'/>
> > >          <readonly/>
> > >          <address type='drive' controller='0' bus='1' unit='0'/>
> > >        </disk>
> > >     -------------------------------------------------------
> > > 
> > > I installed latest qemu-kvm to /usr/local/qemu, and replaced
> > > /usr/libexec/qemu-kvm to /user/local/qemu/bin/qemu-system-x86_64,
> > > but it was booting up from virtio HDD.
> > > 
> > > On RHEL6.0 host, I tested <boot dev> setting version, 
> > > VM didn't boot up from virtio HDD.
> > > it cannot boot up from CD-ROM. (expected behaviour)
> > > 
> > This is not expected behaviour. Expected behaviour is VM boots from HDD.
> > The only way I can explain behaviour you describe above is that the bios
> > you are using for RHEL6.0 rpm does not support booting from virtio HDD.
> > You can test this but making HDD to be ide and retry your test.
> 
> I changed virtio HDD to IDE HDD, and retry my test on RHEL6.0(seabios-0.5.1-3.el6.x86_64),
> but VM didn't boot up from IDE HDD.(same result)
Hmm. Yes, I can reproduce this with rhel60 bios. rhel6.1 works as
expected though. Need to check why rhel60 bios works like this.

> Of cource, if I specified IDE HDD first (not IDE CD-ROM), it can boot up.
> Am I somthing wrong?
> 
> [libvirt setting]
>     <disk type='block' device='disk'>
>       <driver name='qemu' type='raw' cache='none'/>
>       <source dev='/dev/sdb5'/>
>       <target dev='hda' bus='ide'/>
>       <address type='drive' controller='0' bus='1' unit='1'/>
>     </disk>
>     <disk type='file' device='cdrom'>
>       <driver name='qemu' type='raw'/>
>       <target dev='hdc' bus='ide'/>
>       <readonly/>
>       <alias name='ide0-1-0'/>
>       <address type='drive' controller='0' bus='1' unit='0'/>
>     </disk>
>      <controller type='ide' index='0'>
>       <alias name='ide0'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
>     </controller>
> 
> [qemu-kvm option by ps command]
>  qemu      6685     1 21 09:42 ?        00:00:03 /usr/libexec/qemu-kvm -S -M rhel6.0.0 -enable-kvm -m 2048
>  -smp 2,sockets=2,cores=1,threads=1 -name RHEL5.5-x86_64-disk1 -uuid b05c3fa6-a52d-1f15-8412-00a876a0c672
>  -nodefconfig -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/RHEL5.5-x86_64-disk1.monitor,server,nowait
>  -mon chardev=monitor,mode=control -rtc base=utc -boot d -drive file=/dev/sdb5,if=none,id=drive-ide0-1-1,format=raw,cache=none
>  -device ide-drive,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1
>  -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw
>  -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=24,id=hostnet0
>  -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:47:2a:f9,bus=pci.0,addr=0x3 -chardev pty,id=serial0
>  -device isa-serial,chardev=serial0 -usb -vnc 127.0.0.1:1 -vga cirrus -device AC97,id=sound0,bus=pci.0,addr=0x4
>  -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
> 
> 
> > > I checked qemu-kvm option by ps command,
> > > but I think correct -boot or bootindex of -device option is specified.
> > > 
> > > [<boot dev> setting version(RHEL6.1 host)]
> > > qemu      9871     1 74 16:22 ?        00:00:25 /usr/libexec/qemu-kvm -S -M rhel6.1.0 -enable-kvm -m 2048
> > >  -smp 8,sockets=8,cores=1,threads=1 -name RHEL6.1-GA -uuid 49703959-f0b0-9864-5788-4bc07028bac4 -nodefconfig
> > >  -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/RHEL6.1-GA.monitor,server,nowait 
> > >  -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -boot order=d,menu=off
> > >  -drive file=/work/libvirt/images/RHEL6.1GA_x86_64_full_file.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,aio=threads
> > >  -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
> > >  -drive file=/tmp/not_bootable_image.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0
> > >  -device virtio-net-pci,vlan=0,id=net0,mac=52:54:00:0b:68:82,bus=pci.0,addr=0x3 -net tap,fd=21,vlan=0,name=hostnet0
> > >  -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0
> > >  -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4
> > >  -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
> > > 
> > >   -> "-boot order=d" is specified
> > > 
> > > 
> > > [<boot order> setting version(RHEL6.1 host)]
> > > qemu      4073     1 23 10:08 ?        00:00:37 /usr/libexec/qemu-kvm -S -M rhel6.1.0 -enable-kvm -m 2048
> > >  -smp 8,sockets=8,cores=1,threads=1 -name RHEL6.1-GA -uuid 49703959-f0b0-9864-5788-4bc07028bac4 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/RHEL6.1-GA.monitor,server,nowait
> > >  -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -drive file=/work/libvirt/images/RHEL6.1GA_x86_64_full_file.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,aio=threads
> > >  -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
> > >  -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw
> > >  -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1
> > >  -device virtio-net-pci,vlan=0,id=net0,mac=52:54:00:0b:68:82,bus=pci.0,addr=0x3 -net tap,fd=21,vlan=0,name=hostnet0
> > >  -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0
> > >  -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4
> > >  -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
> > > 
> > >   -> "bootindex=1" is specified for IDE CD-ROM device only.
> > > 
> > > 
> > > Next, I suspected seabios, but it was same version of RHEL6.0 and RHEL6.1.
> > > 
> > >   # rpm -q seabios
> > >   seabios-0.6.1.2-3.el6.x86_64
> > > 
> > This bios supports booting from virtio and when I do the same test as
> > you, qemu boots from virtio HDD. Try to reinstall seabios rpm.
> 
> I recheck the version of seabios, it was different version of RHEL6.0 and RHEL6.1.
> I'm sorry for misleading.
> 
> [RHEL6.0]
>   seabios-0.5.1-3.el6.x86_64
> 
> [RHEL6.1]
>   seabios-0.6.1.2-3.el6.x86_64
> 
Both support virtio. The difference somewhere else.

--
			Gleb.

Re: [Qemu-devel] Boot order problem

[Minoru Usui]

On Mon, 25 Jul 2011 12:06:19 +0300
Gleb Natapov <gleb@redhat.com> wrote:

> On Mon, Jul 25, 2011 at 04:07:12PM +0900, Minoru Usui wrote:
> > Hi, Gleb
> > 
> > Thank you for your reply.
> > 
> > On Sun, 24 Jul 2011 09:30:49 +0300
> > Gleb Natapov <gleb@redhat.com> wrote:
> > 
> > > On Fri, Jul 22, 2011 at 09:51:16AM +0900, Minoru Usui wrote:
> > > > Hi, everyone
> > > > 
> > > > I'm in trouble about boot order of VM.
> > > > If anyone know cause of this problem, please let me know.
> > > > 
> > > The cause of the "problem" is the design. booindex and -boot only
> > > modifies the order in which bios will search for bootable device.
> > > It does not exclude devices from a boot device list.
> > 
> > Hmm, this design is little bit strange to me.
> > We can specify BIOS boot order on baremetal BIOS.
> > In this case, we won't expected boots up from not specified boot device, generally.
> >
> My bios has 4 boot device slots filled by factory defaults. If I modify
> the first one (like -boot c does) it leaves other slots with their
> previous values. If first slot is not bootable it will fall back to
> next one.

OK.
I understand -boot and bootindex design.
 
> > Isn't it intuitive that -boot and bootindex option can specify 
> > not only boot order, but also boot device list?
> What is the use case? If this is what it will do you will be required to
> specify full boot order list each time you start a VM. Much more common
> scenario is to modify boot order priority by moving some devices at the
> top. It is possible to add an option that will exclude device from a
> boot list. Need some work both in qemu and seabios.

Thank you for suggestion.
If I want to controll boot device list, I'll try to specify full boot order list.

> > > > On following environment, I tried to boot from IDE CD-ROM device 
> > > > without inserting any bootable media, which is expected to fail,
> > > > but VM was booting up from virtio HDD which was not specified as bootable device.
> > > > 
> > > >   * host : RHEL6.1(x86_64)
> > > >     guest: RHEL6.1(x86_64)
> > > >   * VM has IDE CD-ROM and virtio HDD.
> > > >   * There is no bootable media in IDE CD-ROM.
> > > >   * RHEL6.1 is installed in virtio HDD
> > > >   * Only IDE CD-ROM was spcified as bootable device.
> > > >   * XML configuration of libvirt is below.
> > > >     I tested <boot dev> and <boot order> setting,
> > > >     but both are booting up from virtio HDD.
> > > >     -------------------------------------------------------
> > > >     [<boot dev> setting version]
> > > >       <os>
> > > >         <type arch='x86_64' machine='rhel6.1.0'>hvm</type>
> > > >         <boot dev='cdrom'/>
> > > >         <bootmenu enable='no'/>
> > > >       </os>
> > > > 
> > > >     [<boot order> setting version]
> > > >        <disk type='file' device='cdrom'>
> > > >          <driver name='qemu' type='raw'/>
> > > >          <target dev='hdc' bus='ide'/>
> > > >          <boot order='1'/>
> > > >          <readonly/>
> > > >          <address type='drive' controller='0' bus='1' unit='0'/>
> > > >        </disk>
> > > >     -------------------------------------------------------
> > > > 
> > > > I installed latest qemu-kvm to /usr/local/qemu, and replaced
> > > > /usr/libexec/qemu-kvm to /user/local/qemu/bin/qemu-system-x86_64,
> > > > but it was booting up from virtio HDD.
> > > > 
> > > > On RHEL6.0 host, I tested <boot dev> setting version, 
> > > > VM didn't boot up from virtio HDD.
> > > > it cannot boot up from CD-ROM. (expected behaviour)
> > > > 
> > > This is not expected behaviour. Expected behaviour is VM boots from HDD.
> > > The only way I can explain behaviour you describe above is that the bios
> > > you are using for RHEL6.0 rpm does not support booting from virtio HDD.
> > > You can test this but making HDD to be ide and retry your test.
> > 
> > I changed virtio HDD to IDE HDD, and retry my test on RHEL6.0(seabios-0.5.1-3.el6.x86_64),
> > but VM didn't boot up from IDE HDD.(same result)
> Hmm. Yes, I can reproduce this with rhel60 bios. rhel6.1 works as
> expected though. Need to check why rhel60 bios works like this.

Thank you so much.
I'll wait your investigation.

> > Of cource, if I specified IDE HDD first (not IDE CD-ROM), it can boot up.
> > Am I somthing wrong?
> > 
> > [libvirt setting]
> >     <disk type='block' device='disk'>
> >       <driver name='qemu' type='raw' cache='none'/>
> >       <source dev='/dev/sdb5'/>
> >       <target dev='hda' bus='ide'/>
> >       <address type='drive' controller='0' bus='1' unit='1'/>
> >     </disk>
> >     <disk type='file' device='cdrom'>
> >       <driver name='qemu' type='raw'/>
> >       <target dev='hdc' bus='ide'/>
> >       <readonly/>
> >       <alias name='ide0-1-0'/>
> >       <address type='drive' controller='0' bus='1' unit='0'/>
> >     </disk>
> >      <controller type='ide' index='0'>
> >       <alias name='ide0'/>
> >       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
> >     </controller>
> > 
> > [qemu-kvm option by ps command]
> >  qemu      6685     1 21 09:42 ?        00:00:03 /usr/libexec/qemu-kvm -S -M rhel6.0.0 -enable-kvm -m 2048
> >  -smp 2,sockets=2,cores=1,threads=1 -name RHEL5.5-x86_64-disk1 -uuid b05c3fa6-a52d-1f15-8412-00a876a0c672
> >  -nodefconfig -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/RHEL5.5-x86_64-disk1.monitor,server,nowait
> >  -mon chardev=monitor,mode=control -rtc base=utc -boot d -drive file=/dev/sdb5,if=none,id=drive-ide0-1-1,format=raw,cache=none
> >  -device ide-drive,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1
> >  -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw
> >  -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=24,id=hostnet0
> >  -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:47:2a:f9,bus=pci.0,addr=0x3 -chardev pty,id=serial0
> >  -device isa-serial,chardev=serial0 -usb -vnc 127.0.0.1:1 -vga cirrus -device AC97,id=sound0,bus=pci.0,addr=0x4
> >  -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
> > 
> > 
> > > > I checked qemu-kvm option by ps command,
> > > > but I think correct -boot or bootindex of -device option is specified.
> > > > 
> > > > [<boot dev> setting version(RHEL6.1 host)]
> > > > qemu      9871     1 74 16:22 ?        00:00:25 /usr/libexec/qemu-kvm -S -M rhel6.1.0 -enable-kvm -m 2048
> > > >  -smp 8,sockets=8,cores=1,threads=1 -name RHEL6.1-GA -uuid 49703959-f0b0-9864-5788-4bc07028bac4 -nodefconfig
> > > >  -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/RHEL6.1-GA.monitor,server,nowait 
> > > >  -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -boot order=d,menu=off
> > > >  -drive file=/work/libvirt/images/RHEL6.1GA_x86_64_full_file.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,aio=threads
> > > >  -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
> > > >  -drive file=/tmp/not_bootable_image.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0
> > > >  -device virtio-net-pci,vlan=0,id=net0,mac=52:54:00:0b:68:82,bus=pci.0,addr=0x3 -net tap,fd=21,vlan=0,name=hostnet0
> > > >  -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0
> > > >  -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4
> > > >  -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
> > > > 
> > > >   -> "-boot order=d" is specified
> > > > 
> > > > 
> > > > [<boot order> setting version(RHEL6.1 host)]
> > > > qemu      4073     1 23 10:08 ?        00:00:37 /usr/libexec/qemu-kvm -S -M rhel6.1.0 -enable-kvm -m 2048
> > > >  -smp 8,sockets=8,cores=1,threads=1 -name RHEL6.1-GA -uuid 49703959-f0b0-9864-5788-4bc07028bac4 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/RHEL6.1-GA.monitor,server,nowait
> > > >  -mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc -drive file=/work/libvirt/images/RHEL6.1GA_x86_64_full_file.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,aio=threads
> > > >  -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
> > > >  -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw
> > > >  -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1
> > > >  -device virtio-net-pci,vlan=0,id=net0,mac=52:54:00:0b:68:82,bus=pci.0,addr=0x3 -net tap,fd=21,vlan=0,name=hostnet0
> > > >  -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0
> > > >  -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4
> > > >  -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
> > > > 
> > > >   -> "bootindex=1" is specified for IDE CD-ROM device only.
> > > > 
> > > > 
> > > > Next, I suspected seabios, but it was same version of RHEL6.0 and RHEL6.1.
> > > > 
> > > >   # rpm -q seabios
> > > >   seabios-0.6.1.2-3.el6.x86_64
> > > > 
> > > This bios supports booting from virtio and when I do the same test as
> > > you, qemu boots from virtio HDD. Try to reinstall seabios rpm.
> > 
> > I recheck the version of seabios, it was different version of RHEL6.0 and RHEL6.1.
> > I'm sorry for misleading.
> > 
> > [RHEL6.0]
> >   seabios-0.5.1-3.el6.x86_64
> > 
> > [RHEL6.1]
> >   seabios-0.6.1.2-3.el6.x86_64
> > 
> Both support virtio. The difference somewhere else.
> 
> --
> 			Gleb.


-- 
Minoru Usui <usui@mxm.nes.nec.co.jp>

Re: [Qemu-devel] Boot order problem

[Minoru Usui]

On Sun, 24 Jul 2011 09:30:49 +0300
Gleb Natapov <gleb@redhat.com> wrote:

> On Fri, Jul 22, 2011 at 09:51:16AM +0900, Minoru Usui wrote:
> > Hi, everyone
> > 
> > I'm in trouble about boot order of VM.
> > If anyone know cause of this problem, please let me know.
> > 
> The cause of the "problem" is the design. booindex and -boot only
> modifies the order in which bios will search for bootable device.
> It does not exclude devices from a boot device list.
>  
> > On following environment, I tried to boot from IDE CD-ROM device 
> > without inserting any bootable media, which is expected to fail,
> > but VM was booting up from virtio HDD which was not specified as bootable device.
> > 
> >   * host : RHEL6.1(x86_64)
> >     guest: RHEL6.1(x86_64)
> >   * VM has IDE CD-ROM and virtio HDD.
> >   * There is no bootable media in IDE CD-ROM.
> >   * RHEL6.1 is installed in virtio HDD
> >   * Only IDE CD-ROM was spcified as bootable device.
> >   * XML configuration of libvirt is below.
> >     I tested <boot dev> and <boot order> setting,
> >     but both are booting up from virtio HDD.
> >     -------------------------------------------------------
> >     [<boot dev> setting version]
> >       <os>
> >         <type arch='x86_64' machine='rhel6.1.0'>hvm</type>
> >         <boot dev='cdrom'/>
> >         <bootmenu enable='no'/>
> >       </os>
> > 
> >     [<boot order> setting version]
> >        <disk type='file' device='cdrom'>
> >          <driver name='qemu' type='raw'/>
> >          <target dev='hdc' bus='ide'/>
> >          <boot order='1'/>
> >          <readonly/>
> >          <address type='drive' controller='0' bus='1' unit='0'/>
> >        </disk>
> >     -------------------------------------------------------

I tested another one about <boot order> case on RHEL6.1, and I also faced another problem.

VM has two virtio HDD. HDD1 is installed RHEL6.1, HDD2 is empty.
I specified boot order to HDD1:1, HDD2:2, VM booted up from HDD1,
but boot order HDD1:2, HDD2:1 case, VM couldn't boot up from HDD2.
(It searched CD-ROM, NIC(gPXE), and finally stopped booting.)

It seems seabios searches only 1 device per device list(HDD, CD-ROM, NET, FLOPPY).
Is it true?

<boot order> can specify per device, so shouldn't seabios search all device,
even if it specifies multiple device per device list?

-- 
Minoru Usui <usui@mxm.nes.nec.co.jp>

Re: [Qemu-devel] Boot order problem

[Gleb Natapov]

On Mon, Jul 25, 2011 at 04:44:58PM +0900, Minoru Usui wrote:
> On Sun, 24 Jul 2011 09:30:49 +0300
> Gleb Natapov <gleb@redhat.com> wrote:
> 
[skip]

> I tested another one about <boot order> case on RHEL6.1, and I also faced another problem.
> 
> VM has two virtio HDD. HDD1 is installed RHEL6.1, HDD2 is empty.
> I specified boot order to HDD1:1, HDD2:2, VM booted up from HDD1,
> but boot order HDD1:2, HDD2:1 case, VM couldn't boot up from HDD2.
> (It searched CD-ROM, NIC(gPXE), and finally stopped booting.)
> 
That's BIOS specification limitation. BIOS can't fall back from one HDD
to another, so only HDD with lowest priority among all HDDs will be tried.
 
> It seems seabios searches only 1 device per device list(HDD, CD-ROM, NET, FLOPPY).
> Is it true?
No, it searches only one HDD. Other devices do not have this limitation
IIRC.

> 
> <boot order> can specify per device, so shouldn't seabios search all device,
> even if it specifies multiple device per device list?
> 
> -- 
> Minoru Usui <usui@mxm.nes.nec.co.jp>

--
			Gleb.

Re: [Qemu-devel] Boot order problem

[Minoru Usui]

On Mon, 25 Jul 2011 12:09:31 +0300
Gleb Natapov <gleb@redhat.com> wrote:

> On Mon, Jul 25, 2011 at 04:44:58PM +0900, Minoru Usui wrote:
> > On Sun, 24 Jul 2011 09:30:49 +0300
> > Gleb Natapov <gleb@redhat.com> wrote:
> > 
> [skip]
> 
> > I tested another one about <boot order> case on RHEL6.1, and I also faced another problem.
> > 
> > VM has two virtio HDD. HDD1 is installed RHEL6.1, HDD2 is empty.
> > I specified boot order to HDD1:1, HDD2:2, VM booted up from HDD1,
> > but boot order HDD1:2, HDD2:1 case, VM couldn't boot up from HDD2.
> > (It searched CD-ROM, NIC(gPXE), and finally stopped booting.)
> > 
> That's BIOS specification limitation. BIOS can't fall back from one HDD
> to another, so only HDD with lowest priority among all HDDs will be tried.

Thank you for explanation.
I understand.

> > It seems seabios searches only 1 device per device list(HDD, CD-ROM, NET, FLOPPY).
> > Is it true?
> No, it searches only one HDD. Other devices do not have this limitation
> IIRC.
> 
> > 
> > <boot order> can specify per device, so shouldn't seabios search all device,
> > even if it specifies multiple device per device list?
> > 
> > -- 
> > Minoru Usui <usui@mxm.nes.nec.co.jp>
> 
> --
> 			Gleb.


-- 
Minoru Usui <usui@mxm.nes.nec.co.jp>