incorrect disk numbering with qemu

incorrect disk numbering with qemu

[Olaf Hering]

It seems qemu does not enumerate the configured disks correctly with a
config like this:

disk=[  
        'raw:/some.iso,hda:cdrom,r',
        'raw:/some.raw,xvda,w',
]

With a PV guest it works fine, the guest has a hda and xvda device. 
But a HVM guest fails to start:
qemu-system-i386: -drive file=/some.raw,if=ide,index=0,media=disk,format=raw,cache=writeback: drive with bus=0, unit=0 (index=0) exists

I think that kind of config used to work with xend. Should the code which
enumerates the disks be smarter?

Olaf

-- 
name="enum"
memory=1024
disk=[  
        'file:/some.iso,hda:cdrom,r',
        'raw:/some.raw,xvda,r',
]
vif=[ 'mac=00:16:3e:14:85:06,bridge=br0', ]
vfb=['type=vnc,vncunused=1,keymap=de']
builder="linux"
extra="ignore_loglevel xencons=hvc0 console=hvc0"
kernel="/vmlinuz"
ramdisk="/initrd"

builder="hvm"

Re: incorrect disk numbering with qemu

[Ian Campbell]

On Wed, 2014-01-15 at 18:12 +0100, Olaf Hering wrote:
> It seems qemu does not enumerate the configured disks correctly with a
> config like this:
> 
> disk=[  
>         'raw:/some.iso,hda:cdrom,r',
>         'raw:/some.raw,xvda,w',
> ]
> 
> With a PV guest it works fine, the guest has a hda and xvda device. 
> But a HVM guest fails to start:
> qemu-system-i386: -drive file=/some.raw,if=ide,index=0,media=disk,format=raw,cache=writeback: drive with bus=0, unit=0 (index=0) exists
> 
> I think that kind of config used to work with xend.

Did it? I thought xvda and hda were effectively considered two faces of
the same device, so I'm not so sure. I'd be particularly surprised if
this worked by design rather than coincidence.

Based on docs/misc/vbd-interface.txt I wouldn't be all that surprised if
some guest kernels had a problem with this even if you could convince
the toolstack it was ok.

Ian.

Re: incorrect disk numbering with qemu

[Olaf Hering]

On Wed, Jan 15, Ian Campbell wrote:

> On Wed, 2014-01-15 at 18:12 +0100, Olaf Hering wrote:
> > It seems qemu does not enumerate the configured disks correctly with a
> > config like this:
> > 
> > disk=[  
> >         'raw:/some.iso,hda:cdrom,r',
> >         'raw:/some.raw,xvda,w',
> > ]
> > 
> > With a PV guest it works fine, the guest has a hda and xvda device. 
> > But a HVM guest fails to start:
> > qemu-system-i386: -drive file=/some.raw,if=ide,index=0,media=disk,format=raw,cache=writeback: drive with bus=0, unit=0 (index=0) exists
> > 
> > I think that kind of config used to work with xend.
> 
> Did it? I thought xvda and hda were effectively considered two faces of
> the same device, so I'm not so sure. I'd be particularly surprised if
> this worked by design rather than coincidence.

Putting a 'device_model_version="qemu-xen-traditional"' into the config
fixes it for me.

So the question is how it is supposed to work. My understanding is that
for HVM some sort of IDE is (or was?) required to let it boot from a
block device. Thats why I have hd[abcd] as device name. In addition to
that one could have as many disks named xvd[abc..], which are PV only.

After some testing it seems that today the guest will boot from xvda,
even with qemu-xen-traditional. So either that got fixed with libxl, or
xend from 4.2 got it all wrong.

So what should be done with such configs, if they really exist in the
wild? The obvious workaround is device_model_version="qemu-xen-traditional".

Olaf

Re: incorrect disk numbering with qemu

[Ian Campbell]

On Wed, 2014-01-15 at 18:45 +0100, Olaf Hering wrote:
> On Wed, Jan 15, Ian Campbell wrote:
> 
> > On Wed, 2014-01-15 at 18:12 +0100, Olaf Hering wrote:
> > > It seems qemu does not enumerate the configured disks correctly with a
> > > config like this:
> > > 
> > > disk=[  
> > >         'raw:/some.iso,hda:cdrom,r',
> > >         'raw:/some.raw,xvda,w',
> > > ]
> > > 
> > > With a PV guest it works fine, the guest has a hda and xvda device. 
> > > But a HVM guest fails to start:
> > > qemu-system-i386: -drive file=/some.raw,if=ide,index=0,media=disk,format=raw,cache=writeback: drive with bus=0, unit=0 (index=0) exists
> > > 
> > > I think that kind of config used to work with xend.
> > 
> > Did it? I thought xvda and hda were effectively considered two faces of
> > the same device, so I'm not so sure. I'd be particularly surprised if
> > this worked by design rather than coincidence.
> 
> Putting a 'device_model_version="qemu-xen-traditional"' into the config
> fixes it for me.
> 
> So the question is how it is supposed to work. My understanding is that
> for HVM some sort of IDE is (or was?) required to let it boot from a
> block device. Thats why I have hd[abcd] as device name. In addition to
> that one could have as many disks named xvd[abc..], which are PV only.

Not quite. Each xvd[a-d] creates both a PV and an emulated IDE device
hd[a-d], which refer to the same underlying volume.

This allows you to boot from hda, do an unplug and then switch to using
xvda.

If you want a pure PV disk you should label it xvde onwards as
docs/misc/vbd-interface.txt suggests. I don't think there is a way to
force xvd[a-d] to be pure PV.

Obviously this precludes you explicitly asking for both xvda and hda,
since there will be a clash. I don't know how/why qemu-trad lets this
work.

> After some testing it seems that today the guest will boot from xvda,
> even with qemu-xen-traditional.

So basically the bogus second definition of hda as a cdrom is ignored?

>  So either that got fixed with libxl, or
> xend from 4.2 got it all wrong.
> 
> So what should be done with such configs, if they really exist in the
> wild?

>  The obvious workaround is device_model_version="qemu-xen-traditional".

Yes, either that or switch the disk stuff around / delete the hda.

General advice would be to stick with qemu-xen-traditional for vms which
were created/installed with it, unless you know the in guest OS to be
fine, I suppose that could extend to this case too.

Ian.

Re: incorrect disk numbering with qemu

[Olaf Hering]

On Thu, Jan 16, Ian Campbell wrote:

> Not quite. Each xvd[a-d] creates both a PV and an emulated IDE device
> hd[a-d], which refer to the same underlying volume.
> 
> This allows you to boot from hda, do an unplug and then switch to using
> xvda.

Not really: In the end hda is connected to the emulated IDE, so today
its really "sda" in domU because pata_piix will drive it. sda was
connected to emulated LSI SCSI. But xvda was not connected to any
emulated controller, its PV only. Thats how it is done with qemu-trad.
So having hda and xvda in the same config was working, and maybe even
supported?

With qemu-upstream this appearently changed. I'm not saying this change
in behaviour is good or bad, just that something changed. Some people
still use the kernel names instead of UUID or LABEL. So they have to
adjust their config in domU and also in domU.cfg before they switch from
qemu-trad to qemu-upstream.


Olaf

Re: incorrect disk numbering with qemu

[Ian Campbell]

On Thu, 2014-01-16 at 12:44 +0100, Olaf Hering wrote:
> On Thu, Jan 16, Ian Campbell wrote:
> 
> > Not quite. Each xvd[a-d] creates both a PV and an emulated IDE device
> > hd[a-d], which refer to the same underlying volume.
> > 
> > This allows you to boot from hda, do an unplug and then switch to using
> > xvda.
> 
> Not really: In the end hda is connected to the emulated IDE, so today
> its really "sda" in domU because pata_piix will drive it.

true.

> sda was connected to emulated LSI SCSI.

Where did this come from? Did you request it somehow? I don't believe
there was ever an emulated SCSI controller by default.

Are you sure that this sda wasn't referring to the same backing volume
as xvda?

> But xvda was not connected to any
> emulated controller, its PV only.

Are you sure it wasn't showing up with a different name? Or perhaps by
enabling LSI SCSI you have suppressed the default IDE controllers.

>  Thats how it is done with qemu-trad.

It was not *the* way though.

> So having hda and xvda in the same config was working, and maybe even
> supported?

Maybe working, but not supported IMHO.

> With qemu-upstream this appearently changed. I'm not saying this change
> in behaviour is good or bad, just that something changed. Some people
> still use the kernel names instead of UUID or LABEL. So they have to
> adjust their config in domU and also in domU.cfg before they switch from
> qemu-trad to qemu-upstream.
> 
> 
> Olaf

Re: incorrect disk numbering with qemu

[Olaf Hering]

On Thu, Jan 16, Ian Campbell wrote:

> > sda was connected to emulated LSI SCSI.
> 
> Where did this come from? Did you request it somehow? I don't believe
> there was ever an emulated SCSI controller by default.
> 
> Are you sure that this sda wasn't referring to the same backing volume
> as xvda?
 
Giving sda as name will instruct qemu-trad to create a LSI controller.
At least with SLES it can not be used because the unplug code shuts down
the controller and that also removes the disks. And the PV driver does
not claim them. Something like that, I dont remember the details.

> > But xvda was not connected to any
> > emulated controller, its PV only.
> 
> Are you sure it wasn't showing up with a different name? Or perhaps by
> enabling LSI SCSI you have suppressed the default IDE controllers.

No, having hda, sda and xvda would (in theory) give three disks(IDE,
SCSI and PV). And with just xvda qemu-trad will not boot because its not
connected to an emulated controller.


Olaf

Re: incorrect disk numbering with qemu

[Stefano Stabellini]

On Thu, 16 Jan 2014, Olaf Hering wrote:
> On Thu, Jan 16, Ian Campbell wrote:
> 
> > Not quite. Each xvd[a-d] creates both a PV and an emulated IDE device
> > hd[a-d], which refer to the same underlying volume.
> > 
> > This allows you to boot from hda, do an unplug and then switch to using
> > xvda.
> 
> Not really: In the end hda is connected to the emulated IDE, so today
> its really "sda" in domU because pata_piix will drive it. sda was
> connected to emulated LSI SCSI. But xvda was not connected to any
> emulated controller, its PV only. Thats how it is done with qemu-trad.
> So having hda and xvda in the same config was working, and maybe even
> supported?
> 
> With qemu-upstream this appearently changed. I'm not saying this change
> in behaviour is good or bad, just that something changed. Some people
> still use the kernel names instead of UUID or LABEL. So they have to
> adjust their config in domU and also in domU.cfg before they switch from
> qemu-trad to qemu-upstream.

This behaviour was chosen on purpose 3 years ago, knowing that it might
cause problems to people that try to use xvda and hda in the same
config.

Why? Because all the alternatives that we considered were far worse.
You can have fun digging through the archives, you might discover things
like xvdHD that might have even made it into Linux for a short while.

Like Ian said, SCSI devices get a name from xvde onward.

Re: incorrect disk numbering with qemu

[Jan Beulich]

>>> On 16.01.14 at 16:15, Stefano Stabellini <stefano.stabellini@eu.citrix.com> wrote:
> This behaviour was chosen on purpose 3 years ago, knowing that it might
> cause problems to people that try to use xvda and hda in the same
> config.
> 
> Why? Because all the alternatives that we considered were far worse.
> You can have fun digging through the archives, you might discover things
> like xvdHD that might have even made it into Linux for a short while.

And IIRC all this was done just because the pv-ops incarnation
of blkfront was stripped off the respective device name mapping
(i.e. if all of what XenoLinux'es blkfront did was properly ported
over, no such behavioral change would have been needed).

(Whether what the old blkfront did was nice is a completely
separate question.)

Jan