xen 3.4.0 & pvgrub

xen 3.4.0 & pvgrub

[Mickael Marchand]

Hi,

I have built ubuntu packages for Xen 3.4.0
(https://launchpad.net/~mikmak/+archive/experimental), 
they apparently work fine except when using pvgrub.

booting a domU with kernel/ramdisk in the config file works fine,
booting the same domU with pvgrub (and same kernel inside the domU) it's
not able to find sda1 or xvda1, /dev looks empty in regards to disk
devices
it correctly loads grub, grub boots the kernel and initrd, but it's not
able to mount any root (root=/dev/xvda1 or sda1, ls /dev inside the
initrd shows no sd or xvd disks)

any idea what could be missing ?

(please CC me answers)

Cheers,
Mik

Config example :
#works fine with these 3 lines
#kernel="/boot/vmlinuz-2.6.24-24-xen"
#ramdisk="/boot/initrd.img-2.6.24-24-xen"
#root = '/dev/xvda1 ro'
#does not work with these 2
kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz"
extra = "(hd0)/boot/grub/menu.lst"

#usual stuff
memory      = '1024'
vcpus    = '2'
disk        = [
                  'phy:/dev/bay2/jaunty-disk,xvda1,w',
                  'phy:/dev/bay2/jaunty-swap,xvda2,w',
              ]
name        = 'jaunty'
vif         = [ 'mac=00:16:3E:FF:FC:2D' ]
on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash    = 'restart'

xen 3.4.0 & pvgrub

[Mickael Marchand]

Hi,

I have built ubuntu packages for Xen 3.4.0
(https://launchpad.net/~mikmak/+archive/experimental), 
they apparently work fine except when using pvgrub.

booting a domU with kernel/ramdisk in the config file works fine,
booting the same domU with pvgrub (and same kernel inside the domU) it's
not able to find sda1 or xvda1, /dev looks empty in regards to disk
devices
it correctly loads grub, grub boots the kernel and initrd, but it's not
able to mount any root (root=/dev/xvda1 or sda1, ls /dev inside the
initrd shows no sd or xvd disks)

any idea what could be missing ?

Cheers,
Mik

Config example :
#works fine with these 3 lines
#kernel="/boot/vmlinuz-2.6.24-24-xen"
#ramdisk="/boot/initrd.img-2.6.24-24-xen"
#root = '/dev/xvda1 ro'
#does not work with these 2
kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz"
extra = "(hd0)/boot/grub/menu.lst"

#usual stuff
memory      = '1024'
vcpus    = '2'
disk        = [
                  'phy:/dev/bay2/jaunty-disk,xvda1,w',
                  'phy:/dev/bay2/jaunty-swap,xvda2,w',
              ]
name        = 'jaunty'
vif         = [ 'mac=00:16:3E:FF:FC:2D' ]
on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash    = 'restart'

Re: xen 3.4.0 & pvgrub

[Samuel Thibault]

Mickael Marchand, le Tue 19 May 2009 18:01:14 +0200, a écrit :
> booting the same domU with pvgrub (and same kernel inside the domU) it's
> not able to find sda1 or xvda1, /dev looks empty in regards to disk
> devices

Mmm, your dom0 probably has a kernel that doesn't cope well with
reconnecting devices, which kernel does it have ?

Samuel

Re: xen 3.4.0 & pvgrub

[Mickael Marchand]

On Thu, May 21, 2009 at 07:22:13PM +0200, Samuel Thibault wrote:
> Mickael Marchand, le Tue 19 May 2009 18:01:14 +0200, a écrit :
> > booting the same domU with pvgrub (and same kernel inside the domU) it's
> > not able to find sda1 or xvda1, /dev looks empty in regards to disk
> > devices
> 
> Mmm, your dom0 probably has a kernel that doesn't cope well with
> reconnecting devices, which kernel does it have ?

the standard 2.6.24-24-xen from ubuntu,
it works fine with xen 3.3 afaict (using it in production)

Cheers,
Mik

> 
> Samuel

Re: xen 3.4.0 & pvgrub

[Samuel Thibault]

Mickael Marchand, le Thu 21 May 2009 20:19:53 +0200, a écrit :
> On Thu, May 21, 2009 at 07:22:13PM +0200, Samuel Thibault wrote:
> > Mickael Marchand, le Tue 19 May 2009 18:01:14 +0200, a écrit :
> > > booting the same domU with pvgrub (and same kernel inside the domU) it's
> > > not able to find sda1 or xvda1, /dev looks empty in regards to disk
> > > devices
> > 
> > Mmm, your dom0 probably has a kernel that doesn't cope well with
> > reconnecting devices, which kernel does it have ?
> 
> the standard 2.6.24-24-xen from ubuntu,
> it works fine with xen 3.3 afaict (using it in production)

With pv-grub, you mean?  I.e. with the same dom0 kernel, an upgraded
hypervisor, and the same domU kernel, you now have issues with pv-grub
while you didn't have any before?

Samuel

Re: xen 3.4.0 & pvgrub

[Mickael Marchand]

On Thu, May 21, 2009 at 08:22:07PM +0200, Samuel Thibault wrote:
> Mickael Marchand, le Thu 21 May 2009 20:19:53 +0200, a écrit :
> > On Thu, May 21, 2009 at 07:22:13PM +0200, Samuel Thibault wrote:
> > > Mickael Marchand, le Tue 19 May 2009 18:01:14 +0200, a écrit :
> > > > booting the same domU with pvgrub (and same kernel inside the domU) it's
> > > > not able to find sda1 or xvda1, /dev looks empty in regards to disk
> > > > devices
> > > 
> > > Mmm, your dom0 probably has a kernel that doesn't cope well with
> > > reconnecting devices, which kernel does it have ?
> > 
> > the standard 2.6.24-24-xen from ubuntu,
> > it works fine with xen 3.3 afaict (using it in production)
> 
> With pv-grub, you mean?  I.e. with the same dom0 kernel, an upgraded
> hypervisor, and the same domU kernel, you now have issues with pv-grub
> while you didn't have any before?

yes, just tested again to confirm (I have a 2 nodes cluster, one xen
3.3, one xen 3.4)
first node (3.3) : boots fine
second node (3.4): same config file, same disk image file : no xvda1 or
sda1 found

Cheers,
Mik

> 
> Samuel

Re: xen 3.4.0 & pvgrub

[Mickael Marchand]

On Thu, May 21, 2009 at 08:22:07PM +0200, Samuel Thibault wrote:
> Mickael Marchand, le Thu 21 May 2009 20:19:53 +0200, a écrit :
> > On Thu, May 21, 2009 at 07:22:13PM +0200, Samuel Thibault wrote:
> > > Mickael Marchand, le Tue 19 May 2009 18:01:14 +0200, a écrit :
> > > > booting the same domU with pvgrub (and same kernel inside the domU) it's
> > > > not able to find sda1 or xvda1, /dev looks empty in regards to disk
> > > > devices
> > > 
> > > Mmm, your dom0 probably has a kernel that doesn't cope well with
> > > reconnecting devices, which kernel does it have ?
> > 
> > the standard 2.6.24-24-xen from ubuntu,
> > it works fine with xen 3.3 afaict (using it in production)
> 
> With pv-grub, you mean?  I.e. with the same dom0 kernel, an upgraded
> hypervisor, and the same domU kernel, you now have issues with pv-grub
> while you didn't have any before?
> 
> Samuel

* Xen 3.3.1 :
[    0.096978] Event-channel device installed.                                                                                                                        
[    0.102662] Successfully initialized TPM backend driver.                                                                                                           
[    0.105036] netfront: Initialising virtual ethernet driver.                                                                                                        
[   19.156897] xen-vbd: registered block device major 202                                                                                                             
[   19.156918] blkfront: xvda1: barriers enabled                                                                                                                      
[    0.141005] rtc: IRQ 8 is not free.

* Xen 3.4.0 :
[    0.091188] Event-channel device installed.                                                                                                                        
[    0.096703] Successfully initialized TPM backend driver.                                                                                                           
[    0.096920] netfront: Initialising virtual ethernet driver.                                                                                                        
[    0.132989] rtc: IRQ 8 is not free. 

missing the 2 lines about xen-vbd and blkfront apparently

Mik