More on domU not starting

More on domU not starting

[Rob Gardner]

I get the following warning when running xend start, and a similar 
warning sometimes when doing xm commands:

/usr/lib/python/xen/xend/XendNode.py:26: RuntimeWarning: Python C API 
version mismatch for module xen.lowlevel.xc: This Python has API version 
1012, module xen.lowlevel.xc has version 1011.
  import xen.lowlevel.xc
/usr/lib/python/xen/xend/xenstore/xstransact.py:10: RuntimeWarning: 
Python C API version mismatch for module xen.lowlevel.xs: This Python 
has API version 1012, module xen.lowlevel.xs has version 1011.
  from xen.lowlevel import xs

Xend does seem to start ok, and xm seems to run ok. But domU still can't 
find its boot disk. In the boot messages I see:

xen_blk: Initialising virtual block device driver

But following that I see no message like 
  Registering block device major 3
which I normally see on a working system.

The non-working system has python 2.3.4 on it. Could this be a problem?

My disk config is:
disk = [ 'file:/xen/disk-images/dom1.fs,sda1,w']
The strange thing is if I change the filename to something non-existent, 
the behavior stays the same. The disk image file does exist and is in 
good shape.



Rob

Re: More on domU not starting

[Ryan Grimm]

I had a similar problem and was able to rectify it by including the 
following line:

disk = [ 'phy:/dev/sda3,hda1,w' ]

which, as stated in the user doc in section 5.1, will export /dev/sda3 
(the partition where my image is) to the domU as hda1.

then, my disk config is:

disk = [ 'file:/path/to/image,hda1,w' ]

also, the user doc's example in section 5.1 says that disk = [ 
'phy:sda3,hda1,w' ] would work but it did not unless i put /dev/ in front 
of sda3, like above.

ryan

On Tue, 11 Oct 2005, Rob Gardner wrote:

> Xend does seem to start ok, and xm seems to run ok. But domU still can't find 
> its boot disk. In the boot messages I see:
>
> xen_blk: Initialising virtual block device driver
>
> But following that I see no message like  Registering block device major 3
> which I normally see on a working system.
>
> The non-working system has python 2.3.4 on it. Could this be a problem?
>
> My disk config is:
> disk = [ 'file:/xen/disk-images/dom1.fs,sda1,w']
> The strange thing is if I change the filename to something non-existent, the 
> behavior stays the same. The disk image file does exist and is in good shape.
>
>
>
> Rob
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

Re: More on domU not starting

[Rob Gardner]

Ryan Grimm wrote:

>
> I had a similar problem and was able to rectify it by including the 
> following line:
>
> disk = [ 'phy:/dev/sda3,hda1,w' ]
>
> which, as stated in the user doc in section 5.1, will export /dev/sda3 
> (the partition where my image is) to the domU as hda1.
>
> then, my disk config is:
>
> disk = [ 'file:/path/to/image,hda1,w' ]
>
> also, the user doc's example in section 5.1 says that disk = [ 
> 'phy:sda3,hda1,w' ] would work but it did not unless i put /dev/ in 
> front of sda3, like above.
>


I don't understand that. If you have two "disk = " lines, won't the 
second simply supercede the first? I tried exporting the partition 
containing the image like this:
   disk = [ 'phy:/dev/hda6,hda6,w',
          'file:/xen/disk-images/dom1.fs,sda1,w']
...but no luck there.


Rob

Re: More on domU not starting

[Ryan Grimm]

I don't know what the desired behavior should be, but good point about the 
two disk statements...i am able to now get the desired behavior by using 
one like you suggested.

mine now looks like:

disk = [ 'phy:sda3,hda1,w', 'file:/path/to/image,hda1,w' ]

so i dont need the /dev in front of sda3 after all...but i think that if 
you export hda6 as hda6, you need to specify hda6 instead of sda1.  so i 
think your disk and root statements should be:

disk = [ 'phy:hda6,hda6,w',
          'file:/xen/disk-images/dom1.fs,hda6,w']
root = "/dev/hda6 ro"

at least that is what works on my machine :)

ryan


On Tue, 11 Oct 2005, Rob Gardner wrote:

> Ryan Grimm wrote:
>
>> 
>> I had a similar problem and was able to rectify it by including the 
>> following line:
>> 
>> disk = [ 'phy:/dev/sda3,hda1,w' ]
>> 
>> which, as stated in the user doc in section 5.1, will export /dev/sda3 (the 
>> partition where my image is) to the domU as hda1.
>> 
>> then, my disk config is:
>> 
>> disk = [ 'file:/path/to/image,hda1,w' ]
>> 
>> also, the user doc's example in section 5.1 says that disk = [ 
>> 'phy:sda3,hda1,w' ] would work but it did not unless i put /dev/ in front 
>> of sda3, like above.
>> 
>
>
> I don't understand that. If you have two "disk = " lines, won't the second 
> simply supercede the first? I tried exporting the partition containing the 
> image like this:
>  disk = [ 'phy:/dev/hda6,hda6,w',
>         'file:/xen/disk-images/dom1.fs,sda1,w']
> ...but no luck there.
>
>
> Rob
>
>
>

Re: More on domU not starting -- HELP

[Rob Gardner]

Ryan Grimm wrote:

>
> I don't know what the desired behavior should be, but good point about 
> the two disk statements...i am able to now get the desired behavior by 
> using one like you suggested.
>
> mine now looks like:
>
> disk = [ 'phy:sda3,hda1,w', 'file:/path/to/image,hda1,w' ]
>
> so i dont need the /dev in front of sda3 after all...but i think that 
> if you export hda6 as hda6, you need to specify hda6 instead of sda1.  
> so i think your disk and root statements should be:
>
> disk = [ 'phy:hda6,hda6,w',
>          'file:/xen/disk-images/dom1.fs,hda6,w']
> root = "/dev/hda6 ro"


If you want to export a file to a domain, I don't think you need to also 
export the underlying raw disk to the domain. At least you never had to 
do that before.

Anyway, I tried this, and  it still does not fix my problem. No matter 
what I do, the domain does not see any virtual block device and so 
cannot boot. This must be due to some new behavior of some component. 
I've got an older installation of xen-unstable from Sep. 21 and it works 
fine with the identical config file.

Can anyone suggest a clue to this problem?????  Boot messages attached 
below.

Rob


[root@localhost xen]# xm create -f /xen/dom1-3.0 vmid=1 -c
Using config file "/xen/dom1-3.0".
Started domain Rob-VM1
Linux version 2.6.12-xenU (rdg@cordoba) (gcc version 3.2.2 20030222 (Red 
Hat Linux 3.2.2-5)) #1 SMP Wed Oct 12 09:37:41 MDT 2005
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 0000000008000000 (usable)
0MB HIGHMEM available.
128MB LOWMEM available.
IRQ lockup detection disabled
Built 1 zonelists
Kernel command line:  
ip=15.6.247.101:1.2.3.4:15.6.247.1:255.255.255.0:vm1:eth0:off 
root=/dev/sda1 ro 4 VMID=1
Initializing CPU#0
PID hash table entries: 1024 (order: 10, 16384 bytes)
Xen reported: 2660.168 MHz processor.
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
vmalloc area: c8800000-fb7fe000, maxmem 34000000
Memory: 126336k/131072k available (1771k kernel code, 4476k reserved, 
584k data, 140k init, 0k highmem)
Checking if this processor honours the WP bit even in supervisor mode... Ok.
Mount-cache hash table entries: 512
CPU: Trace cache: 12K uops, L1 D cache: 8K
CPU: L2 cache: 512K
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Checking 'hlt' instruction... disabled
CPU0: Intel(R) Pentium(R) 4 CPU 2.66GHz stepping 09
Total of 1 processors activated (5308.41 BogoMIPS).
Brought up 1 CPUs
NET: Registered protocol family 16
xen_mem: Initialising balloon driver.
Grant table initialized
Initializing Cryptographic API
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
Xen virtual console successfully installed as tty1
Event-channel device installed.
xen_blk: Initialising virtual block device driver
xen_net: Initialising virtual ethernet driver.
NET: Registered protocol family 2
IP: routing cache hash table of 512 buckets, 8Kbytes
TCP established hash table entries: 8192 (order: 5, 131072 bytes)
TCP bind hash table entries: 8192 (order: 4, 98304 bytes)
TCP: Hash tables configured (established 8192 bind 8192)
NET: Registered protocol family 1
NET: Registered protocol family 17
IP-Config: Complete:
      device=eth0, addr=15.6.247.101, mask=255.255.255.0, gw=15.6.247.1,
     host=vm1, domain=, nis-domain=(none),
     bootserver=1.2.3.4, rootserver=1.2.3.4, rootpath=
Looking up port of RPC 100003/2 on 1.2.3.4

Re: More on domU not starting -- HELP

[Ted Kaczmarek]

On Wed, 2005-10-12 at 14:57 -0600, Rob Gardner wrote:
> Ryan Grimm wrote:
> 
> >
> > I don't know what the desired behavior should be, but good point about 
> > the two disk statements...i am able to now get the desired behavior by 
> > using one like you suggested.
> >
> > mine now looks like:
> >
> > disk = [ 'phy:sda3,hda1,w', 'file:/path/to/image,hda1,w' ]
> >
> > so i dont need the /dev in front of sda3 after all...but i think that 
> > if you export hda6 as hda6, you need to specify hda6 instead of sda1.  
> > so i think your disk and root statements should be:
> >
> > disk = [ 'phy:hda6,hda6,w',
> >          'file:/xen/disk-images/dom1.fs,hda6,w']
> > root = "/dev/hda6 ro"
> 

Do not or use any existing designations or reuse any for domU's.
In your case I am assuming you have a /dev/hda.

edit the domU fstab to /dev/sda1 for / and dev/sda2 for swap and the
domu config accordingly.

Do not use sda1 or sda2 for any additional domU's.

I will open up a bug on this shortly, this was bugging me for a while,
but with this workaround my domu's now come up 100% of the time every
time.
 
Regards,
Ted

Re: More on domU not starting -- HELP

[Rob Gardner]

Ted Kaczmarek wrote:

>
>Do not or use any existing designations or reuse any for domU's.
>In your case I am assuming you have a /dev/hda.
>
>edit the domU fstab to /dev/sda1 for / and dev/sda2 for swap and the
>domu config accordingly.
>
>Do not use sda1 or sda2 for any additional domU's.
>
>I will open up a bug on this shortly, this was bugging me for a while,
>but with this workaround my domu's now come up 100% of the time every
>time.
>  
>

What was the exact failure mode you observed? It doesn't seem like it's 
the same problem I'm seeing...

My domU doesn't get booted to anywhere near the point where fstab would 
make a difference. After the message:
    xen_blk: Initialising virtual block device driver
There is no message like:
   Registering block device major xxx
Which looks to me like the kernel is simply not finding any block device 
at all. Presumably this is because xen didn't provide one for it. Why it 
does that, I don't know. But it used to work just fine a few weeks ago.

Rob

Re: More on domU not starting -- HELP

[Ted Kaczmarek]

On Wed, 2005-10-12 at 15:48 -0600, Rob Gardner wrote:
> Ted Kaczmarek wrote:
> 
> >
> >Do not or use any existing designations or reuse any for domU's.
> >In your case I am assuming you have a /dev/hda.
> >
> >edit the domU fstab to /dev/sda1 for / and dev/sda2 for swap and the
> >domu config accordingly.
> >
> >Do not use sda1 or sda2 for any additional domU's.
> >
> >I will open up a bug on this shortly, this was bugging me for a while,
> >but with this workaround my domu's now come up 100% of the time every
> >time.
> >  
> >
> 
> What was the exact failure mode you observed? It doesn't seem like it's 
> the same problem I'm seeing...
> 
> My domU doesn't get booted to anywhere near the point where fstab would 
> make a difference. After the message:
>     xen_blk: Initialising virtual block device driver
> There is no message like:
>    Registering block device major xxx
> Which looks to me like the kernel is simply not finding any block device 
> at all. Presumably this is because xen didn't provide one for it. Why it 
> does that, I don't know. But it used to work just fine a few weeks ago.
> 
> Rob
> 
Mine where failing at unable to mount root device so it is likely you
are having something else happen, unless the pollution is related.

What changeset are you using?

Did your kernel config work before?

My Xen domO

CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_PHYSDEV_ACCESS=y
CONFIG_XEN_BLKDEV_BACKEND=y
# CONFIG_XEN_BLKDEV_TAP_BE is not set
CONFIG_XEN_NETDEV_BACKEND=y
# CONFIG_XEN_TPMDEV_FRONTEND is not set
# CONFIG_XEN_TPMDEV_BACKEND is not set
CONFIG_XEN_BLKDEV_FRONTEND=y
# CONFIG_XEN_NETDEV_FRONTEND is not set
# CONFIG_XEN_BLKDEV_TAP is not set
# CONFIG_XEN_SHADOW_MODE is not set
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_X86=y
# CONFIG_XEN_X86_64 is not set
CONFIG_HAVE_ARCH_DEV_ALLOC_SKB=y


My xen domU

# CONFIG_XEN_PRIVILEGED_GUEST is not set
# CONFIG_XEN_PHYSDEV_ACCESS is not set
# CONFIG_XEN_TPMDEV_FRONTEND is not set
# CONFIG_XEN_TPMDEV_BACKEND is not set
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_NETDEV_FRONTEND=y
# CONFIG_XEN_NETDEV_FRONTEND_PIPELINED_TRANSMITTER is not set
# CONFIG_XEN_BLKDEV_TAP is not set
# CONFIG_XEN_SHADOW_MODE is not set
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_X86=y
# CONFIG_XEN_X86_64 is not set
CONFIG_HAVE_ARCH_DEV_ALLOC_SKB=y


Might also help to try a loop image as well and see if that will play
nice.

Regards,
Ted

Re: More on domU not starting -- HELP

[Rob Gardner]

Ted Kaczmarek wrote:

>
>What changeset are you using?
>  
>
4e335372ace8

Note that I got this by just doing 'hg clone 
http://xenbits.xensource.com/xen-unstable.hg'

I did not change a thing, just did a make world and installed the stuff.

(Am I wrong to assume that this procedure will produce a xen that works?)

>Did your kernel config work before?
>  
>

I had no special config, just was using whatever the default is in the 
repository I cloned.

>My Xen domO
>
>CONFIG_XEN_PRIVILEGED_GUEST=y
>CONFIG_XEN_PHYSDEV_ACCESS=y
>CONFIG_XEN_BLKDEV_BACKEND=y
># CONFIG_XEN_BLKDEV_TAP_BE is not set
>CONFIG_XEN_NETDEV_BACKEND=y
># CONFIG_XEN_TPMDEV_FRONTEND is not set
># CONFIG_XEN_TPMDEV_BACKEND is not set
>CONFIG_XEN_BLKDEV_FRONTEND=y
># CONFIG_XEN_NETDEV_FRONTEND is not set
># CONFIG_XEN_BLKDEV_TAP is not set
># CONFIG_XEN_SHADOW_MODE is not set
>CONFIG_XEN_SCRUB_PAGES=y
>CONFIG_XEN_X86=y
># CONFIG_XEN_X86_64 is not set
>CONFIG_HAVE_ARCH_DEV_ALLOC_SKB=y
>  
>
My xen dom0:

CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_PHYSDEV_ACCESS=y
CONFIG_XEN_BLKDEV_BACKEND=y
# CONFIG_XEN_BLKDEV_TAP_BE is not set
CONFIG_XEN_NETDEV_BACKEND=y
# CONFIG_XEN_TPMDEV_FRONTEND is not set
# CONFIG_XEN_TPMDEV_BACKEND is not set
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_NETDEV_FRONTEND=y
# CONFIG_XEN_NETDEV_FRONTEND_PIPELINED_TRANSMITTER is not set
# CONFIG_XEN_BLKDEV_TAP is not set
# CONFIG_XEN_SHADOW_MODE is not set
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_X86=y
# CONFIG_XEN_X86_64 is not set
CONFIG_HAVE_ARCH_DEV_ALLOC_SKB=y



>
>My xen domU
>
># CONFIG_XEN_PRIVILEGED_GUEST is not set
># CONFIG_XEN_PHYSDEV_ACCESS is not set
># CONFIG_XEN_TPMDEV_FRONTEND is not set
># CONFIG_XEN_TPMDEV_BACKEND is not set
>CONFIG_XEN_BLKDEV_FRONTEND=y
>CONFIG_XEN_NETDEV_FRONTEND=y
># CONFIG_XEN_NETDEV_FRONTEND_PIPELINED_TRANSMITTER is not set
># CONFIG_XEN_BLKDEV_TAP is not set
># CONFIG_XEN_SHADOW_MODE is not set
>CONFIG_XEN_SCRUB_PAGES=y
>CONFIG_XEN_X86=y
># CONFIG_XEN_X86_64 is not set
>CONFIG_HAVE_ARCH_DEV_ALLOC_SKB=y
>
>  
>

My xen domU:

# CONFIG_XEN_PRIVILEGED_GUEST is not set
# CONFIG_XEN_PHYSDEV_ACCESS is not set
# CONFIG_XEN_TPMDEV_FRONTEND is not set
# CONFIG_XEN_TPMDEV_BACKEND is not set
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_NETDEV_FRONTEND=y
# CONFIG_XEN_NETDEV_FRONTEND_PIPELINED_TRANSMITTER is not set
# CONFIG_XEN_BLKDEV_TAP is not set
# CONFIG_XEN_SHADOW_MODE is not set
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_X86=y
# CONFIG_XEN_X86_64 is not set
CONFIG_HAVE_ARCH_DEV_ALLOC_SKB=y




>Might also help to try a loop image as well and see if that will play
>nice.
>  
>

That's what I'm doing. My config file looks like this:
  ...snip...
xm_vars.check()
kernel = "/boot/vmlinuz-2.6.12-xenU"
memory = 128
name = "Rob-VM%d" % vmid
cpu = vmid  # set based on vmid (mod number of CPUs)
nics=1
disk = [ 'file:/xen/disk-images/dom1.fs,sda1,w']
ip = add_offset_to_ip(get_current_ipaddr(),vmid)
netmask = get_current_ipmask()
gateway = get_current_ipgw()
hostname= "vm%d" % vmid
root = "/dev/sda1 ro"
extra = "4 VMID=%d" % vmid

Here's another bit of info... running 'xm block-list' on the domU that 
doesn't boot successfully gives:
   (2065 ((virtual-device 2065) (backend-id 0) (backend 
/local/domain/0/backend/vbd/1/2065) (ring-ref 8) (event-channel 6) 
(error '2 reading   backend fields at /local/domain/0/backend/vbd/1/2065')))



Rob

RE: More on domU not starting -- HELP

[Deuskar, Prafulla]

>Ted Kaczmarek wrote:
>
>>
>>What changeset are you using?
>>
>>
>4e335372ace8
>
>Note that I got this by just doing 'hg clone
>http://xenbits.xensource.com/xen-unstable.hg'
>
>I did not change a thing, just did a make world and installed the
stuff.
>
>(Am I wrong to assume that this procedure will produce a xen that
works?)
>
>>Did your kernel config work before?
>>
>>
>
>I had no special config, just was using whatever the default is in the
>repository I cloned.

I see an intermittent problem on my box (Redhat AS update 1) when I try
to create a guest domain. This is with yesterday's tip.

Again I just did an 'hg clone' of the xen unstable repository.

When it doesn't work xen tries to mount the root file system over NFS.
Looks like a race condition to me.

Thanks,
Prafulla