Xen unprivileged domain not booting

Xen unprivileged domain not booting

[Will Andrews]

Hello,

I am using the Xen 1.3 development sources, from the tarball on
the website.  I was able to build and boot linux-2.4.26-xen0 as
Domain 0 on the beater box.  However, I am unable to boot the
unprivileged domain using a separate partition for its root
filesystem.  Both Domain 0 and Domain 1 are using Debian.

I downloaded the xeno-unstable-src.tgz tarball on August 12th.  I
can try a newer version if anyone thinks that will help.

Here's the dmesg & error I'm seeing while attempting to boot
the unprivileged domain (same error with the xenU kernel):

Linux version 2.4.26-xen0 (root@experiment-07) (gcc version 3.3.4 (Debian 1:3.3.4-6sarge1)) #5 Tue Aug 17 17:13:07 EST 2004
On node 0 totalpages: 24576                
zone(0): 4096 pages.       
zone(1): 20480 pages.
zone(2): 0 pages.    
Kernel command line:  ip=dhcp root=/dev/xda1 ro 4 root=/dev/xda1 ro
Initializing CPU#0                                                 
Xen reported: 547.629 MHz processor.
Calibrating delay loop... 837.22 BogoMIPS
Memory: 93572k/98304k available (2290k kernel code, 4732k reserved, 721k data, 104k init, 0k highmem)
Dentry cache hash table entries: 16384 (order: 5, 131072 bytes)
Inode cache hash table entries: 8192 (order: 4, 65536 bytes)   
Mount cache hash table entries: 512 (order: 0, 4096 bytes)  
Buffer cache hash table entries: 4096 (order: 2, 16384 bytes)
Page-cache hash table entries: 32768 (order: 5, 131072 bytes)
CPU: L1 I cache: 16K, L1 D cache: 16K                        
CPU: L2 cache: 512K                  
CPU: Intel Pentium III (Katmai) stepping 03
POSIX conformance testing by UNIFIX        
PCI: Probing PCI hardware          
Linux NET4.0 for Linux 2.4
Based upon Swansea University Computer Society NET3.039
Initializing RT netlink socket                         
Starting kswapd               
Journalled Block Device driver loaded
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
Event-channel device installed.                         
Xen virtual console successfully installed as tty
Starting Xen Balloon driver                      
pty: 256 Unix98 ptys configured
Initialising Xen virtual block device
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
loop: loaded (max 8 devices)                                         
Intel(R) PRO/1000 Network Driver - version 5.2.30.1-k1
Copyright (c) 1999-2004 Intel Corporation.            
pcnet32.c:v1.28 02.20.2004 tsbogend@alpha.franken.de
Uniform Multi-Platform E-IDE driver Revision: 7.00beta4-2.4
ide: Assuming 50MHz system bus speed for PIO modes; override with idebus=xx
hda: C/H/S=0/0/0 from BIOS ignored                                         
hdb: C/H/S=0/0/0 from BIOS ignored
SCSI subsystem driver Revision: 1.00
Red Hat/Adaptec aacraid driver (1.1-3 Aug 17 2004 15:57:49)
megaraid: v1.18k (Release Date: Thu Aug 28 10:05:11 EDT 2003)
megaraid: no BIOS enabled.                                   
scsi0 : SCSI host adapter emulation for IDE ATAPI devices
LVM version 1.0.8(17/11/2003)                            
Initializing Cryptographic API
Initialising Xen virtual ethernet frontend driver<6>NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP
IP: routing cache hash table of 512 buckets, 4Kbytes
TCP: Hash tables configured (established 8192 bind 16384)
ip_conntrack version 2.1 (768 buckets, 6144 max) - 288 bytes per conntrack
ip_tables: (C) 2000-2002 Netfilter core team                              
NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
NET4: Ethernet Bridge 008 for NET4.0               
Bridge firewalling registered       
Can't write to read-only device 0d:01
kjournald starting.  Commit interval 5 seconds
Can't write to read-only device 0d:01         
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.      
Freeing unused kernel memory: 104k freed     
INIT: version 2.86 booting              
Activating swap.
Checking root file system...
fsck 1.35 (28-Feb-2004)
/dev/xda1: clean, 14724/915712 files, 82579/1831410 blocks
Error writing block 520 (Attempt to write block from filesystem resulted in short write).  

/dev/xda1: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
        (i.e., without -a or -p options)
Error writing block 520 (Attempt to write block from filesystem resulted in short write).  
/dev/xda1: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
        (i.e., without -a or -p options)

fsck failed.  Please repair manually and reboot.  Please note
that the root file system is currently mounted read-only.  To
remount it read-write:

   # mount -n -o remount,rw /

CONTROL-D will exit from this shell and REBOOT the system.

Press enter for maintenance
(or type Control-D to continue): 

I checked the filesystem statistics it provided in Domain 0 and
it appears to be close, but not quite the exact same number of
files (including directories and other special files as "files").
Since I've never seen this error before, I really have no idea
what to do.

Regards,
-- 
wca
Purdue University Rosen Center for Advanced Computing
Office: MATH G135 @ Purdue - http://www.itap.purdue.edu/rcs/
NB: crypt.rcs.purdue.edu has retired recently, please use
    andrewsw@crypt.rcac.purdue.edu for all future contact.


-------------------------------------------------------
SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285

Re: Xen unprivileged domain not booting

[Ian Pratt]

> I am using the Xen 1.3 development sources, from the tarball on
> the website.  I was able to build and boot linux-2.4.26-xen0 as
> Domain 0 on the beater box.  However, I am unable to boot the
> unprivileged domain using a separate partition for its root
> filesystem.  Both Domain 0 and Domain 1 are using Debian.

Very odd. The 'short writes' suggest that there's some serious
disagreement about the size of the partition that's being
exported. 

> I downloaded the xeno-unstable-src.tgz tarball on August 12th.  I
> can try a newer version if anyone thinks that will help.

It might be worth a try, as there have been literally hundreds of
changesets since then!

BTW: which document did you get the 'xda' syntax from? I believe
'xda' still works, but I haven't used it myself in ages as it's
now possible to export a device to appear as a hd[abcd] or
sd[abcd] in the guest domain. This has the advantage that it
typically avoids the need to edit the /etc/fstab.

If you still have problems, please can you enable the debug flags
in drivers/xen/blkback/common.h and rebuild and try again. You
should get some messages appear in dom0 if you do a dmesg.

To help debug this you'll also need to post the output of
/proc/partitions and '/sbin/sfdisk -s' and 'tune2fs -l' on all
the relevant partitions. 

We'll also need to see the output from xend when the new domain
is being started.

Ian


-------------------------------------------------------
SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285

Re: Xen unprivileged domain not booting

[Will Andrews]

On Sat, Aug 21, 2004 at 08:24:22AM +0100, Ian Pratt wrote:
> Very odd. The 'short writes' suggest that there's some serious
> disagreement about the size of the partition that's being
> exported. 
> 
> > I downloaded the xeno-unstable-src.tgz tarball on August 12th.  I
> > can try a newer version if anyone thinks that will help.
> 
> It might be worth a try, as there have been literally hundreds of
> changesets since then!

OK, I tried the version available yesterday.  It had the same
problem, i.e. short writes.

> BTW: which document did you get the 'xda' syntax from? I believe
> 'xda' still works, but I haven't used it myself in ages as it's
> now possible to export a device to appear as a hd[abcd] or
> sd[abcd] in the guest domain. This has the advantage that it
> typically avoids the need to edit the /etc/fstab.

I get the same problem regardless of whether it's called xda or
hda.  I did not try xda with yesterday's snapshot.

> If you still have problems, please can you enable the debug flags
> in drivers/xen/blkback/common.h and rebuild and try again. You
> should get some messages appear in dom0 if you do a dmesg.

This file doesn't seem to be available for 2.4.26, which is what
I'm using.  I do find it in the 2.6.7 sparse tree.

Thanks for your time!

Regards,
-- 
wca
Purdue University Rosen Center for Advanced Computing
Office: MATH G135 @ Purdue - http://www.itap.purdue.edu/rcs/
NB: crypt.rcs.purdue.edu has retired recently, please use
    andrewsw@crypt.rcac.purdue.edu for all future contact.


-------------------------------------------------------
SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285

Re: Xen unprivileged domain not booting

[Will Andrews]

On Sat, Aug 21, 2004 at 08:24:22AM +0100, Ian Pratt wrote:
> BTW: which document did you get the 'xda' syntax from? I believe

Sorry, forgot to answer this in my response.  This syntax is in
the default xm config file.

Regards,
-- 
wca
Purdue University Rosen Center for Advanced Computing
Office: MATH G135 @ Purdue - http://www.itap.purdue.edu/rcs/
NB: crypt.rcs.purdue.edu has retired recently, please use
    andrewsw@crypt.rcac.purdue.edu for all future contact.


-------------------------------------------------------
SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285

Re: Xen unprivileged domain not booting

[Ian Pratt]

> > If you still have problems, please can you enable the debug flags
> > in drivers/xen/blkback/common.h and rebuild and try again. You
> > should get some messages appear in dom0 if you do a dmesg.
> 
> This file doesn't seem to be available for 2.4.26, which is what
> I'm using.  I do find it in the 2.6.7 sparse tree.

The file from the 2.6 tree is linked into the 2.4 build tree as
its common between both. (It makes it easier for us to maintain
than having two files).

Are you sure you're exporting the partition read-write rather
than read-only? I guess that might result in 'short write' messages.

Ian


-------------------------------------------------------
SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285

Re: Xen unprivileged domain not booting

[Niraj Tolia]

Will,

Could you modify your /etc/fstab for the domain you are booting (dom1)
and set the sixth (last) field to 0 for all the mounted partitions?
fsck will try to write to disks that are mounted read-only. Let us
know if that works.

Niraj

On Tue, 24 Aug 2004 11:52:08 -0500, Will Andrews
<andrewsw@crypt.rcac.purdue.edu> wrote:
> On Sat, Aug 21, 2004 at 08:24:22AM +0100, Ian Pratt wrote:
> > BTW: which document did you get the 'xda' syntax from? I believe
> 
> Sorry, forgot to answer this in my response.  This syntax is in
> the default xm config file.
> 
> 
> 
> Regards,
> --
> wca
> Purdue University Rosen Center for Advanced Computing
> Office: MATH G135 @ Purdue - http://www.itap.purdue.edu/rcs/
> NB: crypt.rcs.purdue.edu has retired recently, please use
>     andrewsw@crypt.rcac.purdue.edu for all future contact.
> 
> -------------------------------------------------------
> SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
> 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
> Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
> http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel
>


-------------------------------------------------------
SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285

Re: Xen unprivileged domain not booting

[Will Andrews]

On Tue, Aug 24, 2004 at 05:59:00PM +0100, Ian Pratt wrote:
> The file from the 2.6 tree is linked into the 2.4 build tree as
> its common between both. (It makes it easier for us to maintain
> than having two files).

Ah, ok.  I didn't realize there were files shared between the
sparse trees.  I am rebuilding the kernel now, just replying to
answer your question:

> Are you sure you're exporting the partition read-write rather
> than read-only? I guess that might result in 'short write' messages.

"Exporting" it?  It's just a filesystem I set aside for Xen
domain 1, it's not NFS mounted.

Thanks,
-- 
wca
Purdue University Rosen Center for Advanced Computing
Office: MATH G135 @ Purdue - http://www.itap.purdue.edu/rcs/
NB: crypt.rcs.purdue.edu has retired recently, please use
    andrewsw@crypt.rcac.purdue.edu for all future contact.


-------------------------------------------------------
SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285

Re: Xen unprivileged domain not booting

[Mark Williamson]

The block backend's common.h for 2.4.26 will be under 
arch/xen/drivers/blkback/common.h

Can you have a look at your configuration file and check the the domain is 
definitely being given write permission to its xda1?  Could you also post the 
config file in question to the list.

Cheers,
Mark



-------------------------------------------------------
SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285

Re: Xen unprivileged domain not booting

[Ian Pratt]

> > Are you sure you're exporting the partition read-write rather
> > than read-only? I guess that might result in 'short write' messages.
> 
> "Exporting" it?  It's just a filesystem I set aside for Xen
> domain 1, it's not NFS mounted.

'exporting' is the term we use when telling xend/blkback to make
a given block device accessible from another domain.  E.g. The
following exports /dev/hda3 read-write to the new domain, where
it will appear as /dev/sda1:

 disk = [ 'phy:hda3,sda1,w' ]

In the above example, the sda1 is sometimes referred to as the
target or destination device.


Ian


-------------------------------------------------------
SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285

Re: Xen unprivileged domain not booting

[Mark Williamson]

Hi Will,

The xm defaults file has a list of entries for each device the domain is going 
to use.  These are what we're referring to as the "exports" - in this case 
they are exported from dom0 using Xen device channels, rather than NFS.  They 
nevertheless have to be exported from dom0 because unprivileged domains can't 
access their own disks directly.

Each entry in the list has a field that is 'r' for read-only or 'w' (or, 
equivalently 'rw') to allow write access.  If you don't have 'w' or 'rw' set 
for the xda1 device, then it will not be able to write to that filesystem, 
causing short writes.

At this point, it's worth noting that you should never share a filesystem 
between two domains unless they both have it mounted read only.  So in this 
case, make sure you also have umounted the filesystem in dom0 before booting 
another domain from it.

HTH,
Mark



-------------------------------------------------------
SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285

Re: Xen unprivileged domain not booting

[Mark Williamson]

It's probably worth adding this terminology to the manual docs...  I'll put it 
on my TODO list ;-)



-------------------------------------------------------
SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media
100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33
Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift.
http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285