happy happy happy

happy happy happy

[Eric S. Johansson]

so very happy.  I got xen (mostly) working on gentoo with udev, raid, 
and lvm2 under the vm storage.  here are the hints (read scar tissue) I 
have acquired in the process.

All documentation has been written by the Brothers Grimm including this.

As long as the raid array and lvm2 volumes are not on the standard /, 
/usr, ... directories, it doesn't matter whether the modules are 
compiled in or loadable.  However, if you need them at boot time for any 
of the core directories, play it safe and compile them in.

build and verify your raid array first (mdadm and /etc/mdadm.conf is 
your friend).  Make sure it is discovered appropriately on reboot

create your lvm2 partitions and volume groups per instructions.

install udev

under gentoo, /etc/init.d/createfs will not detect a logical volumes and 
raid partitions properly.  the startup script detects the logical volume 
first and raid second.  the only solution is to hack on the script to 
reverse this process.

reboot and (in theory) everything should be OK.  obviously, verified to 
make sure the raid was still detected as well as the logical volumes 
(/dev/<vol group>/<partitions>)

how you use this under xen is only for consenting adults.

anyway, the hopefully minor errors that I'm trying to figure out are below:

Gentoo Linux; http://www.gentoo.org/
  Copyright 2001-2004 Gentoo Technologies, Inc.; Distributed under the GPL

  * Mounting proc at /proc...  [ ok ]
  * Mounting sysfs at /sys...  [ ok ]
rm: cannot remove `/dev/.devfsd': Read-only file system
  * Mounting devpts at /dev/pts...  [ ok ]
  * Activating (possible) swap...  [ ok ]
  * Remounting root filesystem read-only (if necessary)...  [ ok ]
  * Checking root filesystem...Reiserfs super block in block 16 on 0x801 
of format 3.6 with standard journal
Blocks (total/free): 1048576/575357 by 4096 bytes
Filesystem is clean
Filesystem seems mounted read-only. Skipping journal replay.
Checking internal tree..finished
   [ ok ]
  * Remounting root filesystem read/write...  [ ok ]
  * Setting hostname to harvee...  [ ok ]
  * Calculating module dependencies...  [ ok ]
  * Checking all filesystems...  [ ok ]
  * Mounting local filesystems...  [ ok ]
  * Activating (possibly) more swap...  [ ok ]
  * Caching service dependencies... * Setting system clock to hardware 
clock [UTC]... * Failed to set system clock to hardware clock
   [ !! ]

  * ERROR:  Problem starting needed services.
  *         "bootmisc" was not started.
  * Setting user font...Couldnt open /dev/vc/1
Couldnt open /dev/vc/2
Couldnt open /dev/vc/3
Couldnt open /dev/vc/4
Couldnt open /dev/vc/5
Couldnt open /dev/vc/6
Couldnt open /dev/vc/7
Couldnt open /dev/vc/8
Couldnt open /dev/vc/9
Couldnt open /dev/vc/10
Couldnt open /dev/vc/11
  * Failed to set user font
   [ !! ]

  * Loading key mappings...Couldnt get a file descriptor referring to 
the console
  * Error loading key mappings
   [ !! ]

  * Bringing lo up...  [ ok ]
  * Initializing random number generator...  [ ok ]
INIT: Entering runlevel: 4
   [ ok ]
  * Starting syslog-ng...  [ ok ]
  * Bringing eth0 up (192.168.25.100)...  [ ok ]
  * Mounting network filesystems...  [ ok ]
  * Starting sshd...  [ ok ]
  * Starting vixie-cron...  [ ok ]
  * Starting local...  [ ok ]


This is harvee.unknown_domain (Linux i686 2.6.10-xenU) 22:47:41

harvee login: root
Password:
Last login: Tue Feb 22 02:37:58 from 192.168.0.34


-- 
http://www.wired.com/wired/archive/13.03/view.html?pg=5

The result of the duopoly that currently defines "competition" is that
prices and service suck. We're the world's leader in Internet
technology - except that we're not.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: happy happy happy

[Jon Mason]

On Wed, Feb 23, 2005 at 06:32:44PM -0500, Eric S. Johansson wrote:
[...]
> anyway, the hopefully minor errors that I'm trying to figure out are below:
> 
> Gentoo Linux; http://www.gentoo.org/
>  Copyright 2001-2004 Gentoo Technologies, Inc.; Distributed under the GPL
> 
>  * Mounting proc at /proc...  [ ok ]
>  * Mounting sysfs at /sys...  [ ok ]
> rm: cannot remove `/dev/.devfsd': Read-only file system
>  * Mounting devpts at /dev/pts...  [ ok ]
>  * Activating (possible) swap...  [ ok ]
>  * Remounting root filesystem read-only (if necessary)...  [ ok ]
>  * Checking root filesystem...Reiserfs super block in block 16 on 0x801 
> of format 3.6 with standard journal
> Blocks (total/free): 1048576/575357 by 4096 bytes
> Filesystem is clean
> Filesystem seems mounted read-only. Skipping journal replay.
> Checking internal tree..finished
>   [ ok ]
>  * Remounting root filesystem read/write...  [ ok ]
>  * Setting hostname to harvee...  [ ok ]
>  * Calculating module dependencies...  [ ok ]
>  * Checking all filesystems...  [ ok ]
>  * Mounting local filesystems...  [ ok ]
>  * Activating (possibly) more swap...  [ ok ]
>  * Caching service dependencies... * Setting system clock to hardware 
> clock [UTC]... * Failed to set system clock to hardware clock
>   [ !! ]
> 
>  * ERROR:  Problem starting needed services.
>  *         "bootmisc" was not started.
>  * Setting user font...Couldnt open /dev/vc/1
> Couldnt open /dev/vc/2
> Couldnt open /dev/vc/3
> Couldnt open /dev/vc/4
> Couldnt open /dev/vc/5
> Couldnt open /dev/vc/6
> Couldnt open /dev/vc/7
> Couldnt open /dev/vc/8
> Couldnt open /dev/vc/9
> Couldnt open /dev/vc/10
> Couldnt open /dev/vc/11
>  * Failed to set user font
>   [ !! ]
> 
>  * Loading key mappings...Couldnt get a file descriptor referring to 
> the console
>  * Error loading key mappings
>   [ !! ]
> 
>  * Bringing lo up...  [ ok ]
>  * Initializing random number generator...  [ ok ]
> INIT: Entering runlevel: 4
>   [ ok ]
>  * Starting syslog-ng...  [ ok ]
>  * Bringing eth0 up (192.168.25.100)...  [ ok ]
>  * Mounting network filesystems...  [ ok ]
>  * Starting sshd...  [ ok ]
>  * Starting vixie-cron...  [ ok ]
>  * Starting local...  [ ok ]
> 
> 
> This is harvee.unknown_domain (Linux i686 2.6.10-xenU) 22:47:41
> 
> harvee login: root
> Password:
> Last login: Tue Feb 22 02:37:58 from 192.168.0.34

I see the same errors on my gentoo installation running DevFS.  

-- 
Jon Mason
jdmason@us.ibm.com


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: happy happy happy

[Eric S. Johansson]

Jon Mason wrote:

> I see the same errors on my gentoo installation running DevFS.  

it shows errors I'm not comfortable with.  For example, it says is 
unable to set the system clock yet the time is set correctly.  This 
messages from the clock init script.

harvee-xeno init.d # hwclock --show
hwclock is unable to get I/O port access:  the iopl(3) call failed.

udev quirk or something more sinister?  in either case, turning off the 
clock init script should help except for the fact that you can't turn 
off the clock script without doing more invasive surgery.

the next set of errors come from the consolefont script. turning it off 
makes those errors go away.  I suspect the key mappings error is 
similarly solvable but I just haven't had the time to dig back into it.

one other thing I noticed was that when I try to use Reiser FS, I kept 
getting exceptions from the journaling code.  So I decided to take the 
cowards way out and switch to ext3.  now the exceptions may have been a 
fault of my own doing because there is a high probability I did not have 
my domU filesystem fully set up with udev.  In any case, onward and upward.

emerge --sync works so I am hopeful

---eric





-- 
http://www.wired.com/wired/archive/13.03/view.html?pg=5

The result of the duopoly that currently defines "competition" is that
prices and service suck. We're the world's leader in Internet
technology - except that we're not.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: happy happy happy

[Mark Williamson]

> it shows errors I'm not comfortable with.  For example, it says is
> unable to set the system clock yet the time is set correctly.  This
> messages from the clock init script.
>
> harvee-xeno init.d # hwclock --show
> hwclock is unable to get I/O port access:  the iopl(3) call failed.

This is a domU, right?  That's normal - you don't want them to be able to set 
the hardware clock.  By default, they'll all use the time from Xen, which can 
be maintained by running ntp in dom0.

> the next set of errors come from the consolefont script. turning it off
> makes those errors go away.  I suspect the key mappings error is
> similarly solvable but I just haven't had the time to dig back into it.

I seem to remember the key mappings thing is also normal for a domU and you 
can just turn the script off.

> one other thing I noticed was that when I try to use Reiser FS, I kept
> getting exceptions from the journaling code.  So I decided to take the
> cowards way out and switch to ext3.

That's perculiar - you shouldn't have a problem here.  Other people are 
running ReiserFS OK.

> now the exceptions may have been a 
> fault of my own doing because there is a high probability I did not have
> my domU filesystem fully set up with udev. 

Shouldn't be udev either - it'd be interesting to see if you can mount 
ReiserFS filesystems when everything's up and running.

Cheers,
Mark


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: happy happy happy

[Eric S. Johansson]

Mark Williamson wrote:
>>harvee-xeno init.d # hwclock --show
>>hwclock is unable to get I/O port access:  the iopl(3) call failed.
> 
> 
> This is a domU, right? 

correct

> That's normal - you don't want them to be able to set 
> the hardware clock.  By default, they'll all use the time from Xen, which can 
> be maintained by running ntp in dom0.

OK.  That makes sense.  Just to make sure I understand, set up ntp in 
dom0 and automatically all of the domU machines will get the correct time?

sounds also like it's time to start modifying the clock script to detect 
xen environments and act appropriately.  They are to have such a special 
case handler for uml so it should be a big deal to do the same here.

> I seem to remember the key mappings thing is also normal for a domU and you 
> can just turn the script off.

I will verify and report back.


>>one other thing I noticed was that when I try to use Reiser FS, I kept
>>getting exceptions from the journaling code.  So I decided to take the
>>cowards way out and switch to ext3.
> 
> 
> That's perculiar - you shouldn't have a problem here.  Other people are 
> running ReiserFS OK.

I know.  I thought I was safe but apparently my gift for making things 
fail reared its ugly head again.  it's great when you're trying to 
exercise something but it really sucks when you are just trying to get a 
job done.

>>now the exceptions may have been a 
>>fault of my own doing because there is a high probability I did not have
>>my domU filesystem fully set up with udev. 
> 
> 
> Shouldn't be udev either - it'd be interesting to see if you can mount 
> ReiserFS filesystems when everything's up and running.

I can mount it okay but when I stress it with heavy activity such as 
emerge --sync, it fails.

maybe I should send a copy of sda.. it's only 4 or 5 GB.  ;-)

---eric

-- 
http://www.wired.com/wired/archive/13.03/view.html?pg=5

The result of the duopoly that currently defines "competition" is that
prices and service suck. We're the world's leader in Internet
technology - except that we're not.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: happy happy happy

[Mark Williamson]

> OK.  That makes sense.  Just to make sure I understand, set up ntp in
> dom0 and automatically all of the domU machines will get the correct time?

Yes, unless they decide to "opt out" and keep their own time.  By default they 
should synchronise automagically, I think.

> > That's perculiar - you shouldn't have a problem here.  Other people are
> > running ReiserFS OK.
>
> I know.  I thought I was safe but apparently my gift for making things
> fail reared its ugly head again.  it's great when you're trying to
> exercise something but it really sucks when you are just trying to get a
> job done.

Can you verify the same problem doesn't occur with the same filesystem under 
dom0?  Under vanilla Linux?  It seems possible some kind of corruption could 
have occurred, as Ian suggested.

> maybe I should send a copy of sda.. it's only 4 or 5 GB.  ;-)

Heh, the record for the Worlds Largest Email goes go..... ;-)

Cheers,
Mark


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: happy happy happy

[Jon Mason]

On Thu, Feb 24, 2005 at 02:39:16PM -0500, Eric S. Johansson wrote:
> Jon Mason wrote:
> 
> >I see the same errors on my gentoo installation running DevFS.  
> 
> it shows errors I'm not comfortable with.  For example, it says is 
> unable to set the system clock yet the time is set correctly.  This 
> messages from the clock init script.
> 
> harvee-xeno init.d # hwclock --show
> hwclock is unable to get I/O port access:  the iopl(3) call failed.
> 
> udev quirk or something more sinister?  in either case, turning off the 
> clock init script should help except for the fact that you can't turn 
> off the clock script without doing more invasive surgery.

Not runnig udev, so it is more generic.  It doesn't seem to affect
anything,

> the next set of errors come from the consolefont script. turning it off 
> makes those errors go away.  I suspect the key mappings error is 
> similarly solvable but I just haven't had the time to dig back into it.
> 
> one other thing I noticed was that when I try to use Reiser FS, I kept 
> getting exceptions from the journaling code.  So I decided to take the 
> cowards way out and switch to ext3.  now the exceptions may have been a 
> fault of my own doing because there is a high probability I did not have 
> my domU filesystem fully set up with udev.  In any case, onward and upward.

I am running without any problems on reiserfs.  I am happy to help you
and provide you with my current setup files, if you like.

-- 
Jon Mason
jdmason@us.ibm.com


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: happy happy happy

[Robin Green]

On Thu, 24 Feb 2005, Eric S. Johansson wrote:
> one other thing I noticed was that when I try to use Reiser FS, I kept 
> getting exceptions from the journaling code.  So I decided to take the 
> cowards way out and switch to ext3.  now the exceptions may have been a fault 
> of my own doing because there is a high probability I did not have my domU 
> filesystem fully set up with udev.  In any case, onward and upward.

I am probably barking up the wrong tree, but did you use a file-backed 
device (i.e. file: instead of phys: in the xen conf file) for any of your 
reiserfs partitions? I have heard from multiple sources that journalled 
filesystems (and that includes ext3) do not play well with file-backed 
mounts (although with User Mode Linux it is said to work OK if the host 
writes to the file using O_SYNC).

-- 
Robin


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: happy happy happy

[Eric S. Johansson]

Robin Green wrote:
> I am probably barking up the wrong tree, but did you use a file-backed 
> device (i.e. file: instead of phys: in the xen conf file) for any of 
> your reiserfs partitions? I have heard from multiple sources that 
> journalled filesystems (and that includes ext3) do not play well with 
> file-backed mounts (although with User Mode Linux it is said to work OK 
> if the host writes to the file using O_SYNC).
> 

nope.  raid1 -> lvm2 managed partitions -> filesystem

reasonably ordinary, reasonably vanilla stuff that just doesn't want to 
work right (auto detection of raid failed, LVM and raid array creation 
in wrong order)

right now, if I were to bark up the wrong tree, I would get a mouthful 
of acorns...

---eric

-- 
http://www.wired.com/wired/archive/13.03/view.html?pg=5

The result of the duopoly that currently defines "competition" is that
prices and service suck. We're the world's leader in Internet
technology - except that we're not.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

RE: happy happy happy

[Ian Pratt]

> it shows errors I'm not comfortable with.  For example, it says is 
> unable to set the system clock yet the time is set correctly.  This 
> messages from the clock init script.
> 
> harvee-xeno init.d # hwclock --show
> hwclock is unable to get I/O port access:  the iopl(3) call failed.

That's to be expected, and harmless -- just disable it.

> the next set of errors come from the consolefont script. 
> turning it off 
> makes those errors go away.  I suspect the key mappings error is 
> similarly solvable but I just haven't had the time to dig 
> back into it.

Likewise.

> one other thing I noticed was that when I try to use Reiser 
> FS, I kept 
> getting exceptions from the journaling code.  

Xen/Linux should work fine with reiserfs. I wander if you accidently
mounted it from two domains simultaneously at some point and introduced
corruption?

Ian


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id\x14396&op=click