XCP: Xen tools problem with RedHat VMs?

XCP: Xen tools problem with RedHat VMs?

[dwight at supercomputer.org]

I came across an odd problem with adding the packages from the 
xs-tools.iso today. After mounting the ISO in a fresh CentOS 5.4 
VM, and doing the .../Linux/install.sh, I ended up with this in 
the VM's grub.conf:

----- Start of the new grub.conf entry ----------
title CentOS (2.6.18-164.10.1.el5.xs0.1.1.64xen)
        root (hd0,0)
        kernel /xen.gz-2.6.18-164.10.1.el5.xs0.1.1.64
        module /vmlinuz-2.6.18-164.10.1.el5.xs0.1.1.64xen ro
root=/dev/VolGroup00/LogVol00 rhgb quiet
        module /initrd-2.6.18-164.10.1.el5.xs0.1.1.64xen.img
----- End of the new grub.conf entry ----------

Now, having the Xen hypervisor running in a DomU would be exciting, 
but alas, xen.gz doesn't appear to have been installed.

So, I have to conclude that either:
    1. I'm missing this new DomU hypervisor.
    2. The documentation is wrong.
    3. The xs-tools ISO has a bug in it for the 0.1.1 XCP 
        installation with RedHat distros.

Truth to tell, I was given this VM to examine the problem, and 
didn't have time today to doublecheck the xs-tools.iso. I will do 
so tomorrow, but I wanted to get some feedback ASAP.

So I'm wondering if this is indeed a bug somewhere? And has anyone
added these tools to a CentOS/RedHat VM running on XCP 0.1.1?

If it is a bug, it would be really nice to get this fixed before 
the next release of XCP.

Thanks,

   -dwight-

Re: XCP: Xen tools problem with RedHat VMs?

[Ian Campbell]

On Thu, 2010-06-03 at 06:31 +0100, dwight at supercomputer.org wrote:
> I came across an odd problem with adding the packages from the 
> xs-tools.iso today. After mounting the ISO in a fresh CentOS 5.4 
> VM, and doing the .../Linux/install.sh, I ended up with this in 
> the VM's grub.conf:
> 
> ----- Start of the new grub.conf entry ----------
> title CentOS (2.6.18-164.10.1.el5.xs0.1.1.64xen)
>         root (hd0,0)
>         kernel /xen.gz-2.6.18-164.10.1.el5.xs0.1.1.64
>         module /vmlinuz-2.6.18-164.10.1.el5.xs0.1.1.64xen ro
> root=/dev/VolGroup00/LogVol00 rhgb quiet
>         module /initrd-2.6.18-164.10.1.el5.xs0.1.1.64xen.img
> ----- End of the new grub.conf entry ----------

Is it causing an actual issue? Are the other existing stanzas similar?

This stanza is created by tools which are part of RHEL (/sbin/grubby or
similar) which are called by the kernel RPM's %post script. XCP does not
patch this part of the kernel so the behaviour here should be as for a
CentOS supplied RPM.

My understanding was that pygrub was wise enough to spot entries of this
form and drop the xen when booting in a domU.

Long term we are hoping to get rid of the majority of the kernels
supplied by the xs-tools.iso. The distro provided kernels are perfectly
fine these days.

For example in the next stable release we will drop the RHEL5x kernel
because RHEL 5.4 is in good enough shape (we would still recommend
running the 5.4 kernel even on earlier 5.x).

>From tracking the issues in RH bugzilla I think that when RHEL 4.6 comes
out it will also be good enough that we can drop the RHEL4x kernel from
XCP as well. Even the issues in the existing RHEL 4.8 kernels are not
that critical, they are mostly rare failures to boot which can be worked
around by rebooting.

> 
> Now, having the Xen hypervisor running in a DomU would be exciting, 
> but alas, xen.gz doesn't appear to have been installed.
> 
> So, I have to conclude that either:
>     1. I'm missing this new DomU hypervisor.
>     2. The documentation is wrong.
>     3. The xs-tools ISO has a bug in it for the 0.1.1 XCP 
>         installation with RedHat distros.
> 
> Truth to tell, I was given this VM to examine the problem, and 
> didn't have time today to doublecheck the xs-tools.iso. I will do 
> so tomorrow, but I wanted to get some feedback ASAP.
> 
> So I'm wondering if this is indeed a bug somewhere? And has anyone
> added these tools to a CentOS/RedHat VM running on XCP 0.1.1?
> 
> If it is a bug, it would be really nice to get this fixed before 
> the next release of XCP.
> 
> Thanks,
> 
>    -dwight-
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: XCP: Xen tools problem with RedHat VMs?

[dwight at supercomputer.org]

On Thursday 03 June 2010 01:25:15 am Ian Campbell wrote:
> On Thu, 2010-06-03 at 06:31 +0100, dwight at supercomputer.org 
wrote:
> > I came across an odd problem with adding the packages from the
> > xs-tools.iso today....
>
> Is it causing an actual issue? Are the other existing stanzas
> similar?...

Thank you, Ian. Your reply was helpful, once again.

By itself, no, it's not causing a technical issue. But seeing
something inappropriate which is installed immediately makes me
wonder what else is wrong with the installation? Hence, my
original questions.

The problem which triggered this inquiry was that after an
apparently successful installation, the Linux Agent did not
appear to be working as expected.

I debugged this down further, starting from the installation
script and RPMs. It turns out that I ran into the tail end of
what can be best described as a cascade of failures leading
up to this end result.

One might recall this particularly useful post from January:

http://vburke.wordpress.com/2010/01/28/upgrading-xen-cloud-platform-bugs-and-fixes/

It describes that the el5.4-initrd-additions.cpio was left out
of the 0.1.1 release. It turns out that the following phrase
is incorrect:

"a working CentOS/RHEL 5.4 VM cannot be created, despite there
being a template for it."

This leaves one with the impression that one won't see a
CentOS 5.4 VM created from the template, unless this cpio
is installed.

It turns out that a working 5.4 VM can indeed be created from
the template quite successfully. However, this is an HVM. It is
correct to say that a working PVM cannot be created.

Consequently, when I was brought in to look at this installation
problem with the Tools, I was, alas, under the overly optimistic
assumption that it was on a PVM, not an HVM.

And, since it was an HVM, there was no /proc/xen. Consequently,
the installation succeeded (at first glance). However, needless
to say, the Linux Agent had difficulties working.

I have since corrected the problem, but have some additional
notes which may be of interest.

First, I ran into the reported experience of a failure of the
PVM to boot up during a new installation. Multiple times, and it
was the norm, not the exception. Inspecting the Apache logs
(for this http install) revealed that no attempts were made to
access any files. And yes, the network was working fine, and
the installation paramaters were quite correct.

I have to wonder if this is one of those PXE-boot timing issues
that one sometimes runs into with DHCP?

Finally, after many attempts, the initial booting for the
installation was successful. No parameters were changed,
which leads me to speculate about timing issues.

Secondly, it would be helpful to have some sort of indication
during the installation that installing the Tools on an HVM
is probably not optimal. Note that OpenXenCenter could
use such an indication too.

Please consider a very strong request to have CentOS 5.5
PVMs supported in the next release of XCP. We've been bringing
up many hosts while building our Cloud, and now have to go
back and make certain that all of our machines have this
cpio file for C/R 5.4. It would be helpful to minimize this
in the future.

Finally, my apologies for just talking about this and not
supplying any proposed patches to fix parts of these problems,
but we haven't had time to get all of the source code builds
in place; just those that have been needed, as needed.

If you need someone to alpha-test builds for all of the source,
as this mechanism develops, I would be interested in helping.

Thanks again,

   -dwight-