Re: [Xen-users] XCP with 64-bit dom0?

[Kannan Vijayan <kvijayan@gridcentric.ca>]

On Fri, Apr 22, 2011 at 6:36 AM, Muriel <mucawhite@gmail.com> wrote:
>> I'm trying to integrate live-cloning (via the xen snowflock codebase)
>> into the XCP control path, which requires me to run a 64-bit PV
>> kernel, and also requires 64-bit dom0 tools to talk properly with Xen.
>>
>> Perhaps I should re-ask this question in Xen-devel.  It may be a more
>> appropriate venue.
>>
>> -kannan
>
>
> Ok, i will reply directly to xen-devel to move the discussion.
> I'm very interested and i'm starting to recompile. I'm trying to integrate
> into an existing 64bit cluster a dom0 based on xpc. What packages are
> fundamental to have a stable environment?
>
> Thanks,
> Muriel
>

I've been working at it from the other end, with a goal of "shortest
path to working".  I'm coming in with relative unfamiliarity with the
XCP architecture, and started working top-down.

I can boot the XCP host with a 64-bit snowflock kernel.  I
subsequently ran into problems with a 32-bit blktap userspace toolset
having problems talking to a 64-bit dom0 kernel, so I started
rebuilding that and kept moving down as needed.  I got the blktap
issue resolved, but currently I'm running into an issue with
xc_dom_boot_mem_init:

/var/log/xensource.log:[20110418T16:56:55.946Z|debug|localhost|1003
unix-RPC||cli] Xapi_cli.exception_handler: Got exception
INTERNAL_ERROR: [ XenguestHelper.Xc_dom_linux_build_failure(4, "
xc_dom_boot_mem_init: can't allocate low memory for dom\\\"") ]

Once again, I think this is due to a mismatch between 32-bit/64-bit
addresses.  The call is made through a tool called 'xenguest' which is
built alongside xapi and is used by xapi.  I had to go and dig pretty
deep in the rebuild tree to generate the dependencies needed to build
a 64-bit xapi and xenguest.

So far, these are the high-level packages I've had to rebuild to
target a 64-bit runtime (all as part of getting to a 64-bit build of
xapi and xenguest):

blktap-1.0.0-566.x86_64.rpm
blktap-devel-1.0.0-566.x86_64.rpm
ocaml-3.11.0-unknown.x86_64.rpm
ocaml-camlp4-3.11.0-unknown.x86_64.rpm
ocaml-findlib-1.1.2pl1-16.x86_64.rpm
ocaml-findlib-devel-1.1.2pl1-16.x86_64.rpm
ocaml-getopt-20040811-unknown.x86_64.rpm
ocaml-getopt-devel-20040811-unknown.x86_64.rpm
ocaml-type-conv-1.6.8-unknown.x86_64.rpm
ocaml-xmlm-1.0.2-unknown.x86_64.rpm
ocaml-xmlm-devel-1.0.2-unknown.x86_64.rpm
omake-0.9.8.5-unknown.x86_64.rpm
xapi-client-devel-0.2-unknown.x86_64.rpm
xapi-core-0.2-unknown.x86_64.rpm
xapi-datamodel-devel-0.2-unknown.x86_64.rpm
xapi-docs-0.2-unknown.x86_64.rpm
xapi-libs-devel-0-unknown.x86_64.rpm
xapi-libs-fe-0-unknown.x86_64.rpm
xapi-libs-utils-0-unknown.x86_64.rpm
xapi-squeezed-0.2-unknown.x86_64.rpm
xapi-tests-0.2-unknown.x86_64.rpm
xapi-www-0.2-unknown.x86_64.rpm
xapi-xe-0.2-unknown.x86_64.rpm
xapi-xenops-0.2-unknown.x86_64.rpm
xen-blktap-3.4.2-1.0.0.700.20051.x86_64.rpm
xen-devel-3.4.2-1.0.0.700.20051.x86_64.rpm
xen-firmware-3.4.2-1.0.0.700.20051.x86_64.rpm
xen-hypervisor-3.4.2-1.0.0.700.20051.x86_64.rpm
xen-tools-3.4.2-1.0.0.700.20051.x86_64.rpm


Current problem: Installing these directly on a normal XCP deployment
is infeasible.  It wants to pull a TON of 64-bit support packages, and
quickly descends into conflict hell.

So now I'm looking for ways to build an XCP install image, and hoping
I can replace all the base packages with 64-bit versions and go from
there.  I've looked around for instructions on building a full install
image (found the instructions for building/modifying xapi.. those were
very useful earlier in the process.. but not for this).

If I can't rebuild an XCP install disk, the other tactic I have in
mind is to edit an existing install ISO and switch out the existing
packages for 64-bit surrogates.

I haven't tried either of these approaches yet.  Does anybody know of
docs, or have thoughts, on either of these subjects?  I'm picking up
what I need to know as I go along, so my overall picture of the system
is coalescing over time, but is almost definitely not complete at this
time.

Just a note: from what I _have_ seen of the system, it's very nicely
designed.  I like the xapi object model and schema design.  Simple,
but complete, clear, and understandable.  A short analysis was enough
to figure out how best to approach the integration of live-cloning
semantics into XCP.  Kudos on the good work.

Cheers.
-kannan