* Re: [Xen-users] XCP with 64-bit dom0? [not found] ` <BANLkTimw2818KuW9Ld5oym6skEdztfh1XA@mail.gmail.com> @ 2011-04-22 10:36 ` Muriel 2011-04-25 15:39 ` Kannan Vijayan 0 siblings, 1 reply; 3+ messages in thread From: Muriel @ 2011-04-22 10:36 UTC (permalink / raw) To: xen-devel; +Cc: kvijayan >>> Hi all, >>> >>> For the last couple weeks or so, I've been pounding away at building a >>> version of the XCP host tools to run on a 64-bit dom0 (not just a >>> 64-bit kernel, but a full 64-bit tool stack). I'm starting with the >>> basic build setup of Centos 5.5 with arch x86_64. Naturally, I'm >>> rebuilding a large number of RPM packages from source, and I'm running >>> into package hell as I try to track down proper 64-bit source versions >>> of packages, fix build and compile issues, and get up to the point of >>> being able to build the xapi-libs and xapi packages. >> >> I've never tried it. Is there a particular reason WHY you want to do >> it? I can't think of any benefit, and the downside is your dom0 will >> use more RAM, which means less RAM available for your domUs. >> >> -Dustin > I'm trying to integrate live-cloning (via the xen snowflock codebase) > into the XCP control path, which requires me to run a 64-bit PV > kernel, and also requires 64-bit dom0 tools to talk properly with Xen. > > Perhaps I should re-ask this question in Xen-devel. It may be a more > appropriate venue. > > -kannan Ok, i will reply directly to xen-devel to move the discussion. I'm very interested and i'm starting to recompile. I'm trying to integrate into an existing 64bit cluster a dom0 based on xpc. What packages are fundamental to have a stable environment? Thanks, Muriel ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [Xen-users] XCP with 64-bit dom0? 2011-04-22 10:36 ` [Xen-users] XCP with 64-bit dom0? Muriel @ 2011-04-25 15:39 ` Kannan Vijayan 2011-05-01 0:56 ` Todd Deshane 0 siblings, 1 reply; 3+ messages in thread From: Kannan Vijayan @ 2011-04-25 15:39 UTC (permalink / raw) To: Muriel; +Cc: xen-devel On Fri, Apr 22, 2011 at 6:36 AM, Muriel <mucawhite@gmail.com> wrote: >> I'm trying to integrate live-cloning (via the xen snowflock codebase) >> into the XCP control path, which requires me to run a 64-bit PV >> kernel, and also requires 64-bit dom0 tools to talk properly with Xen. >> >> Perhaps I should re-ask this question in Xen-devel. It may be a more >> appropriate venue. >> >> -kannan > > > Ok, i will reply directly to xen-devel to move the discussion. > I'm very interested and i'm starting to recompile. I'm trying to integrate > into an existing 64bit cluster a dom0 based on xpc. What packages are > fundamental to have a stable environment? > > Thanks, > Muriel > I've been working at it from the other end, with a goal of "shortest path to working". I'm coming in with relative unfamiliarity with the XCP architecture, and started working top-down. I can boot the XCP host with a 64-bit snowflock kernel. I subsequently ran into problems with a 32-bit blktap userspace toolset having problems talking to a 64-bit dom0 kernel, so I started rebuilding that and kept moving down as needed. I got the blktap issue resolved, but currently I'm running into an issue with xc_dom_boot_mem_init: /var/log/xensource.log:[20110418T16:56:55.946Z|debug|localhost|1003 unix-RPC||cli] Xapi_cli.exception_handler: Got exception INTERNAL_ERROR: [ XenguestHelper.Xc_dom_linux_build_failure(4, " xc_dom_boot_mem_init: can't allocate low memory for dom\\\"") ] Once again, I think this is due to a mismatch between 32-bit/64-bit addresses. The call is made through a tool called 'xenguest' which is built alongside xapi and is used by xapi. I had to go and dig pretty deep in the rebuild tree to generate the dependencies needed to build a 64-bit xapi and xenguest. So far, these are the high-level packages I've had to rebuild to target a 64-bit runtime (all as part of getting to a 64-bit build of xapi and xenguest): blktap-1.0.0-566.x86_64.rpm blktap-devel-1.0.0-566.x86_64.rpm ocaml-3.11.0-unknown.x86_64.rpm ocaml-camlp4-3.11.0-unknown.x86_64.rpm ocaml-findlib-1.1.2pl1-16.x86_64.rpm ocaml-findlib-devel-1.1.2pl1-16.x86_64.rpm ocaml-getopt-20040811-unknown.x86_64.rpm ocaml-getopt-devel-20040811-unknown.x86_64.rpm ocaml-type-conv-1.6.8-unknown.x86_64.rpm ocaml-xmlm-1.0.2-unknown.x86_64.rpm ocaml-xmlm-devel-1.0.2-unknown.x86_64.rpm omake-0.9.8.5-unknown.x86_64.rpm xapi-client-devel-0.2-unknown.x86_64.rpm xapi-core-0.2-unknown.x86_64.rpm xapi-datamodel-devel-0.2-unknown.x86_64.rpm xapi-docs-0.2-unknown.x86_64.rpm xapi-libs-devel-0-unknown.x86_64.rpm xapi-libs-fe-0-unknown.x86_64.rpm xapi-libs-utils-0-unknown.x86_64.rpm xapi-squeezed-0.2-unknown.x86_64.rpm xapi-tests-0.2-unknown.x86_64.rpm xapi-www-0.2-unknown.x86_64.rpm xapi-xe-0.2-unknown.x86_64.rpm xapi-xenops-0.2-unknown.x86_64.rpm xen-blktap-3.4.2-1.0.0.700.20051.x86_64.rpm xen-devel-3.4.2-1.0.0.700.20051.x86_64.rpm xen-firmware-3.4.2-1.0.0.700.20051.x86_64.rpm xen-hypervisor-3.4.2-1.0.0.700.20051.x86_64.rpm xen-tools-3.4.2-1.0.0.700.20051.x86_64.rpm Current problem: Installing these directly on a normal XCP deployment is infeasible. It wants to pull a TON of 64-bit support packages, and quickly descends into conflict hell. So now I'm looking for ways to build an XCP install image, and hoping I can replace all the base packages with 64-bit versions and go from there. I've looked around for instructions on building a full install image (found the instructions for building/modifying xapi.. those were very useful earlier in the process.. but not for this). If I can't rebuild an XCP install disk, the other tactic I have in mind is to edit an existing install ISO and switch out the existing packages for 64-bit surrogates. I haven't tried either of these approaches yet. Does anybody know of docs, or have thoughts, on either of these subjects? I'm picking up what I need to know as I go along, so my overall picture of the system is coalescing over time, but is almost definitely not complete at this time. Just a note: from what I _have_ seen of the system, it's very nicely designed. I like the xapi object model and schema design. Simple, but complete, clear, and understandable. A short analysis was enough to figure out how best to approach the integration of live-cloning semantics into XCP. Kudos on the good work. Cheers. -kannan ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Re: [Xen-users] XCP with 64-bit dom0? 2011-04-25 15:39 ` Kannan Vijayan @ 2011-05-01 0:56 ` Todd Deshane 0 siblings, 0 replies; 3+ messages in thread From: Todd Deshane @ 2011-05-01 0:56 UTC (permalink / raw) To: Kannan Vijayan; +Cc: xen-devel, Muriel, Jonathan Ludlam On Mon, Apr 25, 2011 at 11:39 AM, Kannan Vijayan <kvijayan@gridcentric.ca> wrote: > On Fri, Apr 22, 2011 at 6:36 AM, Muriel <mucawhite@gmail.com> wrote: >>> I'm trying to integrate live-cloning (via the xen snowflock codebase) >>> into the XCP control path, which requires me to run a 64-bit PV >>> kernel, and also requires 64-bit dom0 tools to talk properly with Xen. >>> Have you considered running the 64 bit code in a VM? The dom0 in XCP is 32 bit because of performance. It is recommended for performance reasons to use 64bit HVM and then use the PVonHVM drivers. http://wiki.xensource.com/xenwiki/XenLinuxPVonHVMdrivers >>> Perhaps I should re-ask this question in Xen-devel. It may be a more >>> appropriate venue. >>> >>> -kannan >> >> >> Ok, i will reply directly to xen-devel to move the discussion. >> I'm very interested and i'm starting to recompile. I'm trying to integrate >> into an existing 64bit cluster a dom0 based on xpc. What packages are >> fundamental to have a stable environment? >> >> Thanks, >> Muriel >> > > I've been working at it from the other end, with a goal of "shortest > path to working". I'm coming in with relative unfamiliarity with the > XCP architecture, and started working top-down. > > I can boot the XCP host with a 64-bit snowflock kernel. I > subsequently ran into problems with a 32-bit blktap userspace toolset > having problems talking to a 64-bit dom0 kernel, so I started > rebuilding that and kept moving down as needed. I got the blktap > issue resolved, but currently I'm running into an issue with > xc_dom_boot_mem_init: > > /var/log/xensource.log:[20110418T16:56:55.946Z|debug|localhost|1003 > unix-RPC||cli] Xapi_cli.exception_handler: Got exception > INTERNAL_ERROR: [ XenguestHelper.Xc_dom_linux_build_failure(4, " > xc_dom_boot_mem_init: can't allocate low memory for dom\\\"") ] > > Once again, I think this is due to a mismatch between 32-bit/64-bit > addresses. The call is made through a tool called 'xenguest' which is > built alongside xapi and is used by xapi. I had to go and dig pretty > deep in the rebuild tree to generate the dependencies needed to build > a 64-bit xapi and xenguest. > > So far, these are the high-level packages I've had to rebuild to > target a 64-bit runtime (all as part of getting to a 64-bit build of > xapi and xenguest): > > blktap-1.0.0-566.x86_64.rpm > blktap-devel-1.0.0-566.x86_64.rpm > ocaml-3.11.0-unknown.x86_64.rpm > ocaml-camlp4-3.11.0-unknown.x86_64.rpm > ocaml-findlib-1.1.2pl1-16.x86_64.rpm > ocaml-findlib-devel-1.1.2pl1-16.x86_64.rpm > ocaml-getopt-20040811-unknown.x86_64.rpm > ocaml-getopt-devel-20040811-unknown.x86_64.rpm > ocaml-type-conv-1.6.8-unknown.x86_64.rpm > ocaml-xmlm-1.0.2-unknown.x86_64.rpm > ocaml-xmlm-devel-1.0.2-unknown.x86_64.rpm > omake-0.9.8.5-unknown.x86_64.rpm > xapi-client-devel-0.2-unknown.x86_64.rpm > xapi-core-0.2-unknown.x86_64.rpm > xapi-datamodel-devel-0.2-unknown.x86_64.rpm > xapi-docs-0.2-unknown.x86_64.rpm > xapi-libs-devel-0-unknown.x86_64.rpm > xapi-libs-fe-0-unknown.x86_64.rpm > xapi-libs-utils-0-unknown.x86_64.rpm > xapi-squeezed-0.2-unknown.x86_64.rpm > xapi-tests-0.2-unknown.x86_64.rpm > xapi-www-0.2-unknown.x86_64.rpm > xapi-xe-0.2-unknown.x86_64.rpm > xapi-xenops-0.2-unknown.x86_64.rpm > xen-blktap-3.4.2-1.0.0.700.20051.x86_64.rpm > xen-devel-3.4.2-1.0.0.700.20051.x86_64.rpm > xen-firmware-3.4.2-1.0.0.700.20051.x86_64.rpm > xen-hypervisor-3.4.2-1.0.0.700.20051.x86_64.rpm > xen-tools-3.4.2-1.0.0.700.20051.x86_64.rpm > > > Current problem: Installing these directly on a normal XCP deployment > is infeasible. It wants to pull a TON of 64-bit support packages, and > quickly descends into conflict hell. > > So now I'm looking for ways to build an XCP install image, and hoping > I can replace all the base packages with 64-bit versions and go from > there. I've looked around for instructions on building a full install > image (found the instructions for building/modifying xapi.. those were > very useful earlier in the process.. but not for this). > > If I can't rebuild an XCP install disk, the other tactic I have in > mind is to edit an existing install ISO and switch out the existing > packages for 64-bit surrogates. > > I haven't tried either of these approaches yet. Does anybody know of > docs, or have thoughts, on either of these subjects? I'm picking up > what I need to know as I go along, so my overall picture of the system > is coalescing over time, but is almost definitely not complete at this > time. > > Just a note: from what I _have_ seen of the system, it's very nicely > designed. I like the xapi object model and schema design. Simple, > but complete, clear, and understandable. A short analysis was enough > to figure out how best to approach the integration of live-cloning > semantics into XCP. Kudos on the good work. > > Cheers. > -kannan > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > -- Todd Deshane http://www.linkedin.com/in/deshantm http://www.xen.org/products/cloudxen.html ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2011-05-01 0:56 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <BANLkTim=1Oaw=CEjQ3bvx09N0tDpna94mA@mail.gmail.com>
[not found] ` <BANLkTikn9vS+utdL-HcMtC2gYmh5KJ7Egg@mail.gmail.com>
[not found] ` <BANLkTimw2818KuW9Ld5oym6skEdztfh1XA@mail.gmail.com>
2011-04-22 10:36 ` [Xen-users] XCP with 64-bit dom0? Muriel
2011-04-25 15:39 ` Kannan Vijayan
2011-05-01 0:56 ` Todd Deshane
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).