New domain builder in xen-unstable

New domain builder in xen-unstable

[Keir Fraser]

Hi,

This is a heads up, mainly to the PowerPC and IA64 folks, that Gerd's new
domain builder (including new Elf parser) is now checked into xen-unstable.
This has disabled the old builder and elf-parsing code which will shortly be
removed entirely.

So far the new libelf has only been run-tested on x86, so...

For IA64: most of the code is in place, but the new code needs some
run-testing and possibly some small fixups to get it working.

For PPC: looks like you have your own domain builder, which you may or may
not decide to merge with the main code (it would be nice to!). Either way,
you *do* use the common elf parser, and so your domain builder code needs to
be changed to talk to Gerd's libelf.

 -- Keir

Re: New domain builder in xen-unstable

[Hollis Blanchard]

On Fri, 2007-01-26 at 13:43 +0000, Keir Fraser wrote:
> Hi,
> 
> This is a heads up, mainly to the PowerPC and IA64 folks, that Gerd's new
> domain builder (including new Elf parser) is now checked into xen-unstable.
> This has disabled the old builder and elf-parsing code which will shortly be
> removed entirely.
> 
> So far the new libelf has only been run-tested on x86, so...
> 
> For IA64: most of the code is in place, but the new code needs some
> run-testing and possibly some small fixups to get it working.
> 
> For PPC: looks like you have your own domain builder, which you may or may
> not decide to merge with the main code (it would be nice to!).

Not sure what you mean by "our own domain builder"; we've been
implementing xc_linux_build() for quite a while now (and thus integrated
with xend).

> Either way,
> you *do* use the common elf parser, and so your domain builder code needs to
> be changed to talk to Gerd's libelf.

Gerd, what do we need to do to replace probe_elf()/load_funcs ?

-- 
Hollis Blanchard
IBM Linux Technology Center

Re: New domain builder in xen-unstable

[Hollis Blanchard]

On Fri, 2007-01-26 at 10:58 -0600, Hollis Blanchard wrote:
> On Fri, 2007-01-26 at 13:43 +0000, Keir Fraser wrote:
> > Hi,
> > 
> > This is a heads up, mainly to the PowerPC and IA64 folks, that Gerd's new
> > domain builder (including new Elf parser) is now checked into xen-unstable.
> > This has disabled the old builder and elf-parsing code which will shortly be
> > removed entirely.
> > 
> > So far the new libelf has only been run-tested on x86, so...
> > 
> > For IA64: most of the code is in place, but the new code needs some
> > run-testing and possibly some small fixups to get it working.
> > 
> > For PPC: looks like you have your own domain builder, which you may or may
> > not decide to merge with the main code (it would be nice to!).
> 
> Not sure what you mean by "our own domain builder"; we've been
> implementing xc_linux_build() for quite a while now (and thus integrated
> with xend).

Sorry, I misunderstood. I was seeing all the libelf churn, and I missed
the fact that xc_linux_build() is going away entirely (?).

I'm still not sure what you mean by "our own domain builder" -- there is
simply no way we're going to add more ifdefs to
tools/libxc/xc_linux_build.c (or equivalent)...

-- 
Hollis Blanchard
IBM Linux Technology Center

Re: New domain builder in xen-unstable

[Keir Fraser]

On 26/1/07 5:04 pm, "Hollis Blanchard" <hollisb@us.ibm.com> wrote:

>> Not sure what you mean by "our own domain builder"; we've been
>> implementing xc_linux_build() for quite a while now (and thus integrated
>> with xend).
> 
> Sorry, I misunderstood. I was seeing all the libelf churn, and I missed
> the fact that xc_linux_build() is going away entirely (?).
> 
> I'm still not sure what you mean by "our own domain builder" -- there is
> simply no way we're going to add more ifdefs to
> tools/libxc/xc_linux_build.c (or equivalent)...

xc_linux_build() is still provided in the 'new world' and is still used by
xend. For ia64/x86 it is now a thin wrapper around Gerd's new
domain-building infrastructure. For ppc you can continue to have your very
own version under libxc/powerpc if you want to, and not use Gerd's new
domain-building infrastructure. But the elf code that you rely on is going
away, so you need to move to using libelf, which I'm sure Gerd can give you
some pointers about.

 -- Keir

Re: New domain builder in xen-unstable

[Gerd Hoffmann]

Keir Fraser wrote:
> On 26/1/07 5:04 pm, "Hollis Blanchard" <hollisb@us.ibm.com> wrote:
> 
>>> Not sure what you mean by "our own domain builder"; we've been
>>> implementing xc_linux_build() for quite a while now (and thus integrated
>>> with xend).
>> Sorry, I misunderstood. I was seeing all the libelf churn, and I missed
>> the fact that xc_linux_build() is going away entirely (?).
>>
>> I'm still not sure what you mean by "our own domain builder" -- there is
>> simply no way we're going to add more ifdefs to
>> tools/libxc/xc_linux_build.c (or equivalent)...
> 
> xc_linux_build() is still provided in the 'new world' and is still used by
> xend. For ia64/x86 it is now a thin wrapper around Gerd's new
> domain-building infrastructure. For ppc you can continue to have your very
> own version under libxc/powerpc if you want to, and not use Gerd's new
> domain-building infrastructure. But the elf code that you rely on is going
> away, so you need to move to using libelf, which I'm sure Gerd can give you
> some pointers about.

libelf is used both within the xen kernel (dom0 builder) and the tools.

Have a look at changeset 13618, this does the dom0 builder port for x86
and ia64 (build-tested only), something simliar is needed for the ppc
dom0 builder.

For the tools:  You can either switch over the existing ppc builder in
libxc/powerpc/ to use libelf, which likely is simliar to the dom0
builder conversion to libelf.

The other option is to use the new builder code in libxc/xc_dom_*.c.
Some code is already there.  The ia64 bits should at least compile, with
luck even work out-of-the-box.  The ppc bits I didn't even compile.  The
places with architecture-specific code are:

xc_dom_{ia64,powerpc}.c
	setup arch-specific structs (start_info, vcpu_context).

xc_dom_boot.c
	invoke the hypercalls needed to boot the domain.  There are two
	functions for architecture-specific code, ppc must hook in the
	devtree setup there (if you decide to use the new builder).

The domain builder core code should need no changes in theory ;)

Feel free to ask if there are any questions.

HTH & cheers,
  Gerd

-- 
Gerd Hoffmann <kraxel@suse.de>

Re: Re: New domain builder in xen-unstable

[Hollis Blanchard]

On Fri, 2007-01-26 at 17:08 +0000, Keir Fraser wrote:
> xc_linux_build() is still provided in the 'new world' and is still
> used by xend. For ia64/x86 it is now a thin wrapper around Gerd's new
> domain-building infrastructure. For ppc you can continue to have your
> very own version under libxc/powerpc if you want to, and not use
> Gerd's new domain-building infrastructure. But the elf code that you
> rely on is going away, so you need to move to using libelf, which I'm
> sure Gerd can give you some pointers about. 

Hi Keir, it turns out I was confused about libxc, and we were depending
on the just-removed ELF loader. I'm fixing that now; please give me a
heads-up on the 3.0.5 release so I'm not caught out.

Aside from that, it seems that xc_linux_build() is actually not called
by anybody any longer. It looks like changeset a20ec270998b (from
Brendan) removed the call from xc.c, replacing it with a call to the
"new style" domain builder via xc_dom_linux_build().

The new domain builder infrastructure is not flexible enough for
PowerPC, so we're sticking with our own xc_linux_build(). It sounded
before like that would be possible, so I assume a20ec270998b was just an
oversight?

-- 
Hollis Blanchard
IBM Linux Technology Center

Re: [XenPPC] Re: New domain builder in xen-unstable

[Brendan Cully]

On Tuesday, 06 March 2007 at 14:56, Hollis Blanchard wrote:
> On Fri, 2007-01-26 at 17:08 +0000, Keir Fraser wrote:
> > xc_linux_build() is still provided in the 'new world' and is still
> > used by xend. For ia64/x86 it is now a thin wrapper around Gerd's new
> > domain-building infrastructure. For ppc you can continue to have your
> > very own version under libxc/powerpc if you want to, and not use
> > Gerd's new domain-building infrastructure. But the elf code that you
> > rely on is going away, so you need to move to using libelf, which I'm
> > sure Gerd can give you some pointers about. 
> 
> Hi Keir, it turns out I was confused about libxc, and we were depending
> on the just-removed ELF loader. I'm fixing that now; please give me a
> heads-up on the 3.0.5 release so I'm not caught out.
> 
> Aside from that, it seems that xc_linux_build() is actually not called
> by anybody any longer. It looks like changeset a20ec270998b (from
> Brendan) removed the call from xc.c, replacing it with a call to the
> "new style" domain builder via xc_dom_linux_build().
> 
> The new domain builder infrastructure is not flexible enough for
> PowerPC, so we're sticking with our own xc_linux_build(). It sounded
> before like that would be possible, so I assume a20ec270998b was just an
> oversight?

I have to admit, I haven't really looked at how other architectures
are overriding the domain building stuff. Would it be possible to
override xc_dom_linux_build to grab the couple of extra parameters
you need (cmdline, features) from dom and pass them to your existing
xc_linux_build? Then you can pass back elfnotes to python at your
convenience, if/when it makes sense.

Re: [XenPPC] Re: New domain builder in xen-unstable

[Keir Fraser]

On 6/3/07 20:56, "Hollis Blanchard" <hollisb@us.ibm.com> wrote:

> Hi Keir, it turns out I was confused about libxc, and we were depending
> on the just-removed ELF loader. I'm fixing that now; please give me a
> heads-up on the 3.0.5 release so I'm not caught out.

You have a couple of weeks at least.

> Aside from that, it seems that xc_linux_build() is actually not called
> by anybody any longer. It looks like changeset a20ec270998b (from
> Brendan) removed the call from xc.c, replacing it with a call to the
> "new style" domain builder via xc_dom_linux_build().
> 
> The new domain builder infrastructure is not flexible enough for
> PowerPC, so we're sticking with our own xc_linux_build(). It sounded
> before like that would be possible, so I assume a20ec270998b was just an
> oversight?

Hmm yes. You can #ifdef in the Python wrapper for now. But I'm surprised
that you can't move to the new domain builder at all -- it has hooks for
arch-dependent code to be inserted already, and we could add more if there's
a need.

 -- Keir

Re: [XenPPC] Re: New domain builder in xen-unstable

[Hollis Blanchard]

On Wed, 2007-03-07 at 08:27 +0000, Keir Fraser wrote:
> 
> > The new domain builder infrastructure is not flexible enough for
> > PowerPC, so we're sticking with our own xc_linux_build(). It sounded
> > before like that would be possible, so I assume a20ec270998b was
> just an
> > oversight?
> 
> Hmm yes. You can #ifdef in the Python wrapper for now. But I'm
> surprised that you can't move to the new domain builder at all -- it
> has hooks for arch-dependent code to be inserted already, and we could
> add more if there's a need.

It's pretty complex, which you may not realize since Gerd did the x86
work for you. FYI, I've been working on this for three days, and when
I'm done I will only have un-broken PowerPC. Dubious use of time IMHO.

I can't just ifdef PowerPC's xc_linux_build back in, because libelf
doesn't map page-by-page like the old ELF loader did. That means I need
to pre-map the memory, which starts dragging in xc_dom infrastructure.
I'm tempted to copy-paste-and-hack that infrastructure, but I'm trying
to be a good person and fit into the common code (avoiding ifdefs)
wherever possible.

Here's my current confusion: where are these ELF features described?
Since PowerPC domU communicate only via GPFNs, do I need to set the
"auto-translated" feature?

What is the difference between dom->shadow_enable and
xc_dom_feature_translated()?

-- 
Hollis Blanchard
IBM Linux Technology Center

Re: [XenPPC] Re: New domain builder in xen-unstable

[Gerd Hoffmann]

Hollis Blanchard wrote:
> I can't just ifdef PowerPC's xc_linux_build back in, because libelf
> doesn't map page-by-page like the old ELF loader did. That means I need
> to pre-map the memory, which starts dragging in xc_dom infrastructure.

You don't need more xc_dom infrastructure, I've tried to make the libelf
bits independant of the xc_dom stoff for exactly that reason.

Have a look at the hvm loader (xc_hvm_build.c).  It uses libelf too, it
has a loadelfimage() which maps the memory, calls elf_load_binary,
unmaps again.  You could do that too.

Oh, and an example of page-by-page loading using libelf is in
xen/arch/ia64/xen/domain.c.

> What is the difference between dom->shadow_enable and
> xc_dom_feature_translated()?

It's identical now I think.  In the early days (before split-off elf
handling code to libelf) I've tried to use the new domain builder for
hvm too, the logic whevener pfn->mfn translation is needed was a bit
more complex than just looking up shadow_enabled.

HTH,
  Gerd

-- 
Gerd Hoffmann <kraxel@suse.de>