guest kexec / domU boot loader

guest kexec / domU boot loader

[Gerd Hoffmann]

  Hi,

I have some more few question/problems wrt. kexec.  They are not
strictly kexec-related, I think any complete control pass-over from one
kernel to another has them, i.e. if someone wants to combine mini-os and
grub to build a boot loader the same issues will come up:

Problem number one are the different "magic" pages: start info, shared
info, xenstore, console.  Right now they are allocated at a random place
in guest physical memory (depending on kernel and initrd size).  One
problem with that is that they are in the area where you might want to
load a kernel to.  Sure you can map them somewhere else in virtual
space, but if you want to keep things very simple for a mini-kernel or
bootloader and create a simple virt==phys identity-mapping it's a nasty
special case.

Is it possible to move those magic pages to a different place in guest
physical memory?  With and without auto_translated_physmap?  I think
start_info is easy as it can be simply copied.  shared_info isn't in the
guests physical memory so it isn't a issue I think.  The console page
and the xenstore page are not so easy.  I think without
auto_translated_physmap the guest can move them around by updating the
p2m and m2p tables.  With auto_translated_physmap I see no obvious way
to do that though.  Hints?

Problem two is passing the p2m table and handle page holes which might
be in guest physical memory, no idea yet how to handle that best.

Problem tree is that the current initial page table setup created by the
domain builders doesn't include a low memory identity mapping.  Creating
such a mapping from a in-domain boot loader and then jump to the entry
point isn't trivial ...

cheers,

  Gerd

-- 
Gerd 'just married' Hoffmann <kraxel@suse.de>
I'm the hacker formerly known as Gerd Knorr.
http://www.suse.de/~kraxel/just-married.jpeg

Re: guest kexec / domU boot loader

[Jacob Gorm Hansen]

On 3/10/06, Gerd Hoffmann <kraxel@suse.de> wrote:
>   Hi,
>
> I have some more few question/problems wrt. kexec.  They are not
> strictly kexec-related, I think any complete control pass-over from one
> kernel to another has them, i.e. if someone wants to combine mini-os and
> grub to build a boot loader the same issues will come up:

> Problem number one are the different "magic" pages: start info, shared
> info, xenstore, console.  Right now they are allocated at a random place
> in guest physical memory (depending on kernel and initrd size).  One
> problem with that is that they are in the area where you might want to
> load a kernel to.  Sure you can map them somewhere else in virtual
> space, but if you want to keep things very simple for a mini-kernel or
> bootloader and create a simple virt==phys identity-mapping it's a nasty
> special case.

I may be repeating myself here, but if you have a look at the file
linux_loader.c in extras/cstrap, in my tree at
http://www.distlab.dk/hg/index.cgi/xen-gfx.hg , you should see a
solution to the problem of these special pages. Since the domU
bootloader sets up the virtual space of the guest (code mostly pasted
from xc_linux_build.c), pointing the relevant virtual addresses to the
correct mfns (from the start info page) by overriding entries in the
page_array[] is fairly trivial.

Best regards,
Jacob

Re: guest kexec / domU boot loader

[Keir Fraser]

On 10 Mar 2006, at 10:53, Gerd Hoffmann wrote:

> Is it possible to move those magic pages to a different place in guest
> physical memory?  With and without auto_translated_physmap?

As you say, without auto_translated_physmap it's up to the guest 
if/where he puts those pages in his physmap. He has control of the p2m 
and m2p mappings.

With auto_translate, you should use the new MEMOP_add_to_physmap call. 
It allows you to move shared_info and grant_table pages around in the 
physmap space (if you call and they already have a mapping, the old 
mapping goes away). I could also add a gpfn special value to have them 
removed from the physmap entirely (maybe useful for kexec?).

  -- Keir

Re: guest kexec / domU boot loader

[Gerd Hoffmann]

Keir Fraser wrote:

> With auto_translate, you should use the new MEMOP_add_to_physmap call.
> It allows you to move shared_info and grant_table pages around in the
> physmap space (if you call and they already have a mapping, the old
> mapping goes away).

What happens with the place where the mapping was?  Does a hole appear
there or is the "magic" page replaced by a normal page?  Can I move the
console and xenstore pages too?

> I could also add a gpfn special value to have them
> removed from the physmap entirely (maybe useful for kexec?).

I don't see a need for that right now.

cheers,

  Gerd

-- 
Gerd 'just married' Hoffmann <kraxel@suse.de>
I'm the hacker formerly known as Gerd Knorr.
http://www.suse.de/~kraxel/just-married.jpeg

Re: guest kexec / domU boot loader

[Keir Fraser]

On 15 Mar 2006, at 11:40, Gerd Hoffmann wrote:

>> With auto_translate, you should use the new MEMOP_add_to_physmap call.
>> It allows you to move shared_info and grant_table pages around in the
>> physmap space (if you call and they already have a mapping, the old
>> mapping goes away).
>
> What happens with the place where the mapping was?  Does a hole appear
> there or is the "magic" page replaced by a normal page?  Can I move the
> console and xenstore pages too?

You end up with a hole there. Can be filled with MEMOP_populate_physmap 
if you wish. The console and xenstore pages cannot currently be moved, 
but I can add support. Since the only handle the guest has on those is 
their current gpfn, and Xen doesn't know about their special status, 
the best approach is probably to add another 'source space' to the 
MEMOP_add_to_physmap call, effectively allowing you to move a memory 
page from one gpfn to another by specifying the source and target 
gpfns.

Sound good?

  -- Keir

Re: guest kexec / domU boot loader

[Gerd Hoffmann]

Keir Fraser wrote:
> 
> On 15 Mar 2006, at 11:40, Gerd Hoffmann wrote:
> 
> You end up with a hole there. Can be filled with MEMOP_populate_physmap
> if you wish. The console and xenstore pages cannot currently be moved,
> but I can add support. Since the only handle the guest has on those is
> their current gpfn, and Xen doesn't know about their special status, the
> best approach is probably to add another 'source space' to the
> MEMOP_add_to_physmap call, effectively allowing you to move a memory
> page from one gpfn to another by specifying the source and target gpfns.
> 
> Sound good?

Yep, sounds good.

Can XENMEM_translate_gpfn_list be used to figure where holes in the
physmap (due to ballooning for example) are?  Guess I can "plug" these
holes then by moving around pages using the new source space?

cheers,

  Gerd

-- 
Gerd 'just married' Hoffmann <kraxel@suse.de>
I'm the hacker formerly known as Gerd Knorr.
http://www.suse.de/~kraxel/just-married.jpeg

Re: guest kexec / domU boot loader

[Keir Fraser]

On 15 Mar 2006, at 12:18, Gerd Hoffmann wrote:

> Yep, sounds good.
>
> Can XENMEM_translate_gpfn_list be used to figure where holes in the
> physmap (due to ballooning for example) are?  Guess I can "plug" these
> holes then by moving around pages using the new source space?

Well, I think it should. I think a hole will return ~0UL as the mfn. 
That probably deserves an interface macro.

  -- Keir