All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] Xen Guest Kexec
@ 2005-07-07 17:16 Mark Williamson
  2005-07-08  8:59 ` [Fastboot] " Eric W. Biederman
                   ` (2 more replies)
  0 siblings, 3 replies; 15+ messages in thread
From: Mark Williamson @ 2005-07-07 17:16 UTC (permalink / raw)
  To: xen-devel, fastboot

All,

I'm posting to the Xen-devel list and the OSDL fastboot list.  There are a 
number of Xen folks who'll want to look at the code.  For the fastboot folks, 
this is mostly intended as an announcement of the port.  Please limit your 
replies to the list(s) that they applies to, thanks :-)

A number of people have expressed interest in kexec support for Xen guests.  
In particular, it's mainly useful for purposes of implementing an in-guest 
bootloader app and avoiding the need for dom0 to access the guest filesystem.

It's a largish patch, so I stuck it online: 
http://www.cl.cam.ac.uk/~maw48/xenguest_kexec.patch
Tested on i386.  I suspect it'll have build issues on x86_64 but those should 
be easy to fix - actually the code is probably generic enough to work on 
both.

You'll also need a modified version of the kexec tools, which I'll post 
details of later.

Notes on the implementation:
Kexec in a Xen guest doesn't work like on a real machine.  The guest relies on 
outside assistance to complete the job.  The guest communicates details of 
the kexec to Xend.  Xend extracts the data from the guest's memory image and 
uses it to rebuild the domain with a new kernel, etc.  This approach 
simplifies the changes needed to support kexec and minimises code duplication 
between Linux and the domain builder itself.

Kexec-ing the whole host is a separate issue, I think it's best addressed with 
a port of kexec to Xen itself.

Cheers,
Mark

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [Fastboot] [PATCH] Xen Guest Kexec
  2005-07-07 17:16 [PATCH] Xen Guest Kexec Mark Williamson
@ 2005-07-08  8:59 ` Eric W. Biederman
  2005-07-08 14:35   ` Mark Williamson
  2005-07-08 17:35 ` Mark Williamson
  2006-02-23 10:52 ` Horms
  2 siblings, 1 reply; 15+ messages in thread
From: Eric W. Biederman @ 2005-07-08  8:59 UTC (permalink / raw)
  To: Mark Williamson; +Cc: xen-devel, fastboot

Mark Williamson <mark.williamson@cl.cam.ac.uk> writes:

> All,
>
> I'm posting to the Xen-devel list and the OSDL fastboot list.  There are a 
> number of Xen folks who'll want to look at the code.  For the fastboot folks, 
> this is mostly intended as an announcement of the port.  Please limit your 
> replies to the list(s) that they applies to, thanks :-)
>
> A number of people have expressed interest in kexec support for Xen guests.  
> In particular, it's mainly useful for purposes of implementing an in-guest 
> bootloader app and avoiding the need for dom0 to access the guest filesystem.

Is kexec at all useful for generating the initial image as well?
I believe all it would require would be defining an extra kexec type,
to load an image into the new domain.

> It's a largish patch, so I stuck it online: 
> http://www.cl.cam.ac.uk/~maw48/xenguest_kexec.patch
> Tested on i386.  I suspect it'll have build issues on x86_64 but those should 
> be easy to fix - actually the code is probably generic enough to work on 
> both.

Part of that patch is that you have another patch file showing up
in your diff.

> You'll also need a modified version of the kexec tools, which I'll post 
> details of later.
>
> Notes on the implementation:
> Kexec in a Xen guest doesn't work like on a real machine.  The guest relies on 
> outside assistance to complete the job.  The guest communicates details of 
> the kexec to Xend.  Xend extracts the data from the guest's memory image and 
> uses it to rebuild the domain with a new kernel, etc.  This approach 
> simplifies the changes needed to support kexec and minimises code duplication 
> between Linux and the domain builder itself.

How useful will you your kexec implementation handle the kexec on panic
case?  In that case we kexec a kernel at an alternative address and
then read from the memory what the previous kernel had left in it's
physical memory, and generate a crash dump of it.

> Kexec-ing the whole host is a separate issue, I think it's best addressed with 
> a port of kexec to Xen itself.

Sounds fun. :)

Eric

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [Fastboot] [PATCH] Xen Guest Kexec
  2005-07-08  8:59 ` [Fastboot] " Eric W. Biederman
@ 2005-07-08 14:35   ` Mark Williamson
  0 siblings, 0 replies; 15+ messages in thread
From: Mark Williamson @ 2005-07-08 14:35 UTC (permalink / raw)
  To: Eric W. Biederman; +Cc: xen-devel, fastboot

> > A number of people have expressed interest in kexec support for Xen
> > guests. In particular, it's mainly useful for purposes of implementing an
> > in-guest bootloader app and avoiding the need for dom0 to access the
> > guest filesystem.
>
> Is kexec at all useful for generating the initial image as well?
> I believe all it would require would be defining an extra kexec type,
> to load an image into the new domain.

Actually, the mechanism is already practically the same: we already have a 
domain builder that could stick the kernel into memory and build a set of 
bootstrap page tables before kicking the domain into life.  Kexec support 
hooks into that by causing the domain builder to be invoked with the new 
kernel, etc.

> > It's a largish patch, so I stuck it online:
> > http://www.cl.cam.ac.uk/~maw48/xenguest_kexec.patch
> > Tested on i386.  I suspect it'll have build issues on x86_64 but those
> > should be easy to fix - actually the code is probably generic enough to
> > work on both.
>
> Part of that patch is that you have another patch file showing up
> in your diff.

Yep.  We've got a "patches" directory for things we're waiting on being pushed 
upstream.  I added the Linux generic / i386 kexec support into there for the 
time-being.  Possibly I should omit this from a "lite" version of the patch 
for easy reading, since it's a bit confusing if you're just reading...

> How useful will you your kexec implementation handle the kexec on panic
> case?  In that case we kexec a kernel at an alternative address and
> then read from the memory what the previous kernel had left in it's
> physical memory, and generate a crash dump of it.

I've thought a bit about this but there are various restrictions that stop Xen 
from building a new image into an existing domain.  Because of this, my patch 
destroys the domain and builds a completely new one with the new kernel.  
Obviously this precludes kdump for now...

The limitations in Xen could be fixed, however we already have a mechanism for 
core-dumping a domain from the "outside", so the motivation doesn't look that 
strong for now.  KDump might still be useful to people who want their 
standard distribution support for dumping to "just work", so it might still 
be worth looking into.

> > Kexec-ing the whole host is a separate issue, I think it's best addressed
> > with a port of kexec to Xen itself.
>
> Sounds fun. :)

Indeed :-)  kdump could be more useful here for debugging whole-host issues.  
The more sensible way to do this would probably be to jump into a minimal 
(Xen-free) Linux, just as normal kdump does, in the event that either Xen or 
the dom0 Linux crashes.

kexec support here would be quite interesting: machine_kexec.c and 
relocate_kernel.S will have to be in Xen itself.  Code for shutting down PCI 
devices will have to be in dom0!  :-)

A neat trick would be avoid implementing the actual "loading" process in Xen - 
just use a kernel that's been loaded by dom0's kernel.  Then Xen only has to 
contain the machine-specific code.  I've designed the mechanism so that it 
can be used for this in future.

Cheers,
Mark

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] Xen Guest Kexec
  2005-07-07 17:16 [PATCH] Xen Guest Kexec Mark Williamson
  2005-07-08  8:59 ` [Fastboot] " Eric W. Biederman
@ 2005-07-08 17:35 ` Mark Williamson
  2006-02-23 10:52 ` Horms
  2 siblings, 0 replies; 15+ messages in thread
From: Mark Williamson @ 2005-07-08 17:35 UTC (permalink / raw)
  To: xen-devel; +Cc: fastboot

As promised, I'm following up with further patches:

This patch:
* http://www.cl.cam.ac.uk/~maw48/kexec/xenguest_kexec_tools.patch
Adds Xen guest kexec support to the kexec tools.  You'll need this to actually 
use the kexec support.

This patch:
* http://www.cl.cam.ac.uk/~maw48/kexec/xenguest_kexec.patch
Is the patch to the Xen distribution, including all my changes *and* the 
"core" kexec changes.  Applying this to a xen-unstable repository allows you 
to build kexec-enabled guest kernels.

This patch:
* http://www.cl.cam.ac.uk/~maw48/kexec/xenguest_kexec_litepatch.patch
Is a reduced version of the above patch, omitting the "core" kexec changes, so 
that it's easier to see what I've added.

My intention is that the changes to the Xen distribution should get merged 
once we've replaced the control message interface in the control tools.  It 
should be there for the 3.0 release.

Eric: do you have any objections to my approach?  I'd ideally like to aim for 
a merge of the tools support at the same time (or before) Xen support arrives 
in mainline Linux.

Cheers,
Mark

On Thursday 07 July 2005 18:16, Mark Williamson wrote:
> All,
>
> I'm posting to the Xen-devel list and the OSDL fastboot list.  There are a
> number of Xen folks who'll want to look at the code.  For the fastboot
> folks, this is mostly intended as an announcement of the port.  Please
> limit your replies to the list(s) that they applies to, thanks :-)
>
> A number of people have expressed interest in kexec support for Xen guests.
> In particular, it's mainly useful for purposes of implementing an in-guest
> bootloader app and avoiding the need for dom0 to access the guest
> filesystem.
>
> It's a largish patch, so I stuck it online:
> http://www.cl.cam.ac.uk/~maw48/xenguest_kexec.patch
> Tested on i386.  I suspect it'll have build issues on x86_64 but those
> should be easy to fix - actually the code is probably generic enough to
> work on both.
>
> You'll also need a modified version of the kexec tools, which I'll post
> details of later.
>
> Notes on the implementation:
> Kexec in a Xen guest doesn't work like on a real machine.  The guest relies
> on outside assistance to complete the job.  The guest communicates details
> of the kexec to Xend.  Xend extracts the data from the guest's memory image
> and uses it to rebuild the domain with a new kernel, etc.  This approach
> simplifies the changes needed to support kexec and minimises code
> duplication between Linux and the domain builder itself.
>
> Kexec-ing the whole host is a separate issue, I think it's best addressed
> with a port of kexec to Xen itself.
>
> Cheers,
> Mark
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] Xen Guest Kexec
  2005-07-07 17:16 [PATCH] Xen Guest Kexec Mark Williamson
  2005-07-08  8:59 ` [Fastboot] " Eric W. Biederman
  2005-07-08 17:35 ` Mark Williamson
@ 2006-02-23 10:52 ` Horms
  2006-02-23 11:22   ` Gerd Hoffmann
  2006-02-23 11:36   ` Mark Williamson
  2 siblings, 2 replies; 15+ messages in thread
From: Horms @ 2006-02-23 10:52 UTC (permalink / raw)
  To: xen-devel

Mark Williamson <mark.williamson@cl.cam.ac.uk> wrote:
> All,
> 
> I'm posting to the Xen-devel list and the OSDL fastboot list.  There are a 
> number of Xen folks who'll want to look at the code.  For the fastboot folks, 
> this is mostly intended as an announcement of the port.  Please limit your 
> replies to the list(s) that they applies to, thanks :-)
> 
> A number of people have expressed interest in kexec support for Xen guests.  
> In particular, it's mainly useful for purposes of implementing an in-guest 
> bootloader app and avoiding the need for dom0 to access the guest filesystem.
> 
> It's a largish patch, so I stuck it online: 
> http://www.cl.cam.ac.uk/~maw48/xenguest_kexec.patch
> Tested on i386.  I suspect it'll have build issues on x86_64 but those should 
> be easy to fix - actually the code is probably generic enough to work on 
> both.
> 
> You'll also need a modified version of the kexec tools, which I'll post 
> details of later.
> 
> Notes on the implementation:
> Kexec in a Xen guest doesn't work like on a real machine.  The guest relies on 
> outside assistance to complete the job.  The guest communicates details of 
> the kexec to Xend.  Xend extracts the data from the guest's memory image and 
> uses it to rebuild the domain with a new kernel, etc.  This approach 
> simplifies the changes needed to support kexec and minimises code duplication 
> between Linux and the domain builder itself.
> 
> Kexec-ing the whole host is a separate issue, I think it's best addressed with 
> a port of kexec to Xen itself.

Hi,

I have been looking into various crash-dump options, and with that in
mind, exploring kexec for Dom0. I'm wondering if the patch above is
still the latest, and if this is still the prefered approach
for kexec in Dom0.

-- 
Horms

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Re: [PATCH] Xen Guest Kexec
  2006-02-23 10:52 ` Horms
@ 2006-02-23 11:22   ` Gerd Hoffmann
  2006-02-23 11:32     ` Mark Williamson
  2006-02-23 11:36   ` Mark Williamson
  1 sibling, 1 reply; 15+ messages in thread
From: Gerd Hoffmann @ 2006-02-23 11:22 UTC (permalink / raw)
  To: Horms; +Cc: xen-devel

> I have been looking into various crash-dump options, and with that in
> mind, exploring kexec for Dom0. I'm wondering if the patch above is
> still the latest, and if this is still the prefered approach
> for kexec in Dom0.

For crash-dumping you'll can simply ask xend to write out an dump on
domain crash, then inspect it using gdbserver-xen ;)

For kexec I'm looking into getting kexec work right now.  For domU's it
shouldn't be that hard I think.  dom0 likely is much harder given how
xen and dom0 work hand-in-hand to drive the hardware ...

cheers,

  Gerd

-- 
Gerd 'just married' Hoffmann <kraxel@suse.de>
I'm the hacker formerly known as Gerd Knorr.
http://www.suse.de/~kraxel/just-married.jpeg

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Re: [PATCH] Xen Guest Kexec
  2006-02-23 11:22   ` Gerd Hoffmann
@ 2006-02-23 11:32     ` Mark Williamson
  2006-02-23 13:03       ` Gerd Hoffmann
  0 siblings, 1 reply; 15+ messages in thread
From: Mark Williamson @ 2006-02-23 11:32 UTC (permalink / raw)
  To: xen-devel; +Cc: Gerd Hoffmann, Horms

> For crash-dumping you'll can simply ask xend to write out an dump on
> domain crash, then inspect it using gdbserver-xen ;)
>
> For kexec I'm looking into getting kexec work right now.  For domU's it
> shouldn't be that hard I think.  dom0 likely is much harder given how
> xen and dom0 work hand-in-hand to drive the hardware ...

How are you planning to do domU kexec?

For domUs the major problem I found was:
a) the existing kexec code doesn't understand pseudophysical memory
b) we had no way (at the time) of resetting virtual devices - disconnecting 
cleanly and then reconnecting again
c) creating a start-of-day environment for the reboot kernel duplicates a load 
of code that's already in the domain builder

Because of this, I used a technique I called "assisted kexec" where the guest 
would write out a descriptor which could be used by Xend to (safely, without 
trusting the guest) extract the reboot kernel from guest memory, and rebuild 
the domain with that kernel, using the standard domain builder.  This worked 
quite well actually resulted in less code overall.

For dom0 kexec, I thought the best approach would be to port the existing 
Linux kexec machinery to Xen (should be quite straightforward - just the part 
which copies the kernel image to the correct place, switches off paging, 
jumps into the new kernel).  Then the dom0 kexec code would make a hypercall 
after closing its devices, rather than trying to execute that code itself.

What do you think?

Cheers,
Mark

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Re: [PATCH] Xen Guest Kexec
  2006-02-23 10:52 ` Horms
  2006-02-23 11:22   ` Gerd Hoffmann
@ 2006-02-23 11:36   ` Mark Williamson
  2006-02-27  3:32     ` Horms
  1 sibling, 1 reply; 15+ messages in thread
From: Mark Williamson @ 2006-02-23 11:36 UTC (permalink / raw)
  To: xen-devel; +Cc: Gerd Hoffmann, Horms

> I have been looking into various crash-dump options, and with that in
> mind, exploring kexec for Dom0. I'm wondering if the patch above is
> still the latest, and if this is still the prefered approach
> for kexec in Dom0.

My patch was out-of-tree so it's rotted.  I've been waiting for various other 
things to get sorted before it's worth updating it again (updating to a 
kernel that actually has the kexec system call, one or two other patches).

The patch wouldn't allow kexec for dom0 anyhow.  Enabling kdump for dom0 would 
be useful.  I think there was somebody considering looking at this in the 
future...

Cheers,
Mark

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Re: [PATCH] Xen Guest Kexec
  2006-02-23 11:32     ` Mark Williamson
@ 2006-02-23 13:03       ` Gerd Hoffmann
  2006-02-23 14:49         ` Mark Williamson
  0 siblings, 1 reply; 15+ messages in thread
From: Gerd Hoffmann @ 2006-02-23 13:03 UTC (permalink / raw)
  To: Mark Williamson; +Cc: xen-devel, Horms

Mark Williamson wrote:
> How are you planning to do domU kexec?

As close as possible to normal i386 kexec ...

> For domUs the major problem I found was:
> a) the existing kexec code doesn't understand pseudophysical memory
> b) we had no way (at the time) of resetting virtual devices - disconnecting 
> cleanly and then reconnecting again
> c) creating a start-of-day environment for the reboot kernel duplicates a load 
> of code that's already in the domain builder

Dis- and reconnecting should be ok by now I guess.  I expect the paging
setup being the most tricky part:  First because the pseudophysical
memory (probably not a major issue though).  Second because unlike i386
kexec we'll have to run with paging enabled all the time ...

My plan is to first make xenlinux kernels work with correct ELF headers
(patches on the list last days ;), then make kexec userspace work
(unmodified if possible), last step the in-kernel stuff which performs
the actual kexec ...

> For dom0 kexec, I thought the best approach would be to port the existing 
> Linux kexec machinery to Xen (should be quite straightforward - just the part 
> which copies the kernel image to the correct place, switches off paging, 
> jumps into the new kernel).  Then the dom0 kexec code would make a hypercall 
> after closing its devices, rather than trying to execute that code itself.

Right now linux kexec depends on the new kernel having a different
physical (and virtual) start address, so taking the very same approach
likely doesn't work.

cheers,

  Gerd

-- 
Gerd 'just married' Hoffmann <kraxel@suse.de>
I'm the hacker formerly known as Gerd Knorr.
http://www.suse.de/~kraxel/just-married.jpeg

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Re: [PATCH] Xen Guest Kexec
  2006-02-23 13:03       ` Gerd Hoffmann
@ 2006-02-23 14:49         ` Mark Williamson
  2006-02-24 14:35           ` Gerd Hoffmann
  0 siblings, 1 reply; 15+ messages in thread
From: Mark Williamson @ 2006-02-23 14:49 UTC (permalink / raw)
  To: Gerd Hoffmann; +Cc: xen-devel, Horms

> As close as possible to normal i386 kexec ...
>
> Dis- and reconnecting should be ok by now I guess.  I expect the paging
> setup being the most tricky part:  First because the pseudophysical
> memory (probably not a major issue though).  Second because unlike i386
> kexec we'll have to run with paging enabled all the time ...

I doubt having paging enabled would be too painful.  i386 kexec disables 
paging right at the end of the process so that the new kernel will have a 
sensible start-of-day.  We'd just need start-of-day to contain bootstrap 
pagetables, same as for normal Linux.  Ideally you'd need to find a slot in 
the bootstrap tables for the trampoline code to live, if you take that 
approach.

You've got a load of other things to worry about in this approach, like 
un-type-pinning all pages you own, etc.

The generic kexec code doesn't understand phys vs machine memory, IIRC, so you 
may need to worry about it mis-allocating your trampoline page (this is an 
issue because you need to identity map the trampoline page later on in the 
process).

> My plan is to first make xenlinux kernels work with correct ELF headers
> (patches on the list last days ;), then make kexec userspace work
> (unmodified if possible), last step the in-kernel stuff which performs
> the actual kexec ...

I patched kexec userspace, but partly that was so that it wouldn't complain 
about the Xen-format image files.  I also seem to remember I built a 
descriptor that described the size of the kernel, ramdisk, etc for the 
benefit of the kexec code but this probably wasn't strictly necessary.  I 
might even have removed that in the end...  Can't remember :-)

> > For dom0 kexec, I thought the best approach would be to port the existing
> > Linux kexec machinery to Xen (should be quite straightforward - just the
> > part which copies the kernel image to the correct place, switches off
> > paging, jumps into the new kernel).  Then the dom0 kexec code would make
> > a hypercall after closing its devices, rather than trying to execute that
> > code itself.
>
> Right now linux kexec depends on the new kernel having a different
> physical (and virtual) start address, so taking the very same approach
> likely doesn't work.

I'm not convinced: the reboot kernel doesn't need to be any different from the 
standard kernel *unless* you're running kdump (when the kernel will need to 
live in a different place so that it doesn't stomp on the main kernel - not a 
limitation of kexec).  Or am I misunderstanding what you meant?

Anyhow, it'd be nice to have kexec working.  But I'd still suggest that you 
take a quick look at my patch and consider implementing it as a platform 
service, rather than implementing all the low-level gunk.  It really is much 
less code that way.

Cheers,
Mark

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Re: [PATCH] Xen Guest Kexec
  2006-02-23 14:49         ` Mark Williamson
@ 2006-02-24 14:35           ` Gerd Hoffmann
  2006-02-27 14:29             ` Mark Williamson
  0 siblings, 1 reply; 15+ messages in thread
From: Gerd Hoffmann @ 2006-02-24 14:35 UTC (permalink / raw)
  To: Mark Williamson; +Cc: xen-devel, Horms

Mark Williamson wrote:
>> As close as possible to normal i386 kexec ...
>>
>> Dis- and reconnecting should be ok by now I guess.  I expect the paging
>> setup being the most tricky part:  First because the pseudophysical
>> memory (probably not a major issue though).  Second because unlike i386
>> kexec we'll have to run with paging enabled all the time ...
> 
> I doubt having paging enabled would be too painful.  i386 kexec disables 
> paging right at the end of the process so that the new kernel will have a 
> sensible start-of-day.  We'd just need start-of-day to contain bootstrap 
> pagetables, same as for normal Linux.  Ideally you'd need to find a slot in 
> the bootstrap tables for the trampoline code to live, if you take that 
> approach.

x86-64 has paging enabled while trampoline is running too, so I can get
some ideas there ;)

> You've got a load of other things to worry about in this approach, like 
> un-type-pinning all pages you own, etc.

Yep, that is a problem.  What pages are pinned (other than pgd)?  I've
seen plenty of pages with PG_pinned set, but can't figure easily what
pages that are ...

Also switching page tables seems to be not so easy.  Is it possible to
switch atomically to a new, completely independant page table tree?
i.e. old tree is valid (of cource), new tree is too, but the pages of
the old tree are not mapped read-only in the new tree (and visa versa).

> The generic kexec code doesn't understand phys vs machine memory, IIRC, so you 
> may need to worry about it mis-allocating your trampoline page (this is an 
> issue because you need to identity map the trampoline page later on in the 
> process).

Not a big issue if paging is enabled anyway, we can use a identity map
then (virtual == physical, not virtual == machine), so kexec doesn't
even notice outside the arch-specific code.

>> Right now linux kexec depends on the new kernel having a different
>> physical (and virtual) start address, so taking the very same approach
>> likely doesn't work.
> 
> I'm not convinced: the reboot kernel doesn't need to be any different from the 
> standard kernel *unless* you're running kdump (when the kernel will need to 
> live in a different place so that it doesn't stomp on the main kernel - not a 
> limitation of kexec).  Or am I misunderstanding what you meant?

I think it's needed in both cases, at least I had problems making normal
kexec work (without xen) when both kernels had the same physical
address.  Might have been a simple bug though.

cheers,

  Gerd

-- 
Gerd 'just married' Hoffmann <kraxel@suse.de>
I'm the hacker formerly known as Gerd Knorr.
http://www.suse.de/~kraxel/just-married.jpeg

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Re: [PATCH] Xen Guest Kexec
  2006-02-23 11:36   ` Mark Williamson
@ 2006-02-27  3:32     ` Horms
  2006-02-27 14:58       ` Mark Williamson
  0 siblings, 1 reply; 15+ messages in thread
From: Horms @ 2006-02-27  3:32 UTC (permalink / raw)
  To: Mark Williamson; +Cc: Gerd Hoffmann, xen-devel

On Thu, Feb 23, 2006 at 11:36:36AM +0000, Mark Williamson wrote:
> > I have been looking into various crash-dump options, and with that in
> > mind, exploring kexec for Dom0. I'm wondering if the patch above is
> > still the latest, and if this is still the prefered approach
> > for kexec in Dom0.
> 
> My patch was out-of-tree so it's rotted.  I've been waiting for various other 
> things to get sorted before it's worth updating it again (updating to a 
> kernel that actually has the kexec system call, one or two other patches).

Could you be a little more specific about that. I am pretty interested
in getting this working. As far as I understand Xen is now running
on 2.6.16-rc4 which does have kexec. I'm happy to put some cycles into
updating the patch, though I must confess I am quite interested in
Gerd's approach which is closer to what I was originally thinking.

> The patch wouldn't allow kexec for dom0 anyhow.  Enabling kdump for dom0 would 
> be useful.  I think there was somebody considering looking at this in the 
> future...

Understood. I was only talking of DomU with regards to the assisted
kexec patch that you sent earlier. I am also intersted in getting
kexec working in Dom0, however I agree with Gerd's statements elsewhere
in this thread that it can be acchived by porting kexec to Xen (or as
I understand it given the current state of kexec, teaching kexec a few
things about Xen in the prevailing architecture).

I'm actually quite interested in getting both kexec and kdump working
on both DomU and Dom0. I understand that for the most port there are
really four different problems there. I'd really like to help out
with existing efforts to help get all or any of them working.

For now I'm going to poke around with Gerd's idea of asking xend to
do system dumps for crash analysis of DomU, as that seems to be an area
of little activity at this time. 

-- 
Horms

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Re: [PATCH] Xen Guest Kexec
  2006-02-24 14:35           ` Gerd Hoffmann
@ 2006-02-27 14:29             ` Mark Williamson
  2006-02-27 15:39               ` Gerd Hoffmann
  0 siblings, 1 reply; 15+ messages in thread
From: Mark Williamson @ 2006-02-27 14:29 UTC (permalink / raw)
  To: Gerd Hoffmann; +Cc: xen-devel, Horms

> > You've got a load of other things to worry about in this approach, like
> > un-type-pinning all pages you own, etc.
>
> Yep, that is a problem.  What pages are pinned (other than pgd)?  I've
> seen plenty of pages with PG_pinned set, but can't figure easily what
> pages that are ...

When I was looking at this approach I was intending to retreive the pageframe 
info from Xen and just run unpin on anything pinned, according to its type.

> > The generic kexec code doesn't understand phys vs machine memory, IIRC,
> > so you may need to worry about it mis-allocating your trampoline page
> > (this is an issue because you need to identity map the trampoline page
> > later on in the process).
>
> Not a big issue if paging is enabled anyway, we can use a identity map
> then (virtual == physical, not virtual == machine), so kexec doesn't
> even notice outside the arch-specific code.

Actually I was talking rubbish there anyhow; of course if you don't disable 
paging the identity mapping isn't a problem.

> >> Right now linux kexec depends on the new kernel having a different
> >> physical (and virtual) start address, so taking the very same approach
> >> likely doesn't work.
<snip>
> I think it's needed in both cases, at least I had problems making normal
> kexec work (without xen) when both kernels had the same physical
> address.  Might have been a simple bug though.

When did you try that?  I've not tried it outside Xen, but it didn't look like 
a requirement from my reading of the code.

A really big chunk of the generic kexec code is there to make sure the new 
kernel is not loaded in an area that conflicts with its destination, much of 
the trampoline is there to enable the new kernel to be copied on top of the 
old one.  Address conflicts should only be an issue if you want the second 
kernel to be a dump kernel, in which case it needs to run in-place after 
loading.

Cheers,
Mark

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Re: [PATCH] Xen Guest Kexec
  2006-02-27  3:32     ` Horms
@ 2006-02-27 14:58       ` Mark Williamson
  0 siblings, 0 replies; 15+ messages in thread
From: Mark Williamson @ 2006-02-27 14:58 UTC (permalink / raw)
  To: Horms; +Cc: Gerd Hoffmann, xen-devel

> > My patch was out-of-tree so it's rotted.  I've been waiting for various
> > other things to get sorted before it's worth updating it again (updating
> > to a kernel that actually has the kexec system call, one or two other
> > patches).
>
> Could you be a little more specific about that. I am pretty interested
> in getting this working. As far as I understand Xen is now running
> on 2.6.16-rc4 which does have kexec. I'm happy to put some cycles into
> updating the patch, though I must confess I am quite interested in
> Gerd's approach which is closer to what I was originally thinking.

My original intention was also like Gerd's approach, I just decided it was 
simpler overall to make it into a platform service - we depend on daemons in 
other domains *anyhow* in order to talk to devices, make reboot work, etc.

I was waiting for the build-from-memory patches to maybe be accepted (which 
would avoid updating that part of my patch) but I haven't had a chance to 
look over them yet.

> Understood. I was only talking of DomU with regards to the assisted
> kexec patch that you sent earlier. I am also intersted in getting
> kexec working in Dom0, however I agree with Gerd's statements elsewhere
> in this thread that it can be acchived by porting kexec to Xen (or as
> I understand it given the current state of kexec, teaching kexec a few
> things about Xen in the prevailing architecture).

Yes.  This shouldn't be so hard, except possible from the PoV of actually 
allocating the memory correctly.

> I'm actually quite interested in getting both kexec and kdump working
> on both DomU and Dom0. I understand that for the most port there are
> really four different problems there. I'd really like to help out
> with existing efforts to help get all or any of them working.
>
> For now I'm going to poke around with Gerd's idea of asking xend to
> do system dumps for crash analysis of DomU, as that seems to be an area
> of little activity at this time.

That should be pretty much as good as kdump, except that the image obviously 
won't appear within the guests filesystem, and it won't integrate with domU 
distros that want to use kdump.

Cheers,
Mark


-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Re: [PATCH] Xen Guest Kexec
  2006-02-27 14:29             ` Mark Williamson
@ 2006-02-27 15:39               ` Gerd Hoffmann
  0 siblings, 0 replies; 15+ messages in thread
From: Gerd Hoffmann @ 2006-02-27 15:39 UTC (permalink / raw)
  To: Mark Williamson; +Cc: xen-devel, Horms

> When I was looking at this approach I was intending to retreive the pageframe 
> info from Xen and just run unpin on anything pinned, according to its type.

Is that possible?  As far I know the frametable where that info is
stored isn't readable by domains ...

> A really big chunk of the generic kexec code is there to make sure the new 
> kernel is not loaded in an area that conflicts with its destination, much of 
> the trampoline is there to enable the new kernel to be copied on top of the 
> old one.  Address conflicts should only be an issue if you want the second 
> kernel to be a dump kernel, in which case it needs to run in-place after 
> loading.

Yep, sure, all the trampoline stuff is effectively for that.
Retested, works.  Not sure what I did wrong last time ...

cheers,

  Gerd

-- 
Gerd 'just married' Hoffmann <kraxel@suse.de>
I'm the hacker formerly known as Gerd Knorr.
http://www.suse.de/~kraxel/just-married.jpeg

^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2006-02-27 15:39 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2005-07-07 17:16 [PATCH] Xen Guest Kexec Mark Williamson
2005-07-08  8:59 ` [Fastboot] " Eric W. Biederman
2005-07-08 14:35   ` Mark Williamson
2005-07-08 17:35 ` Mark Williamson
2006-02-23 10:52 ` Horms
2006-02-23 11:22   ` Gerd Hoffmann
2006-02-23 11:32     ` Mark Williamson
2006-02-23 13:03       ` Gerd Hoffmann
2006-02-23 14:49         ` Mark Williamson
2006-02-24 14:35           ` Gerd Hoffmann
2006-02-27 14:29             ` Mark Williamson
2006-02-27 15:39               ` Gerd Hoffmann
2006-02-23 11:36   ` Mark Williamson
2006-02-27  3:32     ` Horms
2006-02-27 14:58       ` Mark Williamson

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.