* [PATCH] Xen Guest Kexec
@ 2005-07-07 17:16 Mark Williamson
2005-07-08 8:59 ` [Fastboot] " Eric W. Biederman
` (2 more replies)
0 siblings, 3 replies; 15+ messages in thread
From: Mark Williamson @ 2005-07-07 17:16 UTC (permalink / raw)
To: xen-devel, fastboot
All,
I'm posting to the Xen-devel list and the OSDL fastboot list. There are a
number of Xen folks who'll want to look at the code. For the fastboot folks,
this is mostly intended as an announcement of the port. Please limit your
replies to the list(s) that they applies to, thanks :-)
A number of people have expressed interest in kexec support for Xen guests.
In particular, it's mainly useful for purposes of implementing an in-guest
bootloader app and avoiding the need for dom0 to access the guest filesystem.
It's a largish patch, so I stuck it online:
http://www.cl.cam.ac.uk/~maw48/xenguest_kexec.patch
Tested on i386. I suspect it'll have build issues on x86_64 but those should
be easy to fix - actually the code is probably generic enough to work on
both.
You'll also need a modified version of the kexec tools, which I'll post
details of later.
Notes on the implementation:
Kexec in a Xen guest doesn't work like on a real machine. The guest relies on
outside assistance to complete the job. The guest communicates details of
the kexec to Xend. Xend extracts the data from the guest's memory image and
uses it to rebuild the domain with a new kernel, etc. This approach
simplifies the changes needed to support kexec and minimises code duplication
between Linux and the domain builder itself.
Kexec-ing the whole host is a separate issue, I think it's best addressed with
a port of kexec to Xen itself.
Cheers,
Mark
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Fastboot] [PATCH] Xen Guest Kexec
2005-07-07 17:16 [PATCH] Xen Guest Kexec Mark Williamson
@ 2005-07-08 8:59 ` Eric W. Biederman
2005-07-08 14:35 ` Mark Williamson
2005-07-08 17:35 ` Mark Williamson
2006-02-23 10:52 ` Horms
2 siblings, 1 reply; 15+ messages in thread
From: Eric W. Biederman @ 2005-07-08 8:59 UTC (permalink / raw)
To: Mark Williamson; +Cc: xen-devel, fastboot
Mark Williamson <mark.williamson@cl.cam.ac.uk> writes:
> All,
>
> I'm posting to the Xen-devel list and the OSDL fastboot list. There are a
> number of Xen folks who'll want to look at the code. For the fastboot folks,
> this is mostly intended as an announcement of the port. Please limit your
> replies to the list(s) that they applies to, thanks :-)
>
> A number of people have expressed interest in kexec support for Xen guests.
> In particular, it's mainly useful for purposes of implementing an in-guest
> bootloader app and avoiding the need for dom0 to access the guest filesystem.
Is kexec at all useful for generating the initial image as well?
I believe all it would require would be defining an extra kexec type,
to load an image into the new domain.
> It's a largish patch, so I stuck it online:
> http://www.cl.cam.ac.uk/~maw48/xenguest_kexec.patch
> Tested on i386. I suspect it'll have build issues on x86_64 but those should
> be easy to fix - actually the code is probably generic enough to work on
> both.
Part of that patch is that you have another patch file showing up
in your diff.
> You'll also need a modified version of the kexec tools, which I'll post
> details of later.
>
> Notes on the implementation:
> Kexec in a Xen guest doesn't work like on a real machine. The guest relies on
> outside assistance to complete the job. The guest communicates details of
> the kexec to Xend. Xend extracts the data from the guest's memory image and
> uses it to rebuild the domain with a new kernel, etc. This approach
> simplifies the changes needed to support kexec and minimises code duplication
> between Linux and the domain builder itself.
How useful will you your kexec implementation handle the kexec on panic
case? In that case we kexec a kernel at an alternative address and
then read from the memory what the previous kernel had left in it's
physical memory, and generate a crash dump of it.
> Kexec-ing the whole host is a separate issue, I think it's best addressed with
> a port of kexec to Xen itself.
Sounds fun. :)
Eric
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Fastboot] [PATCH] Xen Guest Kexec
2005-07-08 8:59 ` [Fastboot] " Eric W. Biederman
@ 2005-07-08 14:35 ` Mark Williamson
0 siblings, 0 replies; 15+ messages in thread
From: Mark Williamson @ 2005-07-08 14:35 UTC (permalink / raw)
To: Eric W. Biederman; +Cc: xen-devel, fastboot
> > A number of people have expressed interest in kexec support for Xen
> > guests. In particular, it's mainly useful for purposes of implementing an
> > in-guest bootloader app and avoiding the need for dom0 to access the
> > guest filesystem.
>
> Is kexec at all useful for generating the initial image as well?
> I believe all it would require would be defining an extra kexec type,
> to load an image into the new domain.
Actually, the mechanism is already practically the same: we already have a
domain builder that could stick the kernel into memory and build a set of
bootstrap page tables before kicking the domain into life. Kexec support
hooks into that by causing the domain builder to be invoked with the new
kernel, etc.
> > It's a largish patch, so I stuck it online:
> > http://www.cl.cam.ac.uk/~maw48/xenguest_kexec.patch
> > Tested on i386. I suspect it'll have build issues on x86_64 but those
> > should be easy to fix - actually the code is probably generic enough to
> > work on both.
>
> Part of that patch is that you have another patch file showing up
> in your diff.
Yep. We've got a "patches" directory for things we're waiting on being pushed
upstream. I added the Linux generic / i386 kexec support into there for the
time-being. Possibly I should omit this from a "lite" version of the patch
for easy reading, since it's a bit confusing if you're just reading...
> How useful will you your kexec implementation handle the kexec on panic
> case? In that case we kexec a kernel at an alternative address and
> then read from the memory what the previous kernel had left in it's
> physical memory, and generate a crash dump of it.
I've thought a bit about this but there are various restrictions that stop Xen
from building a new image into an existing domain. Because of this, my patch
destroys the domain and builds a completely new one with the new kernel.
Obviously this precludes kdump for now...
The limitations in Xen could be fixed, however we already have a mechanism for
core-dumping a domain from the "outside", so the motivation doesn't look that
strong for now. KDump might still be useful to people who want their
standard distribution support for dumping to "just work", so it might still
be worth looking into.
> > Kexec-ing the whole host is a separate issue, I think it's best addressed
> > with a port of kexec to Xen itself.
>
> Sounds fun. :)
Indeed :-) kdump could be more useful here for debugging whole-host issues.
The more sensible way to do this would probably be to jump into a minimal
(Xen-free) Linux, just as normal kdump does, in the event that either Xen or
the dom0 Linux crashes.
kexec support here would be quite interesting: machine_kexec.c and
relocate_kernel.S will have to be in Xen itself. Code for shutting down PCI
devices will have to be in dom0! :-)
A neat trick would be avoid implementing the actual "loading" process in Xen -
just use a kernel that's been loaded by dom0's kernel. Then Xen only has to
contain the machine-specific code. I've designed the mechanism so that it
can be used for this in future.
Cheers,
Mark
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] Xen Guest Kexec
2005-07-07 17:16 [PATCH] Xen Guest Kexec Mark Williamson
2005-07-08 8:59 ` [Fastboot] " Eric W. Biederman
@ 2005-07-08 17:35 ` Mark Williamson
2006-02-23 10:52 ` Horms
2 siblings, 0 replies; 15+ messages in thread
From: Mark Williamson @ 2005-07-08 17:35 UTC (permalink / raw)
To: xen-devel; +Cc: fastboot
As promised, I'm following up with further patches:
This patch:
* http://www.cl.cam.ac.uk/~maw48/kexec/xenguest_kexec_tools.patch
Adds Xen guest kexec support to the kexec tools. You'll need this to actually
use the kexec support.
This patch:
* http://www.cl.cam.ac.uk/~maw48/kexec/xenguest_kexec.patch
Is the patch to the Xen distribution, including all my changes *and* the
"core" kexec changes. Applying this to a xen-unstable repository allows you
to build kexec-enabled guest kernels.
This patch:
* http://www.cl.cam.ac.uk/~maw48/kexec/xenguest_kexec_litepatch.patch
Is a reduced version of the above patch, omitting the "core" kexec changes, so
that it's easier to see what I've added.
My intention is that the changes to the Xen distribution should get merged
once we've replaced the control message interface in the control tools. It
should be there for the 3.0 release.
Eric: do you have any objections to my approach? I'd ideally like to aim for
a merge of the tools support at the same time (or before) Xen support arrives
in mainline Linux.
Cheers,
Mark
On Thursday 07 July 2005 18:16, Mark Williamson wrote:
> All,
>
> I'm posting to the Xen-devel list and the OSDL fastboot list. There are a
> number of Xen folks who'll want to look at the code. For the fastboot
> folks, this is mostly intended as an announcement of the port. Please
> limit your replies to the list(s) that they applies to, thanks :-)
>
> A number of people have expressed interest in kexec support for Xen guests.
> In particular, it's mainly useful for purposes of implementing an in-guest
> bootloader app and avoiding the need for dom0 to access the guest
> filesystem.
>
> It's a largish patch, so I stuck it online:
> http://www.cl.cam.ac.uk/~maw48/xenguest_kexec.patch
> Tested on i386. I suspect it'll have build issues on x86_64 but those
> should be easy to fix - actually the code is probably generic enough to
> work on both.
>
> You'll also need a modified version of the kexec tools, which I'll post
> details of later.
>
> Notes on the implementation:
> Kexec in a Xen guest doesn't work like on a real machine. The guest relies
> on outside assistance to complete the job. The guest communicates details
> of the kexec to Xend. Xend extracts the data from the guest's memory image
> and uses it to rebuild the domain with a new kernel, etc. This approach
> simplifies the changes needed to support kexec and minimises code
> duplication between Linux and the domain builder itself.
>
> Kexec-ing the whole host is a separate issue, I think it's best addressed
> with a port of kexec to Xen itself.
>
> Cheers,
> Mark
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] Xen Guest Kexec
2005-07-07 17:16 [PATCH] Xen Guest Kexec Mark Williamson
2005-07-08 8:59 ` [Fastboot] " Eric W. Biederman
2005-07-08 17:35 ` Mark Williamson
@ 2006-02-23 10:52 ` Horms
2006-02-23 11:22 ` Gerd Hoffmann
2006-02-23 11:36 ` Mark Williamson
2 siblings, 2 replies; 15+ messages in thread
From: Horms @ 2006-02-23 10:52 UTC (permalink / raw)
To: xen-devel
Mark Williamson <mark.williamson@cl.cam.ac.uk> wrote:
> All,
>
> I'm posting to the Xen-devel list and the OSDL fastboot list. There are a
> number of Xen folks who'll want to look at the code. For the fastboot folks,
> this is mostly intended as an announcement of the port. Please limit your
> replies to the list(s) that they applies to, thanks :-)
>
> A number of people have expressed interest in kexec support for Xen guests.
> In particular, it's mainly useful for purposes of implementing an in-guest
> bootloader app and avoiding the need for dom0 to access the guest filesystem.
>
> It's a largish patch, so I stuck it online:
> http://www.cl.cam.ac.uk/~maw48/xenguest_kexec.patch
> Tested on i386. I suspect it'll have build issues on x86_64 but those should
> be easy to fix - actually the code is probably generic enough to work on
> both.
>
> You'll also need a modified version of the kexec tools, which I'll post
> details of later.
>
> Notes on the implementation:
> Kexec in a Xen guest doesn't work like on a real machine. The guest relies on
> outside assistance to complete the job. The guest communicates details of
> the kexec to Xend. Xend extracts the data from the guest's memory image and
> uses it to rebuild the domain with a new kernel, etc. This approach
> simplifies the changes needed to support kexec and minimises code duplication
> between Linux and the domain builder itself.
>
> Kexec-ing the whole host is a separate issue, I think it's best addressed with
> a port of kexec to Xen itself.
Hi,
I have been looking into various crash-dump options, and with that in
mind, exploring kexec for Dom0. I'm wondering if the patch above is
still the latest, and if this is still the prefered approach
for kexec in Dom0.
--
Horms
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Re: [PATCH] Xen Guest Kexec
2006-02-23 10:52 ` Horms
@ 2006-02-23 11:22 ` Gerd Hoffmann
2006-02-23 11:32 ` Mark Williamson
2006-02-23 11:36 ` Mark Williamson
1 sibling, 1 reply; 15+ messages in thread
From: Gerd Hoffmann @ 2006-02-23 11:22 UTC (permalink / raw)
To: Horms; +Cc: xen-devel
> I have been looking into various crash-dump options, and with that in
> mind, exploring kexec for Dom0. I'm wondering if the patch above is
> still the latest, and if this is still the prefered approach
> for kexec in Dom0.
For crash-dumping you'll can simply ask xend to write out an dump on
domain crash, then inspect it using gdbserver-xen ;)
For kexec I'm looking into getting kexec work right now. For domU's it
shouldn't be that hard I think. dom0 likely is much harder given how
xen and dom0 work hand-in-hand to drive the hardware ...
cheers,
Gerd
--
Gerd 'just married' Hoffmann <kraxel@suse.de>
I'm the hacker formerly known as Gerd Knorr.
http://www.suse.de/~kraxel/just-married.jpeg
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Re: [PATCH] Xen Guest Kexec
2006-02-23 11:22 ` Gerd Hoffmann
@ 2006-02-23 11:32 ` Mark Williamson
2006-02-23 13:03 ` Gerd Hoffmann
0 siblings, 1 reply; 15+ messages in thread
From: Mark Williamson @ 2006-02-23 11:32 UTC (permalink / raw)
To: xen-devel; +Cc: Gerd Hoffmann, Horms
> For crash-dumping you'll can simply ask xend to write out an dump on
> domain crash, then inspect it using gdbserver-xen ;)
>
> For kexec I'm looking into getting kexec work right now. For domU's it
> shouldn't be that hard I think. dom0 likely is much harder given how
> xen and dom0 work hand-in-hand to drive the hardware ...
How are you planning to do domU kexec?
For domUs the major problem I found was:
a) the existing kexec code doesn't understand pseudophysical memory
b) we had no way (at the time) of resetting virtual devices - disconnecting
cleanly and then reconnecting again
c) creating a start-of-day environment for the reboot kernel duplicates a load
of code that's already in the domain builder
Because of this, I used a technique I called "assisted kexec" where the guest
would write out a descriptor which could be used by Xend to (safely, without
trusting the guest) extract the reboot kernel from guest memory, and rebuild
the domain with that kernel, using the standard domain builder. This worked
quite well actually resulted in less code overall.
For dom0 kexec, I thought the best approach would be to port the existing
Linux kexec machinery to Xen (should be quite straightforward - just the part
which copies the kernel image to the correct place, switches off paging,
jumps into the new kernel). Then the dom0 kexec code would make a hypercall
after closing its devices, rather than trying to execute that code itself.
What do you think?
Cheers,
Mark
--
Dave: Just a question. What use is a unicyle with no seat? And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Re: [PATCH] Xen Guest Kexec
2006-02-23 10:52 ` Horms
2006-02-23 11:22 ` Gerd Hoffmann
@ 2006-02-23 11:36 ` Mark Williamson
2006-02-27 3:32 ` Horms
1 sibling, 1 reply; 15+ messages in thread
From: Mark Williamson @ 2006-02-23 11:36 UTC (permalink / raw)
To: xen-devel; +Cc: Gerd Hoffmann, Horms
> I have been looking into various crash-dump options, and with that in
> mind, exploring kexec for Dom0. I'm wondering if the patch above is
> still the latest, and if this is still the prefered approach
> for kexec in Dom0.
My patch was out-of-tree so it's rotted. I've been waiting for various other
things to get sorted before it's worth updating it again (updating to a
kernel that actually has the kexec system call, one or two other patches).
The patch wouldn't allow kexec for dom0 anyhow. Enabling kdump for dom0 would
be useful. I think there was somebody considering looking at this in the
future...
Cheers,
Mark
--
Dave: Just a question. What use is a unicyle with no seat? And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Re: [PATCH] Xen Guest Kexec
2006-02-23 11:32 ` Mark Williamson
@ 2006-02-23 13:03 ` Gerd Hoffmann
2006-02-23 14:49 ` Mark Williamson
0 siblings, 1 reply; 15+ messages in thread
From: Gerd Hoffmann @ 2006-02-23 13:03 UTC (permalink / raw)
To: Mark Williamson; +Cc: xen-devel, Horms
Mark Williamson wrote:
> How are you planning to do domU kexec?
As close as possible to normal i386 kexec ...
> For domUs the major problem I found was:
> a) the existing kexec code doesn't understand pseudophysical memory
> b) we had no way (at the time) of resetting virtual devices - disconnecting
> cleanly and then reconnecting again
> c) creating a start-of-day environment for the reboot kernel duplicates a load
> of code that's already in the domain builder
Dis- and reconnecting should be ok by now I guess. I expect the paging
setup being the most tricky part: First because the pseudophysical
memory (probably not a major issue though). Second because unlike i386
kexec we'll have to run with paging enabled all the time ...
My plan is to first make xenlinux kernels work with correct ELF headers
(patches on the list last days ;), then make kexec userspace work
(unmodified if possible), last step the in-kernel stuff which performs
the actual kexec ...
> For dom0 kexec, I thought the best approach would be to port the existing
> Linux kexec machinery to Xen (should be quite straightforward - just the part
> which copies the kernel image to the correct place, switches off paging,
> jumps into the new kernel). Then the dom0 kexec code would make a hypercall
> after closing its devices, rather than trying to execute that code itself.
Right now linux kexec depends on the new kernel having a different
physical (and virtual) start address, so taking the very same approach
likely doesn't work.
cheers,
Gerd
--
Gerd 'just married' Hoffmann <kraxel@suse.de>
I'm the hacker formerly known as Gerd Knorr.
http://www.suse.de/~kraxel/just-married.jpeg
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Re: [PATCH] Xen Guest Kexec
2006-02-23 13:03 ` Gerd Hoffmann
@ 2006-02-23 14:49 ` Mark Williamson
2006-02-24 14:35 ` Gerd Hoffmann
0 siblings, 1 reply; 15+ messages in thread
From: Mark Williamson @ 2006-02-23 14:49 UTC (permalink / raw)
To: Gerd Hoffmann; +Cc: xen-devel, Horms
> As close as possible to normal i386 kexec ...
>
> Dis- and reconnecting should be ok by now I guess. I expect the paging
> setup being the most tricky part: First because the pseudophysical
> memory (probably not a major issue though). Second because unlike i386
> kexec we'll have to run with paging enabled all the time ...
I doubt having paging enabled would be too painful. i386 kexec disables
paging right at the end of the process so that the new kernel will have a
sensible start-of-day. We'd just need start-of-day to contain bootstrap
pagetables, same as for normal Linux. Ideally you'd need to find a slot in
the bootstrap tables for the trampoline code to live, if you take that
approach.
You've got a load of other things to worry about in this approach, like
un-type-pinning all pages you own, etc.
The generic kexec code doesn't understand phys vs machine memory, IIRC, so you
may need to worry about it mis-allocating your trampoline page (this is an
issue because you need to identity map the trampoline page later on in the
process).
> My plan is to first make xenlinux kernels work with correct ELF headers
> (patches on the list last days ;), then make kexec userspace work
> (unmodified if possible), last step the in-kernel stuff which performs
> the actual kexec ...
I patched kexec userspace, but partly that was so that it wouldn't complain
about the Xen-format image files. I also seem to remember I built a
descriptor that described the size of the kernel, ramdisk, etc for the
benefit of the kexec code but this probably wasn't strictly necessary. I
might even have removed that in the end... Can't remember :-)
> > For dom0 kexec, I thought the best approach would be to port the existing
> > Linux kexec machinery to Xen (should be quite straightforward - just the
> > part which copies the kernel image to the correct place, switches off
> > paging, jumps into the new kernel). Then the dom0 kexec code would make
> > a hypercall after closing its devices, rather than trying to execute that
> > code itself.
>
> Right now linux kexec depends on the new kernel having a different
> physical (and virtual) start address, so taking the very same approach
> likely doesn't work.
I'm not convinced: the reboot kernel doesn't need to be any different from the
standard kernel *unless* you're running kdump (when the kernel will need to
live in a different place so that it doesn't stomp on the main kernel - not a
limitation of kexec). Or am I misunderstanding what you meant?
Anyhow, it'd be nice to have kexec working. But I'd still suggest that you
take a quick look at my patch and consider implementing it as a platform
service, rather than implementing all the low-level gunk. It really is much
less code that way.
Cheers,
Mark
--
Dave: Just a question. What use is a unicyle with no seat? And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Re: [PATCH] Xen Guest Kexec
2006-02-23 14:49 ` Mark Williamson
@ 2006-02-24 14:35 ` Gerd Hoffmann
2006-02-27 14:29 ` Mark Williamson
0 siblings, 1 reply; 15+ messages in thread
From: Gerd Hoffmann @ 2006-02-24 14:35 UTC (permalink / raw)
To: Mark Williamson; +Cc: xen-devel, Horms
Mark Williamson wrote:
>> As close as possible to normal i386 kexec ...
>>
>> Dis- and reconnecting should be ok by now I guess. I expect the paging
>> setup being the most tricky part: First because the pseudophysical
>> memory (probably not a major issue though). Second because unlike i386
>> kexec we'll have to run with paging enabled all the time ...
>
> I doubt having paging enabled would be too painful. i386 kexec disables
> paging right at the end of the process so that the new kernel will have a
> sensible start-of-day. We'd just need start-of-day to contain bootstrap
> pagetables, same as for normal Linux. Ideally you'd need to find a slot in
> the bootstrap tables for the trampoline code to live, if you take that
> approach.
x86-64 has paging enabled while trampoline is running too, so I can get
some ideas there ;)
> You've got a load of other things to worry about in this approach, like
> un-type-pinning all pages you own, etc.
Yep, that is a problem. What pages are pinned (other than pgd)? I've
seen plenty of pages with PG_pinned set, but can't figure easily what
pages that are ...
Also switching page tables seems to be not so easy. Is it possible to
switch atomically to a new, completely independant page table tree?
i.e. old tree is valid (of cource), new tree is too, but the pages of
the old tree are not mapped read-only in the new tree (and visa versa).
> The generic kexec code doesn't understand phys vs machine memory, IIRC, so you
> may need to worry about it mis-allocating your trampoline page (this is an
> issue because you need to identity map the trampoline page later on in the
> process).
Not a big issue if paging is enabled anyway, we can use a identity map
then (virtual == physical, not virtual == machine), so kexec doesn't
even notice outside the arch-specific code.
>> Right now linux kexec depends on the new kernel having a different
>> physical (and virtual) start address, so taking the very same approach
>> likely doesn't work.
>
> I'm not convinced: the reboot kernel doesn't need to be any different from the
> standard kernel *unless* you're running kdump (when the kernel will need to
> live in a different place so that it doesn't stomp on the main kernel - not a
> limitation of kexec). Or am I misunderstanding what you meant?
I think it's needed in both cases, at least I had problems making normal
kexec work (without xen) when both kernels had the same physical
address. Might have been a simple bug though.
cheers,
Gerd
--
Gerd 'just married' Hoffmann <kraxel@suse.de>
I'm the hacker formerly known as Gerd Knorr.
http://www.suse.de/~kraxel/just-married.jpeg
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Re: [PATCH] Xen Guest Kexec
2006-02-23 11:36 ` Mark Williamson
@ 2006-02-27 3:32 ` Horms
2006-02-27 14:58 ` Mark Williamson
0 siblings, 1 reply; 15+ messages in thread
From: Horms @ 2006-02-27 3:32 UTC (permalink / raw)
To: Mark Williamson; +Cc: Gerd Hoffmann, xen-devel
On Thu, Feb 23, 2006 at 11:36:36AM +0000, Mark Williamson wrote:
> > I have been looking into various crash-dump options, and with that in
> > mind, exploring kexec for Dom0. I'm wondering if the patch above is
> > still the latest, and if this is still the prefered approach
> > for kexec in Dom0.
>
> My patch was out-of-tree so it's rotted. I've been waiting for various other
> things to get sorted before it's worth updating it again (updating to a
> kernel that actually has the kexec system call, one or two other patches).
Could you be a little more specific about that. I am pretty interested
in getting this working. As far as I understand Xen is now running
on 2.6.16-rc4 which does have kexec. I'm happy to put some cycles into
updating the patch, though I must confess I am quite interested in
Gerd's approach which is closer to what I was originally thinking.
> The patch wouldn't allow kexec for dom0 anyhow. Enabling kdump for dom0 would
> be useful. I think there was somebody considering looking at this in the
> future...
Understood. I was only talking of DomU with regards to the assisted
kexec patch that you sent earlier. I am also intersted in getting
kexec working in Dom0, however I agree with Gerd's statements elsewhere
in this thread that it can be acchived by porting kexec to Xen (or as
I understand it given the current state of kexec, teaching kexec a few
things about Xen in the prevailing architecture).
I'm actually quite interested in getting both kexec and kdump working
on both DomU and Dom0. I understand that for the most port there are
really four different problems there. I'd really like to help out
with existing efforts to help get all or any of them working.
For now I'm going to poke around with Gerd's idea of asking xend to
do system dumps for crash analysis of DomU, as that seems to be an area
of little activity at this time.
--
Horms
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Re: [PATCH] Xen Guest Kexec
2006-02-24 14:35 ` Gerd Hoffmann
@ 2006-02-27 14:29 ` Mark Williamson
2006-02-27 15:39 ` Gerd Hoffmann
0 siblings, 1 reply; 15+ messages in thread
From: Mark Williamson @ 2006-02-27 14:29 UTC (permalink / raw)
To: Gerd Hoffmann; +Cc: xen-devel, Horms
> > You've got a load of other things to worry about in this approach, like
> > un-type-pinning all pages you own, etc.
>
> Yep, that is a problem. What pages are pinned (other than pgd)? I've
> seen plenty of pages with PG_pinned set, but can't figure easily what
> pages that are ...
When I was looking at this approach I was intending to retreive the pageframe
info from Xen and just run unpin on anything pinned, according to its type.
> > The generic kexec code doesn't understand phys vs machine memory, IIRC,
> > so you may need to worry about it mis-allocating your trampoline page
> > (this is an issue because you need to identity map the trampoline page
> > later on in the process).
>
> Not a big issue if paging is enabled anyway, we can use a identity map
> then (virtual == physical, not virtual == machine), so kexec doesn't
> even notice outside the arch-specific code.
Actually I was talking rubbish there anyhow; of course if you don't disable
paging the identity mapping isn't a problem.
> >> Right now linux kexec depends on the new kernel having a different
> >> physical (and virtual) start address, so taking the very same approach
> >> likely doesn't work.
<snip>
> I think it's needed in both cases, at least I had problems making normal
> kexec work (without xen) when both kernels had the same physical
> address. Might have been a simple bug though.
When did you try that? I've not tried it outside Xen, but it didn't look like
a requirement from my reading of the code.
A really big chunk of the generic kexec code is there to make sure the new
kernel is not loaded in an area that conflicts with its destination, much of
the trampoline is there to enable the new kernel to be copied on top of the
old one. Address conflicts should only be an issue if you want the second
kernel to be a dump kernel, in which case it needs to run in-place after
loading.
Cheers,
Mark
--
Dave: Just a question. What use is a unicyle with no seat? And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Re: [PATCH] Xen Guest Kexec
2006-02-27 3:32 ` Horms
@ 2006-02-27 14:58 ` Mark Williamson
0 siblings, 0 replies; 15+ messages in thread
From: Mark Williamson @ 2006-02-27 14:58 UTC (permalink / raw)
To: Horms; +Cc: Gerd Hoffmann, xen-devel
> > My patch was out-of-tree so it's rotted. I've been waiting for various
> > other things to get sorted before it's worth updating it again (updating
> > to a kernel that actually has the kexec system call, one or two other
> > patches).
>
> Could you be a little more specific about that. I am pretty interested
> in getting this working. As far as I understand Xen is now running
> on 2.6.16-rc4 which does have kexec. I'm happy to put some cycles into
> updating the patch, though I must confess I am quite interested in
> Gerd's approach which is closer to what I was originally thinking.
My original intention was also like Gerd's approach, I just decided it was
simpler overall to make it into a platform service - we depend on daemons in
other domains *anyhow* in order to talk to devices, make reboot work, etc.
I was waiting for the build-from-memory patches to maybe be accepted (which
would avoid updating that part of my patch) but I haven't had a chance to
look over them yet.
> Understood. I was only talking of DomU with regards to the assisted
> kexec patch that you sent earlier. I am also intersted in getting
> kexec working in Dom0, however I agree with Gerd's statements elsewhere
> in this thread that it can be acchived by porting kexec to Xen (or as
> I understand it given the current state of kexec, teaching kexec a few
> things about Xen in the prevailing architecture).
Yes. This shouldn't be so hard, except possible from the PoV of actually
allocating the memory correctly.
> I'm actually quite interested in getting both kexec and kdump working
> on both DomU and Dom0. I understand that for the most port there are
> really four different problems there. I'd really like to help out
> with existing efforts to help get all or any of them working.
>
> For now I'm going to poke around with Gerd's idea of asking xend to
> do system dumps for crash analysis of DomU, as that seems to be an area
> of little activity at this time.
That should be pretty much as good as kdump, except that the image obviously
won't appear within the guests filesystem, and it won't integrate with domU
distros that want to use kdump.
Cheers,
Mark
--
Dave: Just a question. What use is a unicyle with no seat? And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Re: [PATCH] Xen Guest Kexec
2006-02-27 14:29 ` Mark Williamson
@ 2006-02-27 15:39 ` Gerd Hoffmann
0 siblings, 0 replies; 15+ messages in thread
From: Gerd Hoffmann @ 2006-02-27 15:39 UTC (permalink / raw)
To: Mark Williamson; +Cc: xen-devel, Horms
> When I was looking at this approach I was intending to retreive the pageframe
> info from Xen and just run unpin on anything pinned, according to its type.
Is that possible? As far I know the frametable where that info is
stored isn't readable by domains ...
> A really big chunk of the generic kexec code is there to make sure the new
> kernel is not loaded in an area that conflicts with its destination, much of
> the trampoline is there to enable the new kernel to be copied on top of the
> old one. Address conflicts should only be an issue if you want the second
> kernel to be a dump kernel, in which case it needs to run in-place after
> loading.
Yep, sure, all the trampoline stuff is effectively for that.
Retested, works. Not sure what I did wrong last time ...
cheers,
Gerd
--
Gerd 'just married' Hoffmann <kraxel@suse.de>
I'm the hacker formerly known as Gerd Knorr.
http://www.suse.de/~kraxel/just-married.jpeg
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2006-02-27 15:39 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2005-07-07 17:16 [PATCH] Xen Guest Kexec Mark Williamson
2005-07-08 8:59 ` [Fastboot] " Eric W. Biederman
2005-07-08 14:35 ` Mark Williamson
2005-07-08 17:35 ` Mark Williamson
2006-02-23 10:52 ` Horms
2006-02-23 11:22 ` Gerd Hoffmann
2006-02-23 11:32 ` Mark Williamson
2006-02-23 13:03 ` Gerd Hoffmann
2006-02-23 14:49 ` Mark Williamson
2006-02-24 14:35 ` Gerd Hoffmann
2006-02-27 14:29 ` Mark Williamson
2006-02-27 15:39 ` Gerd Hoffmann
2006-02-23 11:36 ` Mark Williamson
2006-02-27 3:32 ` Horms
2006-02-27 14:58 ` Mark Williamson
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.