Query: Kdump: Core Image ELF Format

Query: Kdump: Core Image ELF Format

[vivek goyal]

Hi,

Kdump (A kexec based crash dumping mechanism) is going to export the
kernel core image in ELF format. ELF was chosen as a format, keeping in
mind that gdb can be used for limited debugging and "Crash" can be used
for advanced debugging.

Core image ELF headers are prepared before crash and stored at a safe
place in memory. These headers are retrieved over a kexec boot and final
elf core image is prepared for analysis. 

Given the fact physical memory can be dis-contiguous, One program header
of type PT_LOAD is created for every contiguous memory chunk present in
the system. Other information like register states etc. is captured in
notes section.

Now the issue is, on i386, whether to prepare core headers in ELF32 or
ELF64 format. gdb can not analyze ELF64 core image for i386 system. I
don't know about "crash". Can "crash" support ELF64 core image file for
i386 system?

Given the limitation of analysis tools, if core headers are prepared in
ELF32 format then how to handle PAE systems? 

Any thoughts or suggestions on this?

Thanks
Vivek

Re: Query: Kdump: Core Image ELF Format

[Dave Anderson]

vivek goyal wrote:

> Hi,
>
> Kdump (A kexec based crash dumping mechanism) is going to export the
> kernel core image in ELF format. ELF was chosen as a format, keeping in
> mind that gdb can be used for limited debugging and "Crash" can be used
> for advanced debugging.
>
> Core image ELF headers are prepared before crash and stored at a safe
> place in memory. These headers are retrieved over a kexec boot and final
> elf core image is prepared for analysis.
>
> Given the fact physical memory can be dis-contiguous, One program header
> of type PT_LOAD is created for every contiguous memory chunk present in
> the system. Other information like register states etc. is captured in
> notes section.
>
> Now the issue is, on i386, whether to prepare core headers in ELF32 or
> ELF64 format. gdb can not analyze ELF64 core image for i386 system. I
> don't know about "crash". Can "crash" support ELF64 core image file for
> i386 system?
>

Not in its current state, but it can certainly be modified to do so.
The embedded gdb module never is even aware of the vmcore file.
(It is essentially executed as "gdb vmlinux").

And currently crash only expects a single PT_LOAD section, but
that's due for a change.  It's been OK for its current set of supported
processors to use sparse file space for non-existent memory,
but it's kind of a pain with ia64's 256GB holes.

The point is that adapting crash to handle whatever format
you come up with is the easy part of the whole equation.

>
> Given the limitation of analysis tools, if core headers are prepared in
> ELF32 format then how to handle PAE systems?
>

Are you asking about what would be the p_vaddr values for the higher
memory segments?   FWIW, with the single-PT_LOAD segment currently
supported by crash, there's only one p_vaddr, but in any case, crash doesn't
use it, so PAE is not a problem.

Dave Anderson


>
> Any thoughts or suggestions on this?
>
> Thanks
> Vivek

Re: Query: Kdump: Core Image ELF Format

[Eric W. Biederman]

Dave Anderson <anderson@redhat.com> writes:

> vivek goyal wrote:
> 
> > Hi,
> >
> > Kdump (A kexec based crash dumping mechanism) is going to export the
> > kernel core image in ELF format. ELF was chosen as a format, keeping in
> > mind that gdb can be used for limited debugging and "Crash" can be used
> > for advanced debugging.
> >
> > Core image ELF headers are prepared before crash and stored at a safe
> > place in memory. These headers are retrieved over a kexec boot and final
> > elf core image is prepared for analysis.
> >
> > Given the fact physical memory can be dis-contiguous, One program header
> > of type PT_LOAD is created for every contiguous memory chunk present in
> > the system. Other information like register states etc. is captured in
> > notes section.
> >
> > Now the issue is, on i386, whether to prepare core headers in ELF32 or
> > ELF64 format. gdb can not analyze ELF64 core image for i386 system. I
> > don't know about "crash". Can "crash" support ELF64 core image file for
> > i386 system?
> >
> 
> Not in its current state, but it can certainly be modified to do so.
> The embedded gdb module never is even aware of the vmcore file.
> (It is essentially executed as "gdb vmlinux").
> 
> And currently crash only expects a single PT_LOAD section, but
> that's due for a change.  It's been OK for its current set of supported
> processors to use sparse file space for non-existent memory,
> but it's kind of a pain with ia64's 256GB holes.

Weird.  A sparse file.  I can almost see that but that looks like
a really bad format to transport from one system to another.
 
> The point is that adapting crash to handle whatever format
> you come up with is the easy part of the whole equation.

Good.  Then the concentration should be a simple well understood
format that we don't need to change all of the time.

> > Given the limitation of analysis tools, if core headers are prepared in
> > ELF32 format then how to handle PAE systems?
> >
> 
> Are you asking about what would be the p_vaddr values for the higher
> memory segments?   FWIW, with the single-PT_LOAD segment currently
> supported by crash, there's only one p_vaddr, but in any case, crash doesn't
> use it, so PAE is not a problem.

PAE (physical address extension) are 64bit addresses on a 32bit box.  So
vivek real question was where do we put the bytes.

Do I understand it correctly that crash currently just gets raw
memory data and the register state from the core file?  Then it
figures out the virtual to physical mapping by looking at vmlinux?

Eric

Re: Query: Kdump: Core Image ELF Format

[Dave Anderson]

"Eric W. Biederman" wrote:

> Dave Anderson <anderson@redhat.com> writes:
>
> > vivek goyal wrote:
> >
> > > Hi,
> > >
> > > Kdump (A kexec based crash dumping mechanism) is going to export the
> > > kernel core image in ELF format. ELF was chosen as a format, keeping in
> > > mind that gdb can be used for limited debugging and "Crash" can be used
> > > for advanced debugging.
> > >
> > > Core image ELF headers are prepared before crash and stored at a safe
> > > place in memory. These headers are retrieved over a kexec boot and final
> > > elf core image is prepared for analysis.
> > >
> > > Given the fact physical memory can be dis-contiguous, One program header
> > > of type PT_LOAD is created for every contiguous memory chunk present in
> > > the system. Other information like register states etc. is captured in
> > > notes section.
> > >
> > > Now the issue is, on i386, whether to prepare core headers in ELF32 or
> > > ELF64 format. gdb can not analyze ELF64 core image for i386 system. I
> > > don't know about "crash". Can "crash" support ELF64 core image file for
> > > i386 system?
> > >
> >
> > Not in its current state, but it can certainly be modified to do so.
> > The embedded gdb module never is even aware of the vmcore file.
> > (It is essentially executed as "gdb vmlinux").
> >
> > And currently crash only expects a single PT_LOAD section, but
> > that's due for a change.  It's been OK for its current set of supported
> > processors to use sparse file space for non-existent memory,
> > but it's kind of a pain with ia64's 256GB holes.
>
> Weird.  A sparse file.  I can almost see that but that looks like
> a really bad format to transport from one system to another.
>

Exactly.  Although the -S tar flag helps, and when uncompressed
it actually makes a smaller file on the receiving end because of
"real" zero-filled pages.

>
> > The point is that adapting crash to handle whatever format
> > you come up with is the easy part of the whole equation.
>
> Good.  Then the concentration should be a simple well understood
> format that we don't need to change all of the time.
>
> > > Given the limitation of analysis tools, if core headers are prepared in
> > > ELF32 format then how to handle PAE systems?
> > >
> >
> > Are you asking about what would be the p_vaddr values for the higher
> > memory segments?   FWIW, with the single-PT_LOAD segment currently
> > supported by crash, there's only one p_vaddr, but in any case, crash doesn't
> > use it, so PAE is not a problem.
>
> PAE (physical address extension) are 64bit addresses on a 32bit box.  So
> vivek real question was where do we put the bytes.

> Do I understand it correctly that crash currently just gets raw
> memory data and the register state from the core file?  Then it
> figures out the virtual to physical mapping by looking at vmlinux?
>
> Eric

Pretty much...

The register set is there, but it's been primarily used to figure out
the panic task from the kernel stack address there, although in later
dump versions, the NT_TASKSTRUCT notes section gives us the same thing.
Because of the various dump formats it's supported over time, yes, it
tries to glean the vast majority of the information it needs by using
the vmlinux file and raw data, and not rely on stuff in the various
different header formats.

In any case, I totally agree with your aims:

> What I aimed at was a simple picture of memory decorated with the
> register state.  We should be able to derive everything beyond that.
> And the fact that it is all in user space should make it straight
> forward to change if needed.
>

Dave Anderson

Re: Query: Kdump: Core Image ELF Format

[Eric W. Biederman]

vivek goyal <vgoyal@in.ibm.com> writes:

> Hi,
> 
> Kdump (A kexec based crash dumping mechanism) is going to export the
> kernel core image in ELF format. ELF was chosen as a format, keeping in
> mind that gdb can be used for limited debugging and "Crash" can be used
> for advanced debugging.

When I suggested ELF for this purpose it was not so much that it was
directly usable.  But rather it was an existing file format that could
do the job, was well understood, and had enough extensibility
through the PT_NOTES segment to handle the weird cases.

> Core image ELF headers are prepared before crash and stored at a safe
> place in memory. These headers are retrieved over a kexec boot and final
> elf core image is prepared for analysis. 
> 
> Given the fact physical memory can be dis-contiguous, One program header
> of type PT_LOAD is created for every contiguous memory chunk present in
> the system. Other information like register states etc. is captured in
> notes section.
> 
> Now the issue is, on i386, whether to prepare core headers in ELF32 or
> ELF64 format. gdb can not analyze ELF64 core image for i386 system. I
> don't know about "crash". Can "crash" support ELF64 core image file for
> i386 system?
> 
> Given the limitation of analysis tools, if core headers are prepared in
> ELF32 format then how to handle PAE systems? 
> 
> Any thoughts or suggestions on this?

Generate it ELF64.  We also have the problem that the kernels virtual
addresses are not used in the core dump either.   Which a post-processing
tool will also have to address as well. 

What I aimed at was a simple picture of memory decorated with the
register state.  We should be able to derive everything beyond that.
And the fact that it is all in user space should make it straight
forward to change if needed.

Eric

Re: Query: Kdump: Core Image ELF Format

[Vivek Goyal]

On Tue, 2005-03-08 at 11:00 -0700, Eric W. Biederman wrote: 
> vivek goyal <vgoyal@in.ibm.com> writes:
> 
> > Hi,
> > 
> > Kdump (A kexec based crash dumping mechanism) is going to export the
> > kernel core image in ELF format. ELF was chosen as a format, keeping in
> > mind that gdb can be used for limited debugging and "Crash" can be used
> > for advanced debugging.
> 
> When I suggested ELF for this purpose it was not so much that it was
> directly usable.  But rather it was an existing file format that could
> do the job, was well understood, and had enough extensibility
> through the PT_NOTES segment to handle the weird cases.
> 
> > Core image ELF headers are prepared before crash and stored at a safe
> > place in memory. These headers are retrieved over a kexec boot and final
> > elf core image is prepared for analysis. 
> > 
> > Given the fact physical memory can be dis-contiguous, One program header
> > of type PT_LOAD is created for every contiguous memory chunk present in
> > the system. Other information like register states etc. is captured in
> > notes section.
> > 
> > Now the issue is, on i386, whether to prepare core headers in ELF32 or
> > ELF64 format. gdb can not analyze ELF64 core image for i386 system. I
> > don't know about "crash". Can "crash" support ELF64 core image file for
> > i386 system?
> > 
> > Given the limitation of analysis tools, if core headers are prepared in
> > ELF32 format then how to handle PAE systems? 
> > 
> > Any thoughts or suggestions on this?
> 
> Generate it ELF64.  We also have the problem that the kernels virtual
> addresses are not used in the core dump either.   Which a post-processing
> tool will also have to address as well. 

That sounds good. But we loose the advantage of doing limited debugging
with gdb. Crash (or other analysis tools) will still take considerable
amount of time before before they are fully ready and tested.

How about giving user the flexibility to choose. What I mean is
introducing a command line option in kexec-tools to choose between ELF32
and ELF64 headers. For the users who are not using PAE systems, they can
very well go with ELF32 headers and do the debugging using gdb.

This also requires, setting the kernel virtual addresses while preparing
the headers. KVA for linearly mapped region is known in advance and can
be filled at header creation time and gdb can directly operate upon this
region.

> 
> What I aim on at was a simple picture of memory decorated with the
> register state.  We should be able to derive everything beyond that.
> And the fact that it is all in user space should make it straight
> forward to change if needed.
> 
> Eric
>  

Re: [Fastboot] Re: Query: Kdump: Core Image ELF Format

[Eric W. Biederman]

Vivek Goyal <vgoyal@in.ibm.com> writes:

> On Tue, 2005-03-08 at 11:00 -0700, Eric W. Biederman wrote: 
> That sounds good. But we loose the advantage of doing limited debugging
> with gdb. Crash (or other analysis tools) will still take considerable
> amount of time before before they are fully ready and tested.
> 
> How about giving user the flexibility to choose. What I mean is
> introducing a command line option in kexec-tools to choose between ELF32
> and ELF64 headers. For the users who are not using PAE systems, they can
> very well go with ELF32 headers and do the debugging using gdb.
> 
> This also requires, setting the kernel virtual addresses while preparing
> the headers. KVA for linearly mapped region is known in advance and can
> be filled at header creation time and gdb can directly operate upon this
> region.

I have no problems decorating the ELF header you are generating
in user space with virtual addresses assuming we can reliably
get that information.  And before a kernel crashes looks like a reasonable
time to ask that question.  I don't currently see where you could
derive that information.

Beyond that I prefer a little command line tool that will do the
ELF64 to ELF32 conversion and possibly add in the kva mapping to
make the core dump usable with gdb.  Doing it in a separate tool
means it is the developer who is doing the analysis who cares
not the user who is capturing the system core dump.

But I do agree that it a use case worth solving.

Eric

Re: [Fastboot] Re: Query: Kdump: Core Image ELF Format

[Dipankar Sarma]

On Wed, Mar 09, 2005 at 07:17:49AM -0700, Eric W. Biederman wrote:
> Vivek Goyal <vgoyal@in.ibm.com> writes:
> 
> > On Tue, 2005-03-08 at 11:00 -0700, Eric W. Biederman wrote: 
> > 
> > This also requires, setting the kernel virtual addresses while preparing
> > the headers. KVA for linearly mapped region is known in advance and can
> > be filled at header creation time and gdb can directly operate upon this
> > region.
> 
> I have no problems decorating the ELF header you are generating
> in user space with virtual addresses assuming we can reliably
> get that information.  And before a kernel crashes looks like a reasonable
> time to ask that question.  I don't currently see where you could
> derive that information.
> 
> Beyond that I prefer a little command line tool that will do the
> ELF64 to ELF32 conversion and possibly add in the kva mapping to
> make the core dump usable with gdb.  Doing it in a separate tool
> means it is the developer who is doing the analysis who cares
> not the user who is capturing the system core dump.

Well, as a kernel developer, I am both :) For me, having to install
half-a-dozen different command line tools to get and analyze a crash dump
is a PITA, not to mention potential version mismatches. As someone
who would like very much to use crash dump for debugging, I would
much rather be able to force a dump and then use gdb for
a quick debug. I agree that a customer would see a different
situation. It would be nice if we can cater to both the kinds.

Thanks
Dipankar

Re: [Fastboot] Re: Query: Kdump: Core Image ELF Format

[Eric W. Biederman]

Dipankar Sarma <dipankar@in.ibm.com> writes:

> On Wed, Mar 09, 2005 at 07:17:49AM -0700, Eric W. Biederman wrote:

> > Beyond that I prefer a little command line tool that will do the
> > ELF64 to ELF32 conversion and possibly add in the kva mapping to
> > make the core dump usable with gdb.  Doing it in a separate tool
> > means it is the developer who is doing the analysis who cares
> > not the user who is capturing the system core dump.
> 
> Well, as a kernel developer, I am both :) For me, having to install
> half-a-dozen different command line tools to get and analyze a crash dump
> is a PITA, not to mention potential version mismatches. As someone
> who would like very much to use crash dump for debugging, I would
> much rather be able to force a dump and then use gdb for
> a quick debug. I agree that a customer would see a different
> situation. It would be nice if we can cater to both the kinds.

Crash dumps seem to be a when all else fails kind of solution.  Or
something to make a detailed record of what happened so information
can be logged.

I think are differences are largely a matter of emphasis.  I worry
about the end user and the whole cycle.  For that we need a fixed
simple crash dump format whit no knobs bells or whistles, that can
be given to developers and eventually supported natively by all of
the tools.

I doubt tweaking gdb so it can handle a 64bit ELF core dump even
on 32bit architectures would be very hard.  Once that is in place
the 64->32bit conversion doesn't matter.  The virtual addresses
matter a little more although I am more inclined to teach gdb
about the physical and virtual address differences of whole machine
crash dumps.

I do agree that the ability to tweak things in the short term
to work like a process that does not have the virtual/physical address
distinction is useful.  

The issue of tool versioning problems is bogus.  That is solved
by simply picking a good interface between tools and sticking
with it.  Occasionally there will be paradigm shifts (like threading)
that will cause change, but generally everything will stay the same.
In addition if tools are distributed together it does not matter,
if there are several of them because there is only one update.

Eric

Re: [Fastboot] Re: Query: Kdump: Core Image ELF Format

[Vivek Goyal]

On Wed, 2005-03-09 at 07:17 -0700, Eric W. Biederman wrote:
> Vivek Goyal <vgoyal@in.ibm.com> writes:
> 
> > On Tue, 2005-03-08 at 11:00 -0700, Eric W. Biederman wrote: 
> > That sounds good. But we loose the advantage of doing limited debugging
> > with gdb. Crash (or other analysis tools) will still take considerable
> > amount of time before before they are fully ready and tested.
> > 
> > How about giving user the flexibility to choose. What I mean is
> > introducing a command line option in kexec-tools to choose between ELF32
> > and ELF64 headers. For the users who are not using PAE systems, they can
> > very well go with ELF32 headers and do the debugging using gdb.
> > 
> > This also requires, setting the kernel virtual addresses while preparing
> > the headers. KVA for linearly mapped region is known in advance and can
> > be filled at header creation time and gdb can directly operate upon this
> > region.
> 
> I have no problems decorating the ELF header you are generating
> in user space with virtual addresses assuming we can reliably
> get that information.  And before a kernel crashes looks like a reasonable
> time to ask that question.  I don't currently see where you could
> derive that information.

I want to fill the virtual addresses of linearly mapped region. That is
physical addresses from 0 to MAXMEM (896 MB) are mapped by kernel at
virtual addresses PAGE_OFFSET to (PAGE_OFFSET + MAXMEM). Values of
PAGE_OFFSET and MAXMEM are already known and hard-coded.

I think I used the terminology kernel virtual address and that is adding
to the confusion. Kernel virtual addresses are not necessarily linearly
mapped. What I meant was kernel logical addresses whose associated
physical addresses differ only by a constant offset.

> 
> Beyond that I prefer a little command line tool that will do the
> ELF64 to ELF32 conversion and possibly add in the kva mapping to
> make the core dump usable with gdb.  Doing it in a separate tool
> means it is the developer who is doing the analysis who cares
> not the user who is capturing the system core dump.
> 
> But I do agree that it a use case worth solving.
> 
> Eric
> 

Re: [Fastboot] Re: Query: Kdump: Core Image ELF Format

[Eric W. Biederman]

Vivek Goyal <vgoyal@in.ibm.com> writes:

> I want to fill the virtual addresses of linearly mapped region. That is
> physical addresses from 0 to MAXMEM (896 MB) are mapped by kernel at
> virtual addresses PAGE_OFFSET to (PAGE_OFFSET + MAXMEM). Values of
> PAGE_OFFSET and MAXMEM are already known and hard-coded.

PAGE_OFFSET has a common value of 0xc0000000, on x86.  However
that value is by no means fixed.  The 4G/4G split changes it
as do some other patches floating around at the time.
On x86-64 I don't know how stable those kinds of offsets are.
 
> I think I used the terminology kernel virtual address and that is adding
> to the confusion. Kernel virtual addresses are not necessarily linearly
> mapped. What I meant was kernel logical addresses whose associated
> physical addresses differ only by a constant offset.

I know what you meant.  I simply meant that things don't look that
constant to me.  Especially in Linux where there are enough people
to try most of the reasonable possibilities.

I don't even think it is a bad idea.  But I do think we have a different
idea of what is constant.

Eric

Re: [Fastboot] Re: Query: Kdump: Core Image ELF Format

[Vivek Goyal]

On Wed, 2005-03-09 at 23:56 -0700, Eric W. Biederman wrote:
> Vivek Goyal <vgoyal@in.ibm.com> writes:
> 
> > I want to fill the virtual addresses of linearly mapped region. That is
> > physical addresses from 0 to MAXMEM (896 MB) are mapped by kernel at
> > virtual addresses PAGE_OFFSET to (PAGE_OFFSET + MAXMEM). Values of
> > PAGE_OFFSET and MAXMEM are already known and hard-coded.
> 
> PAGE_OFFSET has a common value of 0xc0000000, on x86.  However
> that value is by no means fixed.  The 4G/4G split changes it
> as do some other patches floating around at the time.
> On x86-64 I don't know how stable those kinds of offsets are.

Agreed. Then how about, exporting this information to user space.
Probably through sysfs. May be the range of linearly mapped region can
be exported. (PAGE_OFFSET to (PAGE_OFFSET + x)).

>  
> > I think I used the terminology kernel virtual address and that is adding
> > to the confusion. Kernel virtual addresses are not necessarily linearly
> > mapped. What I meant was kernel logical addresses whose associated
> > physical addresses differ only by a constant offset.
> 
> I know what you meant.  I simply meant that things don't look that
> constant to me.  Especially in Linux where there are enough people
> to try most of the reasonable possibilities.
> 
> I don't even think it is a bad idea.  But I do think we have a different
> idea of what is constant.
> 
> Eric
> 

Re: [Fastboot] Re: Query: Kdump: Core Image ELF Format

[Itsuro Oda]

Hi,

Using ELF format to construct dump information (registers, physical adress
range, and physical memory etc.) is OK. It's not bad idea.

But it is not necessary to indend to use a particular analysis tool.
Do simple.

Thanks.
-- 
Itsuro ODA <oda@valinux.co.jp>

Re: Query: Kdump: Core Image ELF Format

[Itsuro Oda]

Hi,

> Now the issue is, on i386, whether to prepare core headers in ELF32 or
> ELF64 format. gdb can not analyze ELF64 core image for i386 system. I
> don't know about "crash". Can "crash" support ELF64 core image file for
> i386 system?

Of course, It must be ELF64.

It is important to include all information. ELF core can be easily 
transfered to LKCD format. Then lcrash can be used.
(I use an own tool.)

Thanks.

On Tue, 08 Mar 2005 18:20:10 +0530
vivek goyal <vgoyal@in.ibm.com> wrote:

> Hi,
> 
> Kdump (A kexec based crash dumping mechanism) is going to export the
> kernel core image in ELF format. ELF was chosen as a format, keeping in
> mind that gdb can be used for limited debugging and "Crash" can be used
> for advanced debugging.
> 
> Core image ELF headers are prepared before crash and stored at a safe
> place in memory. These headers are retrieved over a kexec boot and final
> elf core image is prepared for analysis. 
> 
> Given the fact physical memory can be dis-contiguous, One program header
> of type PT_LOAD is created for every contiguous memory chunk present in
> the system. Other information like register states etc. is captured in
> notes section.
> 
> Now the issue is, on i386, whether to prepare core headers in ELF32 or
> ELF64 format. gdb can not analyze ELF64 core image for i386 system. I
> don't know about "crash". Can "crash" support ELF64 core image file for
> i386 system?
> 
> Given the limitation of analysis tools, if core headers are prepared in
> ELF32 format then how to handle PAE systems? 
> 
> Any thoughts or suggestions on this?
> 
> Thanks
> Vivek
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
Itsuro ODA <oda@valinux.co.jp>

Re: Query: Kdump: Core Image ELF Format

[Fernando Luis Vazquez Cao]

Hi all.

On Tue, 2005-03-08 at 18:20 +0530, vivek goyal wrote:
> Core image ELF headers are prepared before crash and stored at a safe
> place in memory. These headers are retrieved over a kexec boot and final
> elf core image is prepared for analysis. 

Regarding the preparation of the ELF headers, I think we should also
take into consideration hot-plug memory and create appropriate
mechanisms to deal with it.

Assuming that both insertion and removal of memory trigger a hotplug
event that is subsequently handled by the relevant hotplug agent(*), the
latter could be modified so that, on successful memory onlining,
additional PT_LOAD headers are created and tucked in a safe place
together with the others.

Since ELF headers are to be prepared by kexec a new option could be
added to it, so that we can call kexec from a hotplug script to carry
out the aforementioned tasks.

Any thoughts or suggestions on this?

(*) The last patches posted by Dave Hansen already support memory
onlining from an /etc/hotplug script.

Fernando

Re: Query: Kdump: Core Image ELF Format

[Vivek Goyal]

On Wed, 2005-03-23 at 13:26 +0900, Fernando Luis Vazquez Cao wrote:
> Hi all.
> 
> On Tue, 2005-03-08 at 18:20 +0530, vivek goyal wrote:
> > Core image ELF headers are prepared before crash and stored at a safe
> > place in memory. These headers are retrieved over a kexec boot and final
> > elf core image is prepared for analysis. 
> 
> Regarding the preparation of the ELF headers, I think we should also
> take into consideration hot-plug memory and create appropriate
> mechanisms to deal with it.
> 
> Assuming that both insertion and removal of memory trigger a hotplug
> event that is subsequently handled by the relevant hotplug agent(*), the
> latter could be modified so that, on successful memory onlining,
> additional PT_LOAD headers are created and tucked in a safe place
> together with the others.

I think, re-loading the panic kernel in such event should be an easier
solution. Current kexec system call interface does not allow to read
back already stored segments, which is necessary to append new PT_LOAD
headers to the existing program header list and update ELF header.

> 
> Since ELF headers are to be prepared by kexec a new option could be
> added to it, so that we can call kexec from a hotplug script to carry
> out the aforementioned tasks.
> 
> Any thoughts or suggestions on this?
> 
> (*) The last patches posted by Dave Hansen already support memory
> onlining from an /etc/hotplug script.
> 
> Fernando
> 
> 
> 

Re: Query: Kdump: Core Image ELF Format

[Eric W. Biederman]

Vivek Goyal <vgoyal@in.ibm.com> writes:

> On Wed, 2005-03-23 at 13:26 +0900, Fernando Luis Vazquez Cao wrote:
> > Hi all.
> > 
> > On Tue, 2005-03-08 at 18:20 +0530, vivek goyal wrote:
> > > Core image ELF headers are prepared before crash and stored at a safe
> > > place in memory. These headers are retrieved over a kexec boot and final
> > > elf core image is prepared for analysis. 
> > 
> > Regarding the preparation of the ELF headers, I think we should also
> > take into consideration hot-plug memory and create appropriate
> > mechanisms to deal with it.
> > 
> > Assuming that both insertion and removal of memory trigger a hotplug
> > event that is subsequently handled by the relevant hotplug agent(*), the
> > latter could be modified so that, on successful memory onlining,
> > additional PT_LOAD headers are created and tucked in a safe place
> > together with the others.
> 
> I think, re-loading the panic kernel in such event should be an easier
> solution. Current kexec system call interface does not allow to read
> back already stored segments, which is necessary to append new PT_LOAD
> headers to the existing program header list and update ELF header.

I thought that is what he was describing.

Eric

Re: [Fastboot] Re: Query: Kdump: Core Image ELF Format

[Eric W. Biederman]

Fernando Luis Vazquez Cao <fernando@intellilink.co.jp> writes:

> Hi all.
> 
> On Tue, 2005-03-08 at 18:20 +0530, vivek goyal wrote:
> > Core image ELF headers are prepared before crash and stored at a safe
> > place in memory. These headers are retrieved over a kexec boot and final
> > elf core image is prepared for analysis. 
> 
> Regarding the preparation of the ELF headers, I think we should also
> take into consideration hot-plug memory and create appropriate
> mechanisms to deal with it.
> 
> Assuming that both insertion and removal of memory trigger a hotplug
> event that is subsequently handled by the relevant hotplug agent(*), the
> latter could be modified so that, on successful memory onlining,
> additional PT_LOAD headers are created and tucked in a safe place
> together with the others.
> 
> Since ELF headers are to be prepared by kexec a new option could be
> added to it, so that we can call kexec from a hotplug script to carry
> out the aforementioned tasks.
> 
> Any thoughts or suggestions on this?

I like it.  This really shouldn't require anything new except 
a script to call /sbin/kexec.  

As long as memory hotplug is a rare event that should work
fine.  And with real hotplug it will be a rare event.  It looks
like something similar may also be needed for cpu hotplug, but
a compile time maximum on cpu may save us there.

The Xen guys idea of memory hotplug is another matter it sounds
like the want to page an OS with memory hotplug which is just
plain silly, and also unimplemented so I will cross that bridge
when I come to it.


Eric

Re: [Fastboot] Re: Query: Kdump: Core Image ELF Format

[Eric W. Biederman]

Mark Williamson <mark.williamson@cl.cam.ac.uk> writes:

> > The Xen guys idea of memory hotplug is another matter it sounds
> > like the want to page an OS with memory hotplug which is just
> > plain silly, and also unimplemented so I will cross that bridge
> > when I come to it.
> 
> The idea isn't to page the OS per se.  The guest OS is responsible for the 
> fine-grain paging of its applications in the usually way to fit within its 
> physical memory allocation.
> 
> In order to allow coarse-grained changes in physical memory allocation (e.g. I 
> want to shrink a domain by 128MB so I can run another one), XenLinux uses a 
> "balloon driver", which basically allocates a load of memory and gives it 
> back to Xen to be used elsewhere.
> 
> This is currently invoked by the administrator, although we've talked about a 
> daemon that will automatically shift memory allocations around between 
> domains based on their requirements.
> 
> A memory hotplug interface would clean up the ballooning interface somewhat 
> (rather than using pretend allocations) but would still only be activated 
> relatively infrequently.

And what I am really objecting to is xen doing memory allocation in 4KiB
chunks.  Pushing the chunk size up to 2MiB or 4MiB, or even doing
plain extents of memory like the old protected mode OS's did before
paging sounds more reasonable.  Without allowing the OS access to
large contiguous chunks of physical memory you are asking the OS to
give up significant performance tuning opportunities.

Plus with by giving the OS large pages much of the mess of needing a
virtual, logical and physical address is unnecessary and the OS can
simply have virtual and physical addresses as they do not.

In addition large chunks of memory are going to work better with
whatever memory hotplug infrastructure is implemented, than 4KiB
chunks.  As memory hotplug is either going to be memory controller
hotplug (in numa systems) or possible DIMM hotplug is extremely fault
tolerant servers.

So please simply everyone's lives and code and use large pages
in Xen.

Eric