Determine version of kernel that produced vmcore

Determine version of kernel that produced vmcore

[Bernhard Walle]

Hello,

does anybody know a _reliable_ way to determine the version the kernel
that produced a vmcore file? This means not scanning for a specific
string or something like that which can fail on random memory.

Would it make sense to add a ELF PT_NOTE section in the vmcore?

Thanks for input!



Thanks,
   Bernhard

Re: Determine version of kernel that produced vmcore

[Dan Aloni]

On Fri, Jul 06, 2007 at 03:28:14PM +0200, Bernhard Walle wrote:
> Hello,
> 
> does anybody know a _reliable_ way to determine the version the kernel
> that produced a vmcore file? This means not scanning for a specific
> string or something like that which can fail on random memory.
> 
> Would it make sense to add a ELF PT_NOTE section in the vmcore?
> 
> Thanks for input!

(I'm sorry if this E-Mail needlessly turned into a long RFC 
but I had to bring this up when I bumped into this question on 
LKML)

Actually, instead of another ELF PT_NOTE section for this 
information and other things, kdump is currently going into 
another direction, described below.

Redhat has a makedumpinfo util which they intend to use as slim
kernel-version-independent utility on kdump rootfs in order to
save /proc/vmcore in a compact manner.

Normally makedumpinfo would require access to the vmlinux file 
that the crashed kernel was originated from. However instead of 
depending on vmlinux, it is possible to generate a small textual
"CONFIGFILE" that looks like this, for example (generated using
makedumpinfo -g):

OSRELEASE=2.6.20.3
PAGESIZE=4096
SYMBOL(mem_map)=ffffffff806cf5d8
SYMBOL(init_uts_ns)=ffffffff805f8be0
SYMBOL(_stext)=ffffffff80200f20
SYMBOL(node_online_map)=ffffffff80651bc0
SYMBOL(contig_page_data)=ffffffff80600e80
SIZE(page)=56
SIZE(pglist_data)=3712
SIZE(zone)=1152
SIZE(free_area)=24
SIZE(list_head)=16
OFFSET(page.flags)=0
OFFSET(page._count)=8
OFFSET(page.mapping)=24
OFFSET(page.lru)=40
OFFSET(pglist_data.node_zones)=0
OFFSET(pglist_data.nr_zones)=3576
OFFSET(pglist_data.node_mem_map)=3584
OFFSET(pglist_data.node_start_pfn)=3600
OFFSET(pglist_data.node_spanned_pages)=3616
OFFSET(zone.free_pages)=0
OFFSET(zone.free_area)=408
OFFSET(zone.vm_stat)=872
OFFSET(zone.spanned_pages)=1064
OFFSET(free_area.free_list)=0
OFFSET(list_head.next)=0
OFFSET(list_head.prev)=8
LENGTH(zone.free_area)=11
SRCFILE(pud_t)=include/asm/page.h

It contains enough information in order to make a compact kernel
dump (makedumpinfo needs to go over the struct page arrays). As
you see, it also contains the kernel version.

However, this file needs to be passed somehow to the rootfs
of the kdump kernel. This poses a chicken-and-egg problem on 
my setup where the initramfs of the first kernel contains 
a staticlu linked version of the kexec executable along with a
kdump kernel to be loaded before mount rootfs and running 
init. 

I used this 'crazy' kernel-packed-within-a-kernel method so 
that I wouldn't need to manage the other kernel file in the
embedded distribution I'm working on. I described this method 
to Neil Horman back in OLS last week.

A patch that I am working on will make it possible to integrate
the output of 'makedumpinfo -g' into vmlinux as the final build
stage of the kernel. This information will be presented itself
as /proc/kcore.info for the first kernel throughout its entire
execution. Then inside initramfs of the first kernel, a small 
util will modify the vmlinux file of the kdump kernel before it
gets loaded so that another special file appearing as 
/proc/vmcore.info under the kdump kernel will present the same
info. 

This way, distributions will not have to manage the packaging
of that extra file, and this is also similar to the way it was 
done with /proc/config.gz.

Originally I thought perhaps makedumpinfo can be hinted in a
way of being able to access the kallsyms info of the _crashed_
kernel and from there be able to access everything else it 
needs in a version-indepedent manner. For example, SIZE(page)
could have been presented as a simple global varible somewhere
inside the kernel:

    unsigned int kdump_size_of_pglist_data = sizeof(struct pglist_data);

If you could access the kallsyms information of the crashed
kernel from within the rootfs of the kdump kernel you can have
easily reached this variable. I think all information from
the CONFIGFILE can be 'exported' in this manner. The major
downside I can think of this approach, although it would
be quite rare - certain memory corruption bugs (especially
where kallsyms data gets squashed) can make it fail. Also,
kallsyms wastes memory on memory-limited embedded setups,
so you might not want to depend on it. Perhaps EXPORT_SYMBOL
can be used instead..

Any comments?

-- 
Dan Aloni
XIV LTD, http://www.xivstorage.com
da-x (at) monatomic.org, dan (at) xiv.co.il

Re: Determine version of kernel that produced vmcore

[Bernhard Walle]

Hello,

* Dan Aloni <da-x@monatomic.org> [2007-07-06 16:58]:
> 
> Redhat has a makedumpinfo util which they intend to use as slim
> kernel-version-independent utility on kdump rootfs in order to
> save /proc/vmcore in a compact manner.

I think you mean makedumpfile, don't you?

> A patch that I am working on will make it possible to integrate
> the output of 'makedumpinfo -g' into vmlinux as the final build
> stage of the kernel. This information will be presented itself
> as /proc/kcore.info for the first kernel throughout its entire
> execution.

That sounds good. But I doubt that kernel developers like the idea of
needing another utility in the build process ...

> Then inside initramfs of the first kernel, a small 
> util will modify the vmlinux file of the kdump kernel before it
> gets loaded so that another special file appearing as 
> /proc/vmcore.info under the kdump kernel will present the same
> info. 

Where do you get the info from? If you're in the kdump initrd,
then the kdump kernel is already loaded. Do you want to attach the
info from the crashed kernel to the initrd of the kdump kernel?



Thanks,
   Bernhard

Re: Determine version of kernel that produced vmcore

[Dan Aloni]

On Mon, Jul 09, 2007 at 11:21:54AM +0200, Bernhard Walle wrote:
> Hello,
> 
> * Dan Aloni <da-x@monatomic.org> [2007-07-06 16:58]:
> > 
> > Redhat has a makedumpinfo util which they intend to use as slim
> > kernel-version-independent utility on kdump rootfs in order to
> > save /proc/vmcore in a compact manner.
> 
> I think you mean makedumpfile, don't you?

Yes, a typo, sorry.

> > A patch that I am working on will make it possible to integrate
> > the output of 'makedumpinfo -g' into vmlinux as the final build
> > stage of the kernel. This information will be presented itself
> > as /proc/kcore.info for the first kernel throughout its entire
> > execution.
> 
> That sounds good. But I doubt that kernel developers like the idea of
> needing another utility in the build process ...

I don't think it would add much complexity to build process as it 
is now, just like the other tools that transparently do post-linking 
modifications. As far as the developer is concerned, there's just
the vmlinux and/or bzImage files that get emitted at the end.

> > Then inside initramfs of the first kernel, a small 
> > util will modify the vmlinux file of the kdump kernel before it
> > gets loaded so that another special file appearing as 
> > /proc/vmcore.info under the kdump kernel will present the same
> > info. 
> 
> Where do you get the info from? If you're in the kdump initrd,
> then the kdump kernel is already loaded. Do you want to attach the
> info from the crashed kernel to the initrd of the kdump kernel?

Not exactly. Let me describe the procedure in greater detail.
Basically, it would go like this:

1. <normal bootloader boot>
2. <normal initramfs>
3. embed_configfile /proc/kcore.info /vmlinux-kdump
4. kexec -l vmlinux-kdump <....>
5. <boot continues>
6. <crash>
7. <kdump kernel boot>
8. <kdump initramfs runs>
9. makedumpfile -i /proc/vmcore.info <....>

NOTES:
** in step 3 embed_configfile modifies vmlinux-kdump in place, 
   copying /proc/kcore.info into the data section of the vmlinux-kdump 
   at certain symbol (e.g. 'char core_info[0x1000]').
** in step 9 that data section variable (e.g. core_info) which was 
   originally holds the content of /proc/kcore.info is being presented 
   as /proc/vmcore.info for the util to use.

To complete the picture, at the final build stage of vmlinux we 
would have this:

makedumpfile -g vmlinux.configfile -x vmlinuxx
embed_configfile vmlinux.configfile vmlinux
rm vmlinux.configfile

BTW I think there could be a confusion between makedumpfile's 
CONFIGFILE and the .config file, so we should pick a different 
name for it...

-- 
Dan Aloni
XIV LTD, http://www.xivstorage.com
da-x (at) monatomic.org, dan (at) xiv.co.il

Re: Determine version of kernel that produced vmcore

[Bernhard Walle]

Hi,

* Dan Aloni <da-x@monatomic.org> [2007-07-09 13:41]:
> > > A patch that I am working on will make it possible to integrate
> > > the output of 'makedumpinfo -g' into vmlinux as the final build
> > > stage of the kernel. This information will be presented itself
> > > as /proc/kcore.info for the first kernel throughout its entire
> > > execution.
> > 
> > That sounds good. But I doubt that kernel developers like the idea of
> > needing another utility in the build process ...
> 
> I don't think it would add much complexity to build process as it 
> is now, just like the other tools that transparently do post-linking 
> modifications. As far as the developer is concerned, there's just
> the vmlinux and/or bzImage files that get emitted at the end.

I didn't complain about the complexity but about the requirement of an
additional tool. However, I think it will be configurable.

> > > Then inside initramfs of the first kernel, a small 
> > > util will modify the vmlinux file of the kdump kernel before it
> > > gets loaded so that another special file appearing as 
> > > /proc/vmcore.info under the kdump kernel will present the same
> > > info. 
> > 
> > Where do you get the info from? If you're in the kdump initrd,
> > then the kdump kernel is already loaded. Do you want to attach the
> > info from the crashed kernel to the initrd of the kdump kernel?
> 
> Not exactly. Let me describe the procedure in greater detail.
> Basically, it would go like this:
> 
> 1. <normal bootloader boot>
> 2. <normal initramfs>
> 3. embed_configfile /proc/kcore.info /vmlinux-kdump
> 4. kexec -l vmlinux-kdump <....>
> 5. <boot continues>
> 6. <crash>
> 7. <kdump kernel boot>
> 8. <kdump initramfs runs>
> 9. makedumpfile -i /proc/vmcore.info <....>

I don't see why an additional tool would be needed here. kexec already
modifies symbols, so why not adding the functionality into kexec? The
advantage would be that there's no on-disk modification necessary.

Another issue: Some parameters of makedumpfile (currently the page
size) are read from the running system. If you build the configuration
file while building the kernel, you always have the .config which
contains the page size. So it would be possible to extend the
makedumpfile utility to read that parts from a .config, and that would
remove a dependency that page size of the running kernel matches the
page size of the building kernel. On IA64, that's not always the case.

> BTW I think there could be a confusion between makedumpfile's 
> CONFIGFILE and the .config file, so we should pick a different 
> name for it...

Aggreed. Your suggestion? :)


Thanks,
   Bernhard

Re: Determine version of kernel that produced vmcore

[Dan Aloni]

On Mon, Jul 09, 2007 at 10:49:38PM +0200, Bernhard Walle wrote:
> Hi,
> 
> * Dan Aloni <da-x@monatomic.org> [2007-07-09 13:41]:
> > > > A patch that I am working on will make it possible to integrate
> > > > the output of 'makedumpinfo -g' into vmlinux as the final build
> > > > stage of the kernel. This information will be presented itself
> > > > as /proc/kcore.info for the first kernel throughout its entire
> > > > execution.
> > > 
> > > That sounds good. But I doubt that kernel developers like the idea of
> > > needing another utility in the build process ...
> > 
> > I don't think it would add much complexity to build process as it 
> > is now, just like the other tools that transparently do post-linking 
> > modifications. As far as the developer is concerned, there's just
> > the vmlinux and/or bzImage files that get emitted at the end.
> 
> I didn't complain about the complexity but about the requirement of an
> additional tool. However, I think it will be configurable.

Yes, I suppose it should be.
 
> > > > Then inside initramfs of the first kernel, a small 
> > > > util will modify the vmlinux file of the kdump kernel before it
> > > > gets loaded so that another special file appearing as 
> > > > /proc/vmcore.info under the kdump kernel will present the same
> > > > info. 
> > > 
> > > Where do you get the info from? If you're in the kdump initrd,
> > > then the kdump kernel is already loaded. Do you want to attach the
> > > info from the crashed kernel to the initrd of the kdump kernel?
> > 
> > Not exactly. Let me describe the procedure in greater detail.
> > Basically, it would go like this:
> > 
> > 1. <normal bootloader boot>
> > 2. <normal initramfs>
> > 3. embed_configfile /proc/kcore.info /vmlinux-kdump
> > 4. kexec -l vmlinux-kdump <....>
> > 5. <boot continues>
> > 6. <crash>
> > 7. <kdump kernel boot>
> > 8. <kdump initramfs runs>
> > 9. makedumpfile -i /proc/vmcore.info <....>
> 
> I don't see why an additional tool would be needed here. kexec already
> modifies symbols, so why not adding the functionality into kexec? The
> advantage would be that there's no on-disk modification necessary.

Actually, 'embed_configfile /proc/kcore.info /vmlinux-kdump' makes
no on-disk modification, since it is all contained in initramfs (we
step out of initramfs in stage 5). 

BTW another option would be containing the whole CONFIGFILE as a 
vmlinux-specific ELF note, to be easily read from /proc/vmcore.
You'd still need some util like embed_configfile at build time 
in order to integrate it, and you'd also need to modify 
makedumpfile so it can use it. Actually this sounds quite better
to me. I'll try to prepare patches both for makedumpfile and the 
kernel and see how easy it is.

> Another issue: Some parameters of makedumpfile (currently the page
> size) are read from the running system. If you build the configuration
> file while building the kernel, you always have the .config which
> contains the page size. So it would be possible to extend the
> makedumpfile utility to read that parts from a .config, and that would
> remove a dependency that page size of the running kernel matches the
> page size of the building kernel. On IA64, that's not always the case.

That needs to be changed.. the part in makedumpfile that generates 
the CONFIGFILE shouldn't depend on the running kernel at all. As 
Ken'ichi suggests we don't need to have 'OSRELEASE=' in the 
CONFIGFILE (instead, peeking inside init_uts_ns from /proc/vmcore
is enough). For PAGESIZE, it can be replaced by readmem() of some
variable such as:

         unsigned int image_page_size = PAGE_SIZE;

With the system I propose for generation and delivery of the 
CONFIGFILE, it would be impossible to pass an incorrect CONFIGFILE
since the kernel will already contain its own CONFIGFILE generated
during build time (assuming it didn't get corrupted in the crash,
it's quite far-fetched...).

> > BTW I think there could be a confusion between makedumpfile's 
> > CONFIGFILE and the .config file, so we should pick a different 
> > name for it...
> 
> Aggreed. Your suggestion? :)

Ken'ichi's "mkdfinfo" works for me.

-- 
Dan Aloni
XIV LTD, http://www.xivstorage.com
da-x (at) monatomic.org, dan (at) xiv.co.il

Re: Determine version of kernel that produced vmcore

[Bernhard Walle]

Hello,

* Dan Aloni <da-x@monatomic.org> [2007-07-10 06:45]:
> > > 
> > > Not exactly. Let me describe the procedure in greater detail.
> > > Basically, it would go like this:
> > > 
> > > 1. <normal bootloader boot>
> > > 2. <normal initramfs>
> > > 3. embed_configfile /proc/kcore.info /vmlinux-kdump
> > > 4. kexec -l vmlinux-kdump <....>
> > > 5. <boot continues>
> > > 6. <crash>
> > > 7. <kdump kernel boot>
> > > 8. <kdump initramfs runs>
> > > 9. makedumpfile -i /proc/vmcore.info <....>
> > 
> > I don't see why an additional tool would be needed here. kexec already
> > modifies symbols, so why not adding the functionality into kexec? The
> > advantage would be that there's no on-disk modification necessary.
> 
> Actually, 'embed_configfile /proc/kcore.info /vmlinux-kdump' makes
> no on-disk modification, since it is all contained in initramfs (we
> step out of initramfs in stage 5). 

Well, that assumes that you load the kdump kernel in the initrd of the
normal kernel. That's not always the case. For example, in SUSE the
kdump kernel is loaded as init script in the normal boot process.
Also, e.g. when testing, you may load the kdump kernel again. So you
would always need to copy the kdump kernel, modify it with
embed_configfile and then run kexec.

I think extending kexec to perform this task is more user friendly.

> BTW another option would be containing the whole CONFIGFILE as a 
> vmlinux-specific ELF note, to be easily read from /proc/vmcore.
> You'd still need some util like embed_configfile at build time 
> in order to integrate it, and you'd also need to modify 
> makedumpfile so it can use it. Actually this sounds quite better
> to me. I'll try to prepare patches both for makedumpfile and the 
> kernel and see how easy it is.

Well, in our kernel RPMs there's also a CONFIGFILE which can be
located on disk based on the version. (If a user builds the kernel
manually, then he probably builds it with -g anyway because if he
doesn't, he cannot run makedumpfile.)

The inclusion of the CONFIGFILE can be done by kexec when loading the
kdump kernel, either from a file or from the compiled-in
/proc/kcore.info file. I don't see a reason why a tool should modify
the vmlinux file for kdump on-disk.  Actually, I like the idea of
including a ELF note that contains CONFIGFILE.

Currently, we still have another problem: On x86_64, if the kernel is
relocatable, we only can use bzImage, not vmlinux kexec because of a
GDB bug that doesn't get attention by the GDB developers although I
found out the patch that broke GDB in that way ... But: That would
require that embed_configfile must also be able to modify bzImage
which is very problematic. Adding a ELF in kexec that could be read in
the /proc/vmcore hasn't that problem.



Thanks,
   Bernhard

Re: Determine version of kernel that produced vmcore

[Vivek Goyal]

On Mon, Jul 09, 2007 at 02:41:31PM +0300, Dan Aloni wrote:
> 
> I don't think it would add much complexity to build process as it 
> is now, just like the other tools that transparently do post-linking 
> modifications. As far as the developer is concerned, there's just
> the vmlinux and/or bzImage files that get emitted at the end.
> 
> > > Then inside initramfs of the first kernel, a small 
> > > util will modify the vmlinux file of the kdump kernel before it
> > > gets loaded so that another special file appearing as 
> > > /proc/vmcore.info under the kdump kernel will present the same
> > > info. 
> > 
> > Where do you get the info from? If you're in the kdump initrd,
> > then the kdump kernel is already loaded. Do you want to attach the
> > info from the crashed kernel to the initrd of the kdump kernel?
> 
> Not exactly. Let me describe the procedure in greater detail.
> Basically, it would go like this:
> 
> 1. <normal bootloader boot>
> 2. <normal initramfs>
> 3. embed_configfile /proc/kcore.info /vmlinux-kdump
> 4. kexec -l vmlinux-kdump <....>
> 5. <boot continues>
> 6. <crash>
> 7. <kdump kernel boot>
> 8. <kdump initramfs runs>
> 9. makedumpfile -i /proc/vmcore.info <....>
> 
> NOTES:
> ** in step 3 embed_configfile modifies vmlinux-kdump in place, 
>    copying /proc/kcore.info into the data section of the vmlinux-kdump 
>    at certain symbol (e.g. 'char core_info[0x1000]').
> ** in step 9 that data section variable (e.g. core_info) which was 
>    originally holds the content of /proc/kcore.info is being presented 
>    as /proc/vmcore.info for the util to use.
> 
> To complete the picture, at the final build stage of vmlinux we 
> would have this:
> 
> makedumpfile -g vmlinux.configfile -x vmlinuxx
> embed_configfile vmlinux.configfile vmlinux
> rm vmlinux.configfile
> 

Are you planning to move makedumpfile utility in kernel source three then
for this command to run? Output of "makedumpfile -g" is pretty non-standard
and very customized for filtering. At the same time, this will only work
if first kernel was compiled with debug info. (I think makedumpfile
retrieves all the structure member offset info from .debuginfo seciton).

I am still thinking that why can't we change initrd building process
(Be it mkinitrd or mkdumprd depending on distriution). Whole idea is
that while building an initrd/initramfs for the first kernel, one will
ask user for kdump kernel (if user wishes to load kdump kenrel through
initrd) and then it will generate kdump kenrel's initrd and pack into
first kernel's initrd.

So steps would look something like this.

- mkinitrd takes second kernel's vmlinux as argument
- mkinitrd runs "makedumpfile -g" on debug version of first kernel's vmlinux.
- mkinitrd generates the initramfs for kdump kernel and packs output
  of "makedumpfile -g" into that.
- mkinitrd packs statically linked kexec, kdump kernel vmlinux/bzImage,
  and kdump kernel initramfs into first kernel's initramfs.

This will save us from doing any changes to kernel build process and
also avoid any run time updation of vmlinux code. Exporting a particular
non-standard output through proc interfaces does not seem to be a very good
idea (/proc/kcore.info or /proc/vmcore.info)

Thanks
Vivek

Re: Determine version of kernel that produced vmcore

[Neil Horman]

On Tue, Jul 10, 2007 at 08:30:37PM +0530, Vivek Goyal wrote:
> On Mon, Jul 09, 2007 at 02:41:31PM +0300, Dan Aloni wrote:
> > 
> > I don't think it would add much complexity to build process as it 
> > is now, just like the other tools that transparently do post-linking 
> > modifications. As far as the developer is concerned, there's just
> > the vmlinux and/or bzImage files that get emitted at the end.
> > 
> > > > Then inside initramfs of the first kernel, a small 
> > > > util will modify the vmlinux file of the kdump kernel before it
> > > > gets loaded so that another special file appearing as 
> > > > /proc/vmcore.info under the kdump kernel will present the same
> > > > info. 
> > > 
> > > Where do you get the info from? If you're in the kdump initrd,
> > > then the kdump kernel is already loaded. Do you want to attach the
> > > info from the crashed kernel to the initrd of the kdump kernel?
> > 
> > Not exactly. Let me describe the procedure in greater detail.
> > Basically, it would go like this:
> > 
> > 1. <normal bootloader boot>
> > 2. <normal initramfs>
> > 3. embed_configfile /proc/kcore.info /vmlinux-kdump
> > 4. kexec -l vmlinux-kdump <....>
> > 5. <boot continues>
> > 6. <crash>
> > 7. <kdump kernel boot>
> > 8. <kdump initramfs runs>
> > 9. makedumpfile -i /proc/vmcore.info <....>
> > 
> > NOTES:
> > ** in step 3 embed_configfile modifies vmlinux-kdump in place, 
> >    copying /proc/kcore.info into the data section of the vmlinux-kdump 
> >    at certain symbol (e.g. 'char core_info[0x1000]').
> > ** in step 9 that data section variable (e.g. core_info) which was 
> >    originally holds the content of /proc/kcore.info is being presented 
> >    as /proc/vmcore.info for the util to use.
> > 
> > To complete the picture, at the final build stage of vmlinux we 
> > would have this:
> > 
> > makedumpfile -g vmlinux.configfile -x vmlinuxx
> > embed_configfile vmlinux.configfile vmlinux
> > rm vmlinux.configfile
> > 
> 
> Are you planning to move makedumpfile utility in kernel source three then
> for this command to run? Output of "makedumpfile -g" is pretty non-standard
> and very customized for filtering. At the same time, this will only work
> if first kernel was compiled with debug info. (I think makedumpfile
> retrieves all the structure member offset info from .debuginfo seciton).
> 
I think that your step 3 doesn't make too much sense, unless you only run
kernels with dwarf debug information included, in which case you don't need to
embed any files, since the kdump kernel will expose all its needed symbol info
through /proc/kcore.  Currently the RHEL5 kdump service embeds the needed config
file into the kdump initramfs for use on kdump.  I don't see what added
advantage the above series of actions provides.

> I am still thinking that why can't we change initrd building process
> (Be it mkinitrd or mkdumprd depending on distriution). Whole idea is
> that while building an initrd/initramfs for the first kernel, one will
> ask user for kdump kernel (if user wishes to load kdump kenrel through
> initrd) and then it will generate kdump kenrel's initrd and pack into
> first kernel's initrd.
> 
> So steps would look something like this.
> 
> - mkinitrd takes second kernel's vmlinux as argument
> - mkinitrd runs "makedumpfile -g" on debug version of first kernel's vmlinux.
> - mkinitrd generates the initramfs for kdump kernel and packs output
>   of "makedumpfile -g" into that.
> - mkinitrd packs statically linked kexec, kdump kernel vmlinux/bzImage,
>   and kdump kernel initramfs into first kernel's initramfs.
> 
Agreed, this is exactly what happens right now.

> This will save us from doing any changes to kernel build process and
> also avoid any run time updation of vmlinux code. Exporting a particular
> non-standard output through proc interfaces does not seem to be a very good
> idea (/proc/kcore.info or /proc/vmcore.info)
> 
> Thanks
> Vivek

-- 
/***************************************************
 *Neil Horman
 *Software Engineer
 *Red Hat, Inc.
 *nhorman@redhat.com
 *gpg keyid: 1024D / 0x92A74FA1
 *http://pgp.mit.edu
 ***************************************************/

Re: Determine version of kernel that produced vmcore

[Dan Aloni]

On Tue, Jul 10, 2007 at 01:17:40PM -0400, Neil Horman wrote:
> On Tue, Jul 10, 2007 at 08:30:37PM +0530, Vivek Goyal wrote:
> > I am still thinking that why can't we change initrd building process
> > (Be it mkinitrd or mkdumprd depending on distriution). Whole idea is
> > that while building an initrd/initramfs for the first kernel, one will
> > ask user for kdump kernel (if user wishes to load kdump kenrel through
> > initrd) and then it will generate kdump kenrel's initrd and pack into
> > first kernel's initrd.
> > 
> > So steps would look something like this.
> > 
> > - mkinitrd takes second kernel's vmlinux as argument
> > - mkinitrd runs "makedumpfile -g" on debug version of first kernel's vmlinux.
> > - mkinitrd generates the initramfs for kdump kernel and packs output
> >   of "makedumpfile -g" into that.
> > - mkinitrd packs statically linked kexec, kdump kernel vmlinux/bzImage,
> >   and kdump kernel initramfs into first kernel's initramfs.
> > 
> Agreed, this is exactly what happens right now.

Isn't there some sort of a circular dependency going on here? As I 
understand it the vmlinux binary already contains the initramfs as 
built-in data (at least that's what I use here for initramfs). It 
makes more sense if you guys are creating an _initrd_ image (that's 
what mkinitrd originally did AFAIK) and supply it to the boot-loader.

-- 
Dan Aloni
XIV LTD, http://www.xivstorage.com
da-x (at) monatomic.org, dan (at) xiv.co.il

Re: Determine version of kernel that produced vmcore

[Dan Aloni]

On Tue, Jul 10, 2007 at 08:35:41PM +0300, Dan Aloni wrote:
> On Tue, Jul 10, 2007 at 01:17:40PM -0400, Neil Horman wrote:
> > On Tue, Jul 10, 2007 at 08:30:37PM +0530, Vivek Goyal wrote:
> > > I am still thinking that why can't we change initrd building process
> > > (Be it mkinitrd or mkdumprd depending on distriution). Whole idea is
> > > that while building an initrd/initramfs for the first kernel, one will
> > > ask user for kdump kernel (if user wishes to load kdump kenrel through
> > > initrd) and then it will generate kdump kenrel's initrd and pack into
> > > first kernel's initrd.
> > > 
> > > So steps would look something like this.
> > > 
> > > - mkinitrd takes second kernel's vmlinux as argument
> > > - mkinitrd runs "makedumpfile -g" on debug version of first kernel's vmlinux.
> > > - mkinitrd generates the initramfs for kdump kernel and packs output
> > >   of "makedumpfile -g" into that.
> > > - mkinitrd packs statically linked kexec, kdump kernel vmlinux/bzImage,
> > >   and kdump kernel initramfs into first kernel's initramfs.
> > > 
> > Agreed, this is exactly what happens right now.
> 
> Isn't there some sort of a circular dependency going on here? As I 
> understand it the vmlinux binary already contains the initramfs as 
> built-in data (at least that's what I use here for initramfs). It 
> makes more sense if you guys are creating an _initrd_ image (that's 
> what mkinitrd originally did AFAIK) and supply it to the boot-loader.

I see you are using an _external_ initramfs image. Sorry, I forgot 
that option existed...

-- 
Dan Aloni
XIV LTD, http://www.xivstorage.com
da-x (at) monatomic.org, dan (at) xiv.co.il

Re: Determine version of kernel that produced vmcore

[Neil Horman]

On Tue, Jul 10, 2007 at 08:35:41PM +0300, Dan Aloni wrote:
> On Tue, Jul 10, 2007 at 01:17:40PM -0400, Neil Horman wrote:
> > On Tue, Jul 10, 2007 at 08:30:37PM +0530, Vivek Goyal wrote:
> > > I am still thinking that why can't we change initrd building process
> > > (Be it mkinitrd or mkdumprd depending on distriution). Whole idea is
> > > that while building an initrd/initramfs for the first kernel, one will
> > > ask user for kdump kernel (if user wishes to load kdump kenrel through
> > > initrd) and then it will generate kdump kenrel's initrd and pack into
> > > first kernel's initrd.
> > > 
> > > So steps would look something like this.
> > > 
> > > - mkinitrd takes second kernel's vmlinux as argument
> > > - mkinitrd runs "makedumpfile -g" on debug version of first kernel's vmlinux.
> > > - mkinitrd generates the initramfs for kdump kernel and packs output
> > >   of "makedumpfile -g" into that.
> > > - mkinitrd packs statically linked kexec, kdump kernel vmlinux/bzImage,
> > >   and kdump kernel initramfs into first kernel's initramfs.
> > > 
> > Agreed, this is exactly what happens right now.
> 
> Isn't there some sort of a circular dependency going on here? As I 
> understand it the vmlinux binary already contains the initramfs as 
> built-in data (at least that's what I use here for initramfs). It 
You're misunderstood.  The vmlinux binary and the initramfs are stored in the
same protected memory area when you execute a kexec -l, but they are separate
and distinct files.

> makes more sense if you guys are creating an _initrd_ image (that's 
> what mkinitrd originally did AFAIK) and supply it to the boot-loader.
initrd is old, initramfs is the new way to go, but they effecitvley do the same
thing, and while the initramfs _can_ be built into the kernel, it can also be
separately managed (which is what most distros tend to do, AFAICS).
Neil

> 
> -- 
> Dan Aloni
> XIV LTD, http://www.xivstorage.com
> da-x (at) monatomic.org, dan (at) xiv.co.il

-- 
/***************************************************
 *Neil Horman
 *Software Engineer
 *Red Hat, Inc.
 *nhorman@redhat.com
 *gpg keyid: 1024D / 0x92A74FA1
 *http://pgp.mit.edu
 ***************************************************/

Re: Determine version of kernel that produced vmcore

[Dan Aloni]

On Tue, Jul 10, 2007 at 03:00:09PM -0400, Neil Horman wrote:
> On Tue, Jul 10, 2007 at 08:35:41PM +0300, Dan Aloni wrote:
>[...]
> > 
> > Isn't there some sort of a circular dependency going on here? As I 
> > understand it the vmlinux binary already contains the initramfs as 
> > built-in data (at least that's what I use here for initramfs). It 
> You're misunderstood.  The vmlinux binary and the initramfs are stored in the
> same protected memory area when you execute a kexec -l, but they are separate
> and distinct files.
> 
> > makes more sense if you guys are creating an _initrd_ image (that's 
> > what mkinitrd originally did AFAIK) and supply it to the boot-loader.
> initrd is old, initramfs is the new way to go, but they effecitvley do the same
> thing, and while the initramfs _can_ be built into the kernel, it can also be
> separately managed (which is what most distros tend to do, AFAICS).
> Neil

Yes, I was under the impression that the external initramfs was the 
'special case' and I seem to forget about it, I guess that's what most  
kernel distribution still do (I haven't been using a distribution-supplied 
kernel for quite a while...).

Anyway, the patches I contributed in one the previous mail (didn't reach 
the kexec list due to moderation but available on LKML) can help with 
what Bernhard is aiming for. I took quite a dive into makedumpfile's code 
to figure out what's the best way to integrate it. What I was thinking 
was something in the lines of:

 * early on, detect if /proc/vmcore has a LINUX elf note 
   * if it has, then don't require '-x' or '-i' on the command line,
     extract it to a temporary file and treat it as a CONFIGFILE for 
     the rest of the code flow.
   * otherwise, continue with the regular flow.

For just extracting OSRELEASE, it is possible to write a simple program or 
script that searches for it at the first few kilobytes of the file.

-- 
Dan Aloni
XIV LTD, http://www.xivstorage.com
da-x (at) monatomic.org, dan (at) xiv.co.il

Re: Determine version of kernel that produced vmcore

[Neil Horman]

On Tue, Jul 10, 2007 at 11:36:45PM +0300, Dan Aloni wrote:
> On Tue, Jul 10, 2007 at 03:00:09PM -0400, Neil Horman wrote:
> > On Tue, Jul 10, 2007 at 08:35:41PM +0300, Dan Aloni wrote:
> >[...]
> > > 
> > > Isn't there some sort of a circular dependency going on here? As I 
> > > understand it the vmlinux binary already contains the initramfs as 
> > > built-in data (at least that's what I use here for initramfs). It 
> > You're misunderstood.  The vmlinux binary and the initramfs are stored in the
> > same protected memory area when you execute a kexec -l, but they are separate
> > and distinct files.
> > 
> > > makes more sense if you guys are creating an _initrd_ image (that's 
> > > what mkinitrd originally did AFAIK) and supply it to the boot-loader.
> > initrd is old, initramfs is the new way to go, but they effecitvley do the same
> > thing, and while the initramfs _can_ be built into the kernel, it can also be
> > separately managed (which is what most distros tend to do, AFAICS).
> > Neil
> 
> Yes, I was under the impression that the external initramfs was the 
> 'special case' and I seem to forget about it, I guess that's what most  
> kernel distribution still do (I haven't been using a distribution-supplied 
> kernel for quite a while...).
> 
> Anyway, the patches I contributed in one the previous mail (didn't reach 
> the kexec list due to moderation but available on LKML) can help with 
> what Bernhard is aiming for. I took quite a dive into makedumpfile's code 
> to figure out what's the best way to integrate it. What I was thinking 
> was something in the lines of:
> 
>  * early on, detect if /proc/vmcore has a LINUX elf note 
>    * if it has, then don't require '-x' or '-i' on the command line,
>      extract it to a temporary file and treat it as a CONFIGFILE for 
>      the rest of the code flow.
>    * otherwise, continue with the regular flow.
> 
This would make sense to me.

> For just extracting OSRELEASE, it is possible to write a simple program or 
> script that searches for it at the first few kilobytes of the file.
> 
Also seems reasonable.

Regards
Neil

> -- 
> Dan Aloni
> XIV LTD, http://www.xivstorage.com
> da-x (at) monatomic.org, dan (at) xiv.co.il

-- 
/***************************************************
 *Neil Horman
 *Software Engineer
 *Red Hat, Inc.
 *nhorman@redhat.com
 *gpg keyid: 1024D / 0x92A74FA1
 *http://pgp.mit.edu
 ***************************************************/

Re: Determine version of kernel that produced vmcore

[Vivek Goyal]

On Fri, Jul 06, 2007 at 05:58:04PM +0300, Dan Aloni wrote:
> On Fri, Jul 06, 2007 at 03:28:14PM +0200, Bernhard Walle wrote:
> > Hello,
> > 
> > does anybody know a _reliable_ way to determine the version the kernel
> > that produced a vmcore file? This means not scanning for a specific
> > string or something like that which can fail on random memory.
> > 
> > Would it make sense to add a ELF PT_NOTE section in the vmcore?
> > 
> > Thanks for input!
> 
> (I'm sorry if this E-Mail needlessly turned into a long RFC 
> but I had to bring this up when I bumped into this question on 
> LKML)
> 
> Actually, instead of another ELF PT_NOTE section for this 
> information and other things, kdump is currently going into 
> another direction, described below.
> 
> Redhat has a makedumpinfo util which they intend to use as slim
> kernel-version-independent utility on kdump rootfs in order to
> save /proc/vmcore in a compact manner.
> 
> Normally makedumpinfo would require access to the vmlinux file 
> that the crashed kernel was originated from. However instead of 
> depending on vmlinux, it is possible to generate a small textual
> "CONFIGFILE" that looks like this, for example (generated using
> makedumpinfo -g):
> 
> OSRELEASE=2.6.20.3
> PAGESIZE=4096
> SYMBOL(mem_map)=ffffffff806cf5d8
> SYMBOL(init_uts_ns)=ffffffff805f8be0
> SYMBOL(_stext)=ffffffff80200f20
> SYMBOL(node_online_map)=ffffffff80651bc0
> SYMBOL(contig_page_data)=ffffffff80600e80
> SIZE(page)=56
> SIZE(pglist_data)=3712
> SIZE(zone)=1152
> SIZE(free_area)=24
> SIZE(list_head)=16
> OFFSET(page.flags)=0
> OFFSET(page._count)=8
> OFFSET(page.mapping)=24
> OFFSET(page.lru)=40
> OFFSET(pglist_data.node_zones)=0
> OFFSET(pglist_data.nr_zones)=3576
> OFFSET(pglist_data.node_mem_map)=3584
> OFFSET(pglist_data.node_start_pfn)=3600
> OFFSET(pglist_data.node_spanned_pages)=3616
> OFFSET(zone.free_pages)=0
> OFFSET(zone.free_area)=408
> OFFSET(zone.vm_stat)=872
> OFFSET(zone.spanned_pages)=1064
> OFFSET(free_area.free_list)=0
> OFFSET(list_head.next)=0
> OFFSET(list_head.prev)=8
> LENGTH(zone.free_area)=11
> SRCFILE(pud_t)=include/asm/page.h
> 
> It contains enough information in order to make a compact kernel
> dump (makedumpinfo needs to go over the struct page arrays). As
> you see, it also contains the kernel version.
> 

But this will not solve Bernhard's problem where looking at a vmcore
he wants to know which vmlinux (kernel version with time stamp) has
generated this vmcore. So adding a ELF NOTE should help.

> However, this file needs to be passed somehow to the rootfs
> of the kdump kernel. This poses a chicken-and-egg problem on 
> my setup where the initramfs of the first kernel contains 
> a staticlu linked version of the kexec executable along with a
> kdump kernel to be loaded before mount rootfs and running 
> init. 
> 

Why are you loading kdump kernel from first kernel's initramfs?
I guess to enable the dump capture as soon as possible so if some
driver panics() you can capture the dump?

Can't we modify the initramfs generation process (mkinitrd) to take
care of this situation?. By the way, how is second kernel's initramfs
is generated currently? The moment we start packing a file which contains
the debuginfo for first kernel, initramfs of second kernel becomes dependent
on first kernel and to me its not a good approach.

A dump kernel and its initrd should be independent of the crashing kernel.

Neil/Keni'chi, how is this taken care in current RHEL5 build? Is second
kernel's initramfs dependent on first kernel if one is doing filtering
in second kernel from initramfs?

Thanks
Vivek

Re: Determine version of kernel that produced vmcore

[Dan Aloni]

On Tue, Jul 10, 2007 at 12:18:17PM +0530, Vivek Goyal wrote:
> On Fri, Jul 06, 2007 at 05:58:04PM +0300, Dan Aloni wrote:
> > 
>[..]
> > It contains enough information in order to make a compact kernel
> > dump (makedumpinfo needs to go over the struct page arrays). As
> > you see, it also contains the kernel version.
> > 
> 
> But this will not solve Bernhard's problem where looking at a vmcore
> he wants to know which vmlinux (kernel version with time stamp) has
> generated this vmcore. So adding a ELF NOTE should help.

That, or pass it in *runtime* by other means.
 
> > However, this file needs to be passed somehow to the rootfs
> > of the kdump kernel. This poses a chicken-and-egg problem on 
> > my setup where the initramfs of the first kernel contains 
> > a staticlu linked version of the kexec executable along with a
> > kdump kernel to be loaded before mount rootfs and running 
> > init. 
> > 
> 
> Why are you loading kdump kernel from first kernel's initramfs?
> I guess to enable the dump capture as soon as possible so if some
> driver panics() you can capture the dump?

Yes, that's one of the reasons.
 
> Can't we modify the initramfs generation process (mkinitrd) to take
> care of this situation?. By the way, how is second kernel's initramfs
> is generated currently? The moment we start packing a file which contains
> the debuginfo for first kernel, initramfs of second kernel becomes dependent
> on first kernel and to me its not a good approach.

That's not what I'm aiming for, they are actually independent (my 
initramfs is a build-time constant). I have described it in another
mail.
 
> A dump kernel and its initrd should be independent of the crashing kernel.

I agree.

-- 
Dan Aloni
XIV LTD, http://www.xivstorage.com
da-x (at) monatomic.org, dan (at) xiv.co.il

Re: Determine version of kernel that produced vmcore

[Neil Horman]

On Tue, Jul 10, 2007 at 11:14:14AM +0300, Dan Aloni wrote:
> On Tue, Jul 10, 2007 at 12:18:17PM +0530, Vivek Goyal wrote:
> > On Fri, Jul 06, 2007 at 05:58:04PM +0300, Dan Aloni wrote:
> > > 
> >[..]
> > > It contains enough information in order to make a compact kernel
> > > dump (makedumpinfo needs to go over the struct page arrays). As
> > > you see, it also contains the kernel version.
> > > 
> > 
> > But this will not solve Bernhard's problem where looking at a vmcore
> > he wants to know which vmlinux (kernel version with time stamp) has
> > generated this vmcore. So adding a ELF NOTE should help.
> 
> That, or pass it in *runtime* by other means.
Wait a second,  I may have been confused before.  Do you want to know the
version of the crashing kernel when you look at a core in the crash utilty, or
do you want to know it when you are capturing the crash?  If you want the
former, then an ELF NOTE would be perfect.  If you want to know the latter, my
suggestion would be to simply modify mkdumprd to place the name of the crashing
kernel, along with its version directly into the kdump initramfs init script.
We can use makedumpfile or readelf to fish out the utsname and just embed it.
Either approach is rather easy to do I think, the former would require a kernel
modification, while the latter would require some extra scripting.

>  
> > > However, this file needs to be passed somehow to the rootfs
> > > of the kdump kernel. This poses a chicken-and-egg problem on 
> > > my setup where the initramfs of the first kernel contains 
> > > a staticlu linked version of the kexec executable along with a
> > > kdump kernel to be loaded before mount rootfs and running 
> > > init. 
> > > 
> > 
> > Why are you loading kdump kernel from first kernel's initramfs?
> > I guess to enable the dump capture as soon as possible so if some
> > driver panics() you can capture the dump?
> 
> Yes, that's one of the reasons.
>  
> > Can't we modify the initramfs generation process (mkinitrd) to take
> > care of this situation?. By the way, how is second kernel's initramfs
> > is generated currently? The moment we start packing a file which contains
> > the debuginfo for first kernel, initramfs of second kernel becomes dependent
> > on first kernel and to me its not a good approach.
> 
> That's not what I'm aiming for, they are actually independent (my 
> initramfs is a build-time constant). I have described it in another
> mail.
>  
> > A dump kernel and its initrd should be independent of the crashing kernel.
> 
> I agree.
> 
And they are.

Neil

> -- 
> Dan Aloni
> XIV LTD, http://www.xivstorage.com
> da-x (at) monatomic.org, dan (at) xiv.co.il

-- 
/***************************************************
 *Neil Horman
 *Software Engineer
 *Red Hat, Inc.
 *nhorman@redhat.com
 *gpg keyid: 1024D / 0x92A74FA1
 *http://pgp.mit.edu
 ***************************************************/

Re: Determine version of kernel that produced vmcore

[Bernhard Walle]

* Neil Horman <nhorman@redhat.com> [2007-07-10 14:12]:
> On Tue, Jul 10, 2007 at 11:14:14AM +0300, Dan Aloni wrote:
> > On Tue, Jul 10, 2007 at 12:18:17PM +0530, Vivek Goyal wrote:
> > > On Fri, Jul 06, 2007 at 05:58:04PM +0300, Dan Aloni wrote:
> > > > 
> > >[..]
> > > > It contains enough information in order to make a compact kernel
> > > > dump (makedumpinfo needs to go over the struct page arrays). As
> > > > you see, it also contains the kernel version.
> > > > 
> > > 
> > > But this will not solve Bernhard's problem where looking at a vmcore
> > > he wants to know which vmlinux (kernel version with time stamp) has
> > > generated this vmcore. So adding a ELF NOTE should help.
> > 
> > That, or pass it in *runtime* by other means.
> Wait a second,  I may have been confused before.  Do you want to know the
> version of the crashing kernel when you look at a core in the crash utilty, or
> do you want to know it when you are capturing the crash?  If you want the
> former, then an ELF NOTE would be perfect.  If you want to know the latter, my
> suggestion would be to simply modify mkdumprd to place the name of the crashing
> kernel, along with its version directly into the kdump initramfs init script.
> We can use makedumpfile or readelf to fish out the utsname and just embed it.
> Either approach is rather easy to do I think, the former would require a kernel
> modification, while the latter would require some extra scripting.

Well, mkdumprd is RedHat specific. (We use the normal mkinitrd for
this.) 

Also, I don't want to rebuild the initrd of the kdump kernel just only
after changing the running kernel. That would require to rebuild the
initrd for the kdump kernel on *every* boot of a kernel that's another
version than the last booted kernel.



Thanks,
   Bernhard

Re: Determine version of kernel that produced vmcore

[Neil Horman]

On Tue, Jul 10, 2007 at 12:18:17PM +0530, Vivek Goyal wrote:
> On Fri, Jul 06, 2007 at 05:58:04PM +0300, Dan Aloni wrote:
> > On Fri, Jul 06, 2007 at 03:28:14PM +0200, Bernhard Walle wrote:
> > > Hello,
> > > 
> > > does anybody know a _reliable_ way to determine the version the kernel
> > > that produced a vmcore file? This means not scanning for a specific
> > > string or something like that which can fail on random memory.
> > > 
> > > Would it make sense to add a ELF PT_NOTE section in the vmcore?
> > > 
> > > Thanks for input!
> > 
> > (I'm sorry if this E-Mail needlessly turned into a long RFC 
> > but I had to bring this up when I bumped into this question on 
> > LKML)
> > 
> > Actually, instead of another ELF PT_NOTE section for this 
> > information and other things, kdump is currently going into 
> > another direction, described below.
> > 
> > Redhat has a makedumpinfo util which they intend to use as slim
> > kernel-version-independent utility on kdump rootfs in order to
> > save /proc/vmcore in a compact manner.
> > 
> > Normally makedumpinfo would require access to the vmlinux file 
> > that the crashed kernel was originated from. However instead of 
> > depending on vmlinux, it is possible to generate a small textual
> > "CONFIGFILE" that looks like this, for example (generated using
> > makedumpinfo -g):
> > 
> > OSRELEASE=2.6.20.3
> > PAGESIZE=4096
> > SYMBOL(mem_map)=ffffffff806cf5d8
> > SYMBOL(init_uts_ns)=ffffffff805f8be0
> > SYMBOL(_stext)=ffffffff80200f20
> > SYMBOL(node_online_map)=ffffffff80651bc0
> > SYMBOL(contig_page_data)=ffffffff80600e80
> > SIZE(page)=56
> > SIZE(pglist_data)=3712
> > SIZE(zone)=1152
> > SIZE(free_area)=24
> > SIZE(list_head)=16
> > OFFSET(page.flags)=0
> > OFFSET(page._count)=8
> > OFFSET(page.mapping)=24
> > OFFSET(page.lru)=40
> > OFFSET(pglist_data.node_zones)=0
> > OFFSET(pglist_data.nr_zones)=3576
> > OFFSET(pglist_data.node_mem_map)=3584
> > OFFSET(pglist_data.node_start_pfn)=3600
> > OFFSET(pglist_data.node_spanned_pages)=3616
> > OFFSET(zone.free_pages)=0
> > OFFSET(zone.free_area)=408
> > OFFSET(zone.vm_stat)=872
> > OFFSET(zone.spanned_pages)=1064
> > OFFSET(free_area.free_list)=0
> > OFFSET(list_head.next)=0
> > OFFSET(list_head.prev)=8
> > LENGTH(zone.free_area)=11
> > SRCFILE(pud_t)=include/asm/page.h
> > 
> > It contains enough information in order to make a compact kernel
> > dump (makedumpinfo needs to go over the struct page arrays). As
> > you see, it also contains the kernel version.
> > 
> 
> But this will not solve Bernhard's problem where looking at a vmcore
> he wants to know which vmlinux (kernel version with time stamp) has
> generated this vmcore. So adding a ELF NOTE should help.
> 
I think an ELF note would be a fine idea.

> > However, this file needs to be passed somehow to the rootfs
> > of the kdump kernel. This poses a chicken-and-egg problem on 
> > my setup where the initramfs of the first kernel contains 
> > a staticlu linked version of the kexec executable along with a
> > kdump kernel to be loaded before mount rootfs and running 
> > init. 
> > 
> 
> Why are you loading kdump kernel from first kernel's initramfs?
> I guess to enable the dump capture as soon as possible so if some
> driver panics() you can capture the dump?
> 
> Can't we modify the initramfs generation process (mkinitrd) to take
> care of this situation?. By the way, how is second kernel's initramfs
> is generated currently? The moment we start packing a file which contains
We use a modified version of mkinitrd, called mkdumprd to generate the kdump
initramfs.

> the debuginfo for first kernel, initramfs of second kernel becomes dependent
> on first kernel and to me its not a good approach.
> 
> A dump kernel and its initrd should be independent of the crashing kernel.
> 
They are, even if you manually pack them into the first kernels initramfs.  Not
sure where you see a dependency here.

> Neil/Keni'chi, how is this taken care in current RHEL5 build? Is second
> kernel's initramfs dependent on first kernel if one is doing filtering
> in second kernel from initramfs?
> 
No not at all.  Well, not really.  If you configure kdump to use makedumpfile,
the mkdumprd script runs makedumpfile -g to generate a config file that gets
packed into the kdump initramfs for use during the dump process, which is all
makedumpfile needs to successfully filter the crashing kernel.  So we are
dependent on the first kernel during kdump service startup, since we need to
generate that config file, but after that, we're completely independent.

Thanks
Neil

> Thanks
> Vivek

-- 
/***************************************************
 *Neil Horman
 *Software Engineer
 *Red Hat, Inc.
 *nhorman@redhat.com
 *gpg keyid: 1024D / 0x92A74FA1
 *http://pgp.mit.edu
 ***************************************************/

Re: Determine version of kernel that produced vmcore

[Dan Aloni]

[-- Attachment #1: Type: text/plain, Size: 1500 bytes --]

On Tue, Jul 10, 2007 at 08:09:04AM -0400, Neil Horman wrote:
> On Tue, Jul 10, 2007 at 12:18:17PM +0530, Vivek Goyal wrote:
> > On Fri, Jul 06, 2007 at 05:58:04PM +0300, Dan Aloni wrote:
> > > On Fri, Jul 06, 2007 at 03:28:14PM +0200, Bernhard Walle wrote:
> > > > Hello,
[...]
> > > It contains enough information in order to make a compact kernel
> > > dump (makedumpinfo needs to go over the struct page arrays). As
> > > you see, it also contains the kernel version.
> > > 
> > 
> > But this will not solve Bernhard's problem where looking at a vmcore
> > he wants to know which vmlinux (kernel version with time stamp) has
> > generated this vmcore. So adding a ELF NOTE should help.
> > 
> I think an ELF note would be a fine idea.

Okay, so here's an implemenation.

See the attached proof-of-concept patches to the kernel-side kexec and 
kexec-tools (might need some cleanup though). Next to follow, a patch 
to makedumpfile. With these patches a new "LINUX" elf note generated 
by the kernel in the format that makedumpfile expects and is being 
passed on by the kexec util to the kdump kernel.

As a bonus, with this patch you don't even have to compile the kernel 
with debug information in order for the filtering to work.

As Vivek mentioned in another mail, the output of makedumpfile is not
really a standard. However, I believe we should start making standards
where no standard exists. :)

-- 
Dan Aloni
XIV LTD, http://www.xivstorage.com
da-x (at) monatomic.org, dan (at) xiv.co.il

[-- Attachment #2: kernel-kexec-mkdfinfo-note.diff --]
[-- Type: text/x-diff, Size: 4918 bytes --]

diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index 8c2c7fc..a0412ce 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -121,6 +121,8 @@ extern struct page *kimage_alloc_control_pages(struct kimage *image,
 extern void crash_kexec(struct pt_regs *);
 int kexec_should_crash(struct task_struct *);
 void crash_save_cpu(struct pt_regs *regs, int cpu);
+void crash_save_mkdfinfo(void);
+
 extern struct kimage *kexec_image;
 extern struct kimage *kexec_crash_image;
 
@@ -153,7 +155,10 @@ extern struct kimage *kexec_crash_image;
 extern struct resource crashk_res;
 typedef u32 note_buf_t[KEXEC_NOTE_BYTES/4];
 extern note_buf_t *crash_notes;
-
+extern unsigned char mkdfinfo_data[MAX_NOTE_BYTES];
+extern unsigned int mkdfinfo_size;
+extern unsigned int mkdfinfo_max_size;
+extern note_buf_t mkdfinfo_note;
 
 #else /* !CONFIG_KEXEC */
 struct pt_regs;
diff --git a/kernel/kexec.c b/kernel/kexec.c
index 25db14b..25228e3 100644
--- a/kernel/kexec.c
+++ b/kernel/kexec.c
@@ -22,6 +22,8 @@
 #include <linux/hardirq.h>
 #include <linux/elf.h>
 #include <linux/elfcore.h>
+#include <linux/utsrelease.h>
+#include <linux/utsname.h> 
 
 #include <asm/page.h>
 #include <asm/uaccess.h>
@@ -32,6 +34,12 @@
 /* Per cpu memory for storing cpu states in case of system crash. */
 note_buf_t* crash_notes;
 
+/* mkdfinfo stuff */
+unsigned char mkdfinfo_data[MAX_NOTE_BYTES];
+unsigned int mkdfinfo_size = 0;
+unsigned int mkdfinfo_max_size = sizeof(mkdfinfo_data);
+note_buf_t mkdfinfo_note;
+
 /* Location of the reserved area for the crash kernel */
 struct resource crashk_res = {
 	.name  = "Crash kernel",
@@ -1062,6 +1070,7 @@ void crash_kexec(struct pt_regs *regs)
 			struct pt_regs fixed_regs;
 			crash_setup_regs(&fixed_regs, regs);
 			machine_crash_shutdown(&fixed_regs);
+			crash_save_mkdfinfo();
 			machine_kexec(kexec_crash_image);
 		}
 		locked = xchg(&kexec_lock, 0);
@@ -1135,3 +1144,88 @@ static int __init crash_notes_memory_init(void)
 	return 0;
 }
 module_init(crash_notes_memory_init)
+
+void crash_save_mkdfinfo(void)
+{
+	u32 *buf;
+
+	if (!mkdfinfo_size)
+		return;
+
+	buf = (u32 *)mkdfinfo_note;
+	buf = append_elf_note(buf, "LINUX", 0, mkdfinfo_data,
+			      mkdfinfo_size);
+	final_note(buf);
+}
+
+void mkdfinfo_append_str(const char *fmt, ...)
+{
+	va_list args;
+	char buf[0x50];
+	int r;
+
+	va_start(args, fmt);
+	r = vsnprintf(buf, sizeof(buf), fmt, args);
+	va_end(args);
+	
+	if (r + mkdfinfo_size > mkdfinfo_max_size)
+		r = mkdfinfo_max_size - mkdfinfo_size;
+
+	memcpy(&mkdfinfo_data[mkdfinfo_size], buf, r);
+
+	mkdfinfo_size += r;
+}
+
+#define SYMBOL(name) \
+	mkdfinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)&name)
+#define SIZE(name) \
+	mkdfinfo_append_str("SIZE(%s)=%d\n", #name, sizeof(struct name))
+#define OFFSET(name, field) \
+	mkdfinfo_append_str("OFFSET(%s.%s)=%d\n", #name, #field, &(((struct name *)0)->field))
+#define LENGTH(name, field) \
+	mkdfinfo_append_str("LENGTH(%s.%s)=%d\n", #name, #field, sizeof(((struct name *)0)->field))
+
+static int __init crash_save_mkdfinfo_init(void)
+{
+	extern char _stext;
+
+	mkdfinfo_append_str("OSRELEASE=%s\n", UTS_RELEASE);
+	mkdfinfo_append_str("PAGESIZE=%d\n", PAGE_SIZE);
+
+	SYMBOL(mem_map);
+	SYMBOL(init_uts_ns);
+	SYMBOL(_stext);
+	SYMBOL(node_online_map);
+	SYMBOL(contig_page_data);
+	SIZE(page);
+	SIZE(pglist_data);
+	SIZE(zone);
+	SIZE(free_area);
+	SIZE(list_head);
+	OFFSET(page, flags);
+	OFFSET(page, _count);
+	OFFSET(page, mapping);
+	OFFSET(page, lru);
+	OFFSET(pglist_data, node_zones);
+	OFFSET(pglist_data, nr_zones);
+	OFFSET(pglist_data, node_mem_map);
+	OFFSET(pglist_data, node_start_pfn);
+	OFFSET(pglist_data, node_spanned_pages);
+	OFFSET(zone, free_area);
+	OFFSET(zone, vm_stat);
+	OFFSET(zone, spanned_pages);
+	OFFSET(free_area, free_list);
+	OFFSET(list_head, next);
+	OFFSET(list_head, prev);
+	LENGTH(zone, free_area);
+
+#ifdef __PAGETABLE_PUD_FOLDED
+	mkdfinfo_append_str("SRCFILE(pud_t)=include/asm-generic/pgtable-nopud.h\n");
+#else
+	mkdfinfo_append_str("SRCFILE(pud_t)=include/asm/page.h\n");
+#endif
+
+	return 0;
+}
+
+module_init(crash_save_mkdfinfo_init)
diff --git a/kernel/ksysfs.c b/kernel/ksysfs.c
index 559deca..70ddca2 100644
--- a/kernel/ksysfs.c
+++ b/kernel/ksysfs.c
@@ -60,6 +60,15 @@ static ssize_t kexec_crash_loaded_show(struct kset *kset, char *page)
 	return sprintf(page, "%d\n", !!kexec_crash_image);
 }
 KERNEL_ATTR_RO(kexec_crash_loaded);
+
+static ssize_t mkdfinfo_show(struct subsystem *subsys, char *page)
+{
+	return sprintf(page, "%lx %x\n", 
+		       __pa((unsigned long)(char *)&mkdfinfo_note), 
+		       MAX_NOTE_BYTES);
+}
+KERNEL_ATTR_RO(mkdfinfo);
+
 #endif /* CONFIG_KEXEC */
 
 decl_subsys(kernel, NULL, NULL);
@@ -73,6 +82,7 @@ static struct attribute * kernel_attrs[] = {
 #ifdef CONFIG_KEXEC
 	&kexec_loaded_attr.attr,
 	&kexec_crash_loaded_attr.attr,
+	&mkdfinfo_attr.attr,
 #endif
 	NULL
 };

[-- Attachment #3: kexec-tools-mkdfinfo-note.diff --]
[-- Type: text/x-diff, Size: 2849 bytes --]

diff --git a/kexec/crashdump-elf.c b/kexec/crashdump-elf.c
index c048f75..e77b9db 100644
--- a/kexec/crashdump-elf.c
+++ b/kexec/crashdump-elf.c
@@ -34,6 +34,8 @@ int FUNC(struct kexec_info *info,
 	char *bufp;
 	long int nr_cpus = 0;
 	uint64_t notes_addr, notes_len;
+	uint64_t mkdfinfo_addr, mkdfinfo_len;
+	int has_mkdfinfo = 0;
 	int (*get_note_info)(int cpu, uint64_t *addr, uint64_t *len);
 
 	if (xen_present())
@@ -45,7 +47,11 @@ int FUNC(struct kexec_info *info,
 		return -1;
 	}
 
-	sz = sizeof(EHDR) + nr_cpus * sizeof(PHDR) + ranges * sizeof(PHDR);
+	if (get_kernel_mkdfinfo( &mkdfinfo_addr, &mkdfinfo_len) == 0) {
+		has_mkdfinfo = 1;
+	}
+
+	sz = sizeof(EHDR) + (nr_cpus + has_mkdfinfo) * sizeof(PHDR) + ranges * sizeof(PHDR);
 
 	/*
 	 * Certain architectures such as x86_64 and ia64 require a separate
@@ -146,6 +152,21 @@ int FUNC(struct kexec_info *info,
 		dbgprintf_phdr("Elf header", phdr);
 	}
 
+	if (has_mkdfinfo) {
+		phdr = (PHDR *) bufp;
+		bufp += sizeof(PHDR);
+		phdr->p_type	= PT_NOTE;
+		phdr->p_flags	= 0;
+		phdr->p_offset  = phdr->p_paddr = mkdfinfo_addr;
+		phdr->p_vaddr   = 0;
+		phdr->p_filesz	= phdr->p_memsz	= mkdfinfo_len;
+		/* Do we need any alignment of segments? */
+		phdr->p_align	= 0;
+
+		(elf->e_phnum)++;
+		dbgprintf_phdr("mkde header", phdr);
+	}
+
 	/* Setup an PT_LOAD type program header for the region where
 	 * Kernel is mapped if info->kern_size is non-zero.
 	 */
diff --git a/kexec/crashdump.c b/kexec/crashdump.c
index 1c08606..7a46efa 100644
--- a/kexec/crashdump.c
+++ b/kexec/crashdump.c
@@ -108,3 +108,35 @@ int get_crash_notes_per_cpu(int cpu, uint64_t *addr, uint64_t *len)
 
 	return 0;
 }
+
+/* Returns the physical address of start of crash notes buffer for a kernel. */
+int get_kernel_mkdfinfo(uint64_t *addr, uint64_t *len)
+{
+	char kdump_info[PATH_MAX];
+	char line[MAX_LINE];
+	int count;
+	FILE *fp;
+	unsigned long long temp, temp2;
+
+	*addr = 0;
+	*len = 0;
+
+	sprintf(kdump_info, "/sys/kernel/mkdfinfo");
+	fp = fopen(kdump_info, "r");
+	if (!fp) {
+		die("Could not open \"%s\": %s\n", kdump_info,
+		    strerror(errno));
+		return -1;
+	}
+
+	if (!fgets(line, sizeof(line), fp))
+		die("Cannot parse %s: %s\n", kdump_info, strerror(errno));
+	count = sscanf(line, "%Lx %Lx", &temp, &temp2);
+	if (count != 2)
+		die("Cannot parse %s: %s\n", kdump_info, strerror(errno));
+
+	*addr = (uint64_t) temp;
+	*len = (uint64_t) temp2;
+
+	return 0;
+}
diff --git a/kexec/crashdump.h b/kexec/crashdump.h
index e99bdd2..9d31169 100644
--- a/kexec/crashdump.h
+++ b/kexec/crashdump.h
@@ -2,6 +2,7 @@
 #define CRASHDUMP_H
 
 extern int get_crash_notes_per_cpu(int cpu, uint64_t *addr, uint64_t *len);
+extern int get_kernel_mkdfinfo(uint64_t *addr, uint64_t *len);
 
 /* Need to find a better way to determine per cpu notes section size. */
 #define MAX_NOTE_BYTES		1024

Re: Determine version of kernel that produced vmcore

[Vivek Goyal]

On Tue, Jul 10, 2007 at 07:52:01PM +0300, Dan Aloni wrote:
> On Tue, Jul 10, 2007 at 08:09:04AM -0400, Neil Horman wrote:
> > On Tue, Jul 10, 2007 at 12:18:17PM +0530, Vivek Goyal wrote:
> > > On Fri, Jul 06, 2007 at 05:58:04PM +0300, Dan Aloni wrote:
> > > > On Fri, Jul 06, 2007 at 03:28:14PM +0200, Bernhard Walle wrote:
> > > > > Hello,
> [...]
> > > > It contains enough information in order to make a compact kernel
> > > > dump (makedumpinfo needs to go over the struct page arrays). As
> > > > you see, it also contains the kernel version.
> > > > 
> > > 
> > > But this will not solve Bernhard's problem where looking at a vmcore
> > > he wants to know which vmlinux (kernel version with time stamp) has
> > > generated this vmcore. So adding a ELF NOTE should help.
> > > 
> > I think an ELF note would be a fine idea.
> 
> Okay, so here's an implemenation.
> 
> See the attached proof-of-concept patches to the kernel-side kexec and 
> kexec-tools (might need some cleanup though). Next to follow, a patch 
> to makedumpfile. With these patches a new "LINUX" elf note generated 
> by the kernel in the format that makedumpfile expects and is being 
> passed on by the kexec util to the kdump kernel.
> 
> As a bonus, with this patch you don't even have to compile the kernel 
> with debug information in order for the filtering to work.
> 

This implementation looks interesting. No need of debug compiled
vmlinux for dump filtering purposes. No run time vmlinux binary modifications
as suggested by your previous mails. People can export kernel CONFIG info
like PAGESIZE etc with the help of this ELF note and any tool which is
processing kernel core file can benifit from this. No guesses required for
determining the page size of crashed kernel.

All the info required by dump filtering tool, put it into an ELF note, 
pass it to second kernel which appends it to final vmcore. The only concern
here again is that type of ELF note is not standard. The contents of 
ELF notes will keep on changing over a period of time. For example if a 
new memory model evolves. Now everytime this note data changes one shall
have to update mkdumpfile and user shall have to newer version. If there
are any issues, user will not come to know about the issue during kdump
service enablement time, instead it will be detected during dump 
capture time. It is too late till then.

I think we can make OSRELEASE info into a separate ELF note. Something
like a note type "UTS_RELEASE". This ELF note type then can also be 
reused for appending an ELF note to vmlinux to determine the kernel
version.
 

> As Vivek mentioned in another mail, the output of makedumpfile is not
> really a standard. However, I believe we should start making standards
> where no standard exists. :)
> 
> -- 
> Dan Aloni
> XIV LTD, http://www.xivstorage.com
> da-x (at) monatomic.org, dan (at) xiv.co.il

> diff --git a/include/linux/kexec.h b/include/linux/kexec.h
> index 8c2c7fc..a0412ce 100644
> --- a/include/linux/kexec.h
> +++ b/include/linux/kexec.h
> @@ -121,6 +121,8 @@ extern struct page *kimage_alloc_control_pages(struct kimage *image,
>  extern void crash_kexec(struct pt_regs *);
>  int kexec_should_crash(struct task_struct *);
>  void crash_save_cpu(struct pt_regs *regs, int cpu);
> +void crash_save_mkdfinfo(void);
> +

We don't have to save this info in ELF note format after the crash. This
info gets fixed once kernel is booted. I think we should save it in some
init routine and not in crash_kexec().

>  extern struct kimage *kexec_image;
>  extern struct kimage *kexec_crash_image;
> 
> @@ -153,7 +155,10 @@ extern struct kimage *kexec_crash_image;
>  extern struct resource crashk_res;
>  typedef u32 note_buf_t[KEXEC_NOTE_BYTES/4];
>  extern note_buf_t *crash_notes;
> -
> +extern unsigned char mkdfinfo_data[MAX_NOTE_BYTES];
> +extern unsigned int mkdfinfo_size;
> +extern unsigned int mkdfinfo_max_size;
> +extern note_buf_t mkdfinfo_note;

mkdfinfo does not seem to be very appropriate name. We probably need to have a
better name. This all seems to be debug data (symbol info, structure size,
offsets of struct members) so how about "debuginfo"?

Thanks
Vivek

Re: Determine version of kernel that produced vmcore

[Dan Aloni]

On Wed, Jul 11, 2007 at 11:37:26AM +0530, Vivek Goyal wrote:
> On Tue, Jul 10, 2007 at 07:52:01PM +0300, Dan Aloni wrote:
> > On Tue, Jul 10, 2007 at 08:09:04AM -0400, Neil Horman wrote:
> > > On Tue, Jul 10, 2007 at 12:18:17PM +0530, Vivek Goyal wrote:
> > > > On Fri, Jul 06, 2007 at 05:58:04PM +0300, Dan Aloni wrote:
> > > > > On Fri, Jul 06, 2007 at 03:28:14PM +0200, Bernhard Walle wrote:
> > > > > > Hello,
> > [...]
> > > > > It contains enough information in order to make a compact kernel
> > > > > dump (makedumpinfo needs to go over the struct page arrays). As
> > > > > you see, it also contains the kernel version.
> > > > > 
> > > > 
> > > > But this will not solve Bernhard's problem where looking at a vmcore
> > > > he wants to know which vmlinux (kernel version with time stamp) has
> > > > generated this vmcore. So adding a ELF NOTE should help.
> > > > 
> > > I think an ELF note would be a fine idea.
> > 
> > Okay, so here's an implemenation.
> > 
> > See the attached proof-of-concept patches to the kernel-side kexec and 
> > kexec-tools (might need some cleanup though). Next to follow, a patch 
> > to makedumpfile. With these patches a new "LINUX" elf note generated 
> > by the kernel in the format that makedumpfile expects and is being 
> > passed on by the kexec util to the kdump kernel.
> > 
> > As a bonus, with this patch you don't even have to compile the kernel 
> > with debug information in order for the filtering to work.
> > 
> 
> This implementation looks interesting. No need of debug compiled
> vmlinux for dump filtering purposes. No run time vmlinux binary modifications
> as suggested by your previous mails. People can export kernel CONFIG info
> like PAGESIZE etc with the help of this ELF note and any tool which is
> processing kernel core file can benifit from this. No guesses required for
> determining the page size of crashed kernel.
> 
> All the info required by dump filtering tool, put it into an ELF note, 
> pass it to second kernel which appends it to final vmcore. The only concern
> here again is that type of ELF note is not standard. The contents of 
> ELF notes will keep on changing over a period of time. For example if a 
> new memory model evolves. Now everytime this note data changes one shall
> have to update mkdumpfile and user shall have to newer version. If there
> are any issues, user will not come to know about the issue during kdump
> service enablement time, instead it will be detected during dump 
> capture time. It is too late till then.

I guess that dump filtering will needed to be tested at least once 
under various platforms for every major release of the kernel to see 
if the information exported by the kernel through this elf note is 
still sufficient. 

Adding more info to this elf note as needed can be done through a simple 
patch to the kernel and it can be tested along with mkdumpfile changes 
even before a major version gets released.

More things to consider:

 * Elf note size is currently limited to 1024, we might need a bit more
   (currently this elf note is in the 900-area).
 * If we move the generation of the elf note to the boot time instead 
   of crash time as you suggested, we can also make it viewable as a 
   virtual file.
 * We should come up with a standard name for the note (LINUX, or 
   LINUX-VMCORE, etc).
 
> I think we can make OSRELEASE info into a separate ELF note. Something
> like a note type "UTS_RELEASE". This ELF note type then can also be 
> reused for appending an ELF note to vmlinux to determine the kernel
> version.

Agreed.

Actually, having a UTS_RELEASE note attached to vmlinux itself is quite 
easy to implement. All we need is some .s file in the kernel doing 
something similar to this (inspired by some post made by Ulrich Drepper):

#include <linux/utsrelease.h>

	.section ".note.UTS_RELEASE", "a"
	.align 4
	.long 1f - 0f		/* name length */
	.long 3f - 2f		/* data length */
	.long  1		/* note type */
0:	.asciz "UTS_RELEASE"	/* uts release */
1:	.align 4	
2:	.asciz UTS_RELEASE     	/* note data */
3:	.align 4		/* pad out section */

> We don't have to save this info in ELF note format after the crash. This
> info gets fixed once kernel is booted. I think we should save it in some
> init routine and not in crash_kexec().

Agreed.
 
> > +extern unsigned char mkdfinfo_data[MAX_NOTE_BYTES];
> > +extern unsigned int mkdfinfo_size;
> > +extern unsigned int mkdfinfo_max_size;
> > +extern note_buf_t mkdfinfo_note;
> 
> mkdfinfo does not seem to be very appropriate name. We probably need to have a
> better name. This all seems to be debug data (symbol info, structure size,
> offsets of struct members) so how about "debuginfo"?

'debuginfo' can be confused with the more standard DWARF debuginfo. 
How about vmcoreinfo, dumpinfo, debugaux, debugauxinfo, or 
dumpauxinfo?

-- 
Dan Aloni
XIV LTD, http://www.xivstorage.com
da-x (at) monatomic.org, dan (at) xiv.co.il

Re: Determine version of kernel that produced vmcore

[Ken'ichi Ohmichi]

Hi Dan,

2007/07/11 10:32:16 +0300, Dan Aloni <da-x@monatomic.org> wrote:
>> This implementation looks interesting. No need of debug compiled
>> vmlinux for dump filtering purposes. No run time vmlinux binary modifications
>> as suggested by your previous mails. People can export kernel CONFIG info
>> like PAGESIZE etc with the help of this ELF note and any tool which is
>> processing kernel core file can benifit from this. No guesses required for
>> determining the page size of crashed kernel.
>> 
>> All the info required by dump filtering tool, put it into an ELF note, 
>> pass it to second kernel which appends it to final vmcore. The only concern
>> here again is that type of ELF note is not standard. The contents of 
>> ELF notes will keep on changing over a period of time. For example if a 
>> new memory model evolves. Now everytime this note data changes one shall
>> have to update mkdumpfile and user shall have to newer version. If there
>> are any issues, user will not come to know about the issue during kdump
>> service enablement time, instead it will be detected during dump 
>> capture time. It is too late till then.
>
>I guess that dump filtering will needed to be tested at least once 
>under various platforms for every major release of the kernel to see 
>if the information exported by the kernel through this elf note is 
>still sufficient. 
>
>Adding more info to this elf note as needed can be done through a simple 
>patch to the kernel and it can be tested along with mkdumpfile changes 
>even before a major version gets released.

As Vivek said, I think it is not a good idea that both kernel's code and
makedumpfile's code should be changed when makedumpfile needs more debug
information (ex. To extend the supported CPU architectures. To fix some
machine-specific problems, etc.).
Also it is difficult to change the list of necessary items in kernel,
because we should be careful about CPU architecture, linux version, and
memory model. makedumpfile needs the following items, and those existence
depends on CPU architecture, linux version, and memory model. If we don't
be careful about them, the kernel building may fail.
I think it is not a good idea that each items are defined by ifdef, it
is too complex.

* Symbols
  - mem_map
  - mem_section
  - pkmap_count
  - system_utsname
  - init_uts_ns
  - _stext
  - swapper_pg_dir
  - phys_base
  - node_online_map
  - node_data
  - pgdat_list
  - contig_page_data

* Structure sizes
  - page
  - mem_section
  - pglist_data
  - zone
  - free_area
  - list_head

* Field offsets
  - page.flags
  - page._count
  - page.mapping
  - page.lru
  - mem_section.section_mem_map
  - pglist_data.node_zones
  - pglist_data.nr_zones
  - pglist_data.node_mem_map
  - pglist_data.node_start_pfn
  - pglist_data.node_spanned_pages
  - pglist_data.pgdat_next
  - zone.free_pages
  - zone.free_area
  - zone.vm_stat
  - zone.spanned_pages
  - free_area.free_list
  - list_head.next
  - list_head.prev

* The numbers of array
  - mem_section
  - pgdat_list
  - node_data
  - zone.free_area


I'd like to know your motivation for this work.
I think you don't want to manage an extra file (mkdfinfo file) besides
a vmlinux file from distributor's point of view, right ?


Thanks
Ken'ichi Ohmichi

Re: Determine version of kernel that produced vmcore

[Vivek Goyal]

On Wed, Jul 11, 2007 at 10:43:04PM +0900, Ken'ichi Ohmichi wrote:
> 
> Hi Dan,
> 
> 2007/07/11 10:32:16 +0300, Dan Aloni <da-x@monatomic.org> wrote:
> >> This implementation looks interesting. No need of debug compiled
> >> vmlinux for dump filtering purposes. No run time vmlinux binary modifications
> >> as suggested by your previous mails. People can export kernel CONFIG info
> >> like PAGESIZE etc with the help of this ELF note and any tool which is
> >> processing kernel core file can benifit from this. No guesses required for
> >> determining the page size of crashed kernel.
> >> 
> >> All the info required by dump filtering tool, put it into an ELF note, 
> >> pass it to second kernel which appends it to final vmcore. The only concern
> >> here again is that type of ELF note is not standard. The contents of 
> >> ELF notes will keep on changing over a period of time. For example if a 
> >> new memory model evolves. Now everytime this note data changes one shall
> >> have to update mkdumpfile and user shall have to newer version. If there
> >> are any issues, user will not come to know about the issue during kdump
> >> service enablement time, instead it will be detected during dump 
> >> capture time. It is too late till then.
> >
> >I guess that dump filtering will needed to be tested at least once 
> >under various platforms for every major release of the kernel to see 
> >if the information exported by the kernel through this elf note is 
> >still sufficient. 
> >
> >Adding more info to this elf note as needed can be done through a simple 
> >patch to the kernel and it can be tested along with mkdumpfile changes 
> >even before a major version gets released.
> 
> As Vivek said, I think it is not a good idea that both kernel's code and
> makedumpfile's code should be changed when makedumpfile needs more debug
> information (ex. To extend the supported CPU architectures. To fix some
> machine-specific problems, etc.).
> Also it is difficult to change the list of necessary items in kernel,
> because we should be careful about CPU architecture, linux version, and
> memory model. makedumpfile needs the following items, and those existence
> depends on CPU architecture, linux version, and memory model. If we don't
> be careful about them, the kernel building may fail.
> I think it is not a good idea that each items are defined by ifdef, it
> is too complex.
> 

Hi Ken'ichi,

This approach seems to have advantages also. For example, now one will not
require a kernel debuginfo file to configure kdump with dump filtering.
I think having to have a debug info file is a big restriction.

Now vmcore ELF note contain all the data rquired to filter core dump and
there will be no need to parse DWARF debuginfo and makedumpfile code will be
smaller and simpler.

Given the fact that more and more architectures are supporting multiple
page sizes. Now one can also export relevant kernel CONFIG variables
through this ELF note. makedumpfile's job will be simpler. You don't have
to make any guesses about various config options like page size.

This approach will make second kernel truly independent of first kernel.
Currently, if first kernel changes one shall have to regenerate the 
initrd for second kenrel. Hence in a sense, second kernel is dependent on
first kernel. Now kernel will export all the required data for dump filtering
so one does not have to rebuild the seond kernel initrd even if first kernel
changes.

I think overall the approach of exporting right info from kernel should 
help. Only thing is that we shall have to manage the ELF note better in
terms of making it more generic.

Thanks
Vivek

Re: Determine version of kernel that produced vmcore

[Bernhard Walle]

* Vivek Goyal <vgoyal@in.ibm.com> [2007-07-13 05:58]:
> Given the fact that more and more architectures are supporting multiple
> page sizes. Now one can also export relevant kernel CONFIG variables
> through this ELF note. makedumpfile's job will be simpler. You don't have
> to make any guesses about various config options like page size.
> 
> This approach will make second kernel truly independent of first kernel.
> Currently, if first kernel changes one shall have to regenerate the 
> initrd for second kenrel. Hence in a sense, second kernel is dependent on
> first kernel. Now kernel will export all the required data for dump filtering
> so one does not have to rebuild the seond kernel initrd even if first kernel
> changes.

That's only the RedHat mechanism. You can copy all CONFIGFILEs to the
initrd, and then pick the right one via the version. Then, you only
have to regenerate the initrd when you *install* a new first kernel,
not when you boot a new first kernel. And that's also why I wanted the
kernel version to be accessible in the crashkernel ...

> I think overall the approach of exporting right info from kernel should 
> help. Only thing is that we shall have to manage the ELF note better in
> terms of making it more generic.

Basically I like the idea of exporting the information from the kernel
without generating the CONFIGFILE. The only big problem I see that the
makedumpfile release cycles would depend on the kernel release cycles,
and, you cannot add new features to makedumpfile for old kernels since
you always rely on the information which is exported.


Thanks,
   Bernhard

Re: Determine version of kernel that produced vmcore

[Vivek Goyal]

On Wed, Jul 11, 2007 at 10:32:16AM +0300, Dan Aloni wrote:
[..]
> > This implementation looks interesting. No need of debug compiled
> > vmlinux for dump filtering purposes. No run time vmlinux binary modifications
> > as suggested by your previous mails. People can export kernel CONFIG info
> > like PAGESIZE etc with the help of this ELF note and any tool which is
> > processing kernel core file can benifit from this. No guesses required for
> > determining the page size of crashed kernel.
> > 
> > All the info required by dump filtering tool, put it into an ELF note, 
> > pass it to second kernel which appends it to final vmcore. The only concern
> > here again is that type of ELF note is not standard. The contents of 
> > ELF notes will keep on changing over a period of time. For example if a 
> > new memory model evolves. Now everytime this note data changes one shall
> > have to update mkdumpfile and user shall have to newer version. If there
> > are any issues, user will not come to know about the issue during kdump
> > service enablement time, instead it will be detected during dump 
> > capture time. It is too late till then.
> 
> I guess that dump filtering will needed to be tested at least once 
> under various platforms for every major release of the kernel to see 
> if the information exported by the kernel through this elf note is 
> still sufficient. 
> 
> Adding more info to this elf note as needed can be done through a simple 
> patch to the kernel and it can be tested along with mkdumpfile changes 
> even before a major version gets released.
> 
> More things to consider:
> 
>  * Elf note size is currently limited to 1024, we might need a bit more
>    (currently this elf note is in the 900-area).

How about putting all the required info in a structure and then do
sizeof(struct) and reserve that much of memory at compile time for the
note. This will mean there are no hardcodings about the size of the note.

But I guess, one shall have to keep user space kexec-tools in sync to
create a PT_NOTE for that. May be exporting this info in header files to
user space might help.

>  * If we move the generation of the elf note to the boot time instead 
>    of crash time as you suggested, we can also make it viewable as a 
>    virtual file.

That would make sense. In that case, makedumpfile can do some verification
on the exported data and scream if something is not right at kdump service
start time.

>  * We should come up with a standard name for the note (LINUX, or 
>    LINUX-VMCORE, etc).
> 

How about VMCOREINFO?

> > I think we can make OSRELEASE info into a separate ELF note. Something
> > like a note type "UTS_RELEASE". This ELF note type then can also be 
> > reused for appending an ELF note to vmlinux to determine the kernel
> > version.
> 
> Agreed.
> 
> Actually, having a UTS_RELEASE note attached to vmlinux itself is quite 
> easy to implement. All we need is some .s file in the kernel doing 
> something similar to this (inspired by some post made by Ulrich Drepper):
> 
> #include <linux/utsrelease.h>
> 
> 	.section ".note.UTS_RELEASE", "a"
> 	.align 4
> 	.long 1f - 0f		/* name length */
> 	.long 3f - 2f		/* data length */
> 	.long  1		/* note type */
> 0:	.asciz "UTS_RELEASE"	/* uts release */
> 1:	.align 4	
> 2:	.asciz UTS_RELEASE     	/* note data */
> 3:	.align 4		/* pad out section */
> 
> > We don't have to save this info in ELF note format after the crash. This
> > info gets fixed once kernel is booted. I think we should save it in some
> > init routine and not in crash_kexec().
> 
> Agreed.
> 
> > > +extern unsigned char mkdfinfo_data[MAX_NOTE_BYTES];
> > > +extern unsigned int mkdfinfo_size;
> > > +extern unsigned int mkdfinfo_max_size;
> > > +extern note_buf_t mkdfinfo_note;
> > 
> > mkdfinfo does not seem to be very appropriate name. We probably need to have a
> > better name. This all seems to be debug data (symbol info, structure size,
> > offsets of struct members) so how about "debuginfo"?
> 
> 'debuginfo' can be confused with the more standard DWARF debuginfo. 
> How about vmcoreinfo, dumpinfo, debugaux, debugauxinfo, or 
> dumpauxinfo?
> 

To me vmcoreinfo looks like a good name to begin with. As of today vmcore
is the only consumer of the info. We might have to migrate to a more generic
name down the line if consumers of the info pop-up.

Thanks
Vivek

Re: Determine version of kernel that produced vmcore

[Bernhard Walle]

* Vivek Goyal <vgoyal@in.ibm.com> [2007-07-11 08:07]:
> On Tue, Jul 10, 2007 at 07:52:01PM +0300, Dan Aloni wrote:
> > On Tue, Jul 10, 2007 at 08:09:04AM -0400, Neil Horman wrote:
> > > On Tue, Jul 10, 2007 at 12:18:17PM +0530, Vivek Goyal wrote:
> > > > On Fri, Jul 06, 2007 at 05:58:04PM +0300, Dan Aloni wrote:
> > > > > On Fri, Jul 06, 2007 at 03:28:14PM +0200, Bernhard Walle wrote:
> > > > > > Hello,
> > [...]
> > > > > It contains enough information in order to make a compact kernel
> > > > > dump (makedumpinfo needs to go over the struct page arrays). As
> > > > > you see, it also contains the kernel version.
> > > > > 
> > > > 
> > > > But this will not solve Bernhard's problem where looking at a vmcore
> > > > he wants to know which vmlinux (kernel version with time stamp) has
> > > > generated this vmcore. So adding a ELF NOTE should help.
> > > > 
> > > I think an ELF note would be a fine idea.
> > 
> > Okay, so here's an implemenation.
> > 
> > See the attached proof-of-concept patches to the kernel-side kexec and 
> > kexec-tools (might need some cleanup though). Next to follow, a patch 
> > to makedumpfile. With these patches a new "LINUX" elf note generated 
> > by the kernel in the format that makedumpfile expects and is being 
> > passed on by the kexec util to the kdump kernel.
> > 
> > As a bonus, with this patch you don't even have to compile the kernel 
> > with debug information in order for the filtering to work.
> > 
> 
> This implementation looks interesting. No need of debug compiled
> vmlinux for dump filtering purposes. No run time vmlinux binary modifications
> as suggested by your previous mails. People can export kernel CONFIG info
> like PAGESIZE etc with the help of this ELF note and any tool which is
> processing kernel core file can benifit from this. No guesses required for
> determining the page size of crashed kernel.

I also like way of implementing it. :)


Thanks,
   Bernhard

Re: Determine version of kernel that produced vmcore

[Ken'ichi Ohmichi]

Hi,

On Fri, 6 Jul 2007 17:58:04 +0300, Dan Aloni <da-x@monatomic.org> wrote:
> On Fri, Jul 06, 2007 at 03:28:14PM +0200, Bernhard Walle wrote:
> > Hello,
> > 
> > does anybody know a _reliable_ way to determine the version the kernel
> > that produced a vmcore file? This means not scanning for a specific
> > string or something like that which can fail on random memory.
> > 
> > Would it make sense to add a ELF PT_NOTE section in the vmcore?
> > 
> > Thanks for input!

(CONFIGFILE means makedumpfile's config file.)

makedumpfile checks kernel version by reading system_utsname.release
from /proc/vmcore, as you know. If the release and OSRELEASE in CONFIGFILE
don't match, makedumpfile fails.

Besides Dan's plan, I'm planning the change of CONFIGFILE for distributors.
In the kernel building process, distributors need to make CONFIGFILE
on an older kernel (ex. RHEL5 kernel is built on RHEL4), and OSRELEASE
may be an older kernel. So OSRELEASE should be modified to the building
kernel version by hand, but it is not smart.

To solve this problem, I'm proposing 2 plans.
Could you give me your opinion ?

Plan 1:
  A new option [--osrelease="string"] is added.
  The OSRELEASE of CONFIGFILE is overwritten by "string".
  In the kernel building process, distributors should specify "string"
  as the building kernel version.

Plan2:
  Remove the OSRELEASE from CONFIGFILE.
  Instead of checking the OSRELEASE, makedumpfile only checks whether the
  area of /proc/vmcore specified by the symbol "system_utsname" in CONFIGFILE
  is the string "2.6.". If CONFIGFILE and /proc/vmcore don't match, the
  "system_utsname" must not point to the correct area in most cases.
  Old makedumpfile needs OSRELEASE, and it cannot work by new CONFIGFILE.
  But I think there are not any problems because old makedumpfile will not
  read new CONFIGFILE. Now, CONFIGFILE is used only by RHEL5's kdump initramfs,
  the CONFIGFILE is generated during 1st-kernel running. Even if CONFIGFILE
  will be updated, makedumpfile can read the CONFIGFILE because makedumpfile
  should be updated with CONFIGFILE.


I'd like to change the name of CONFIGFILE to mkdfinfo.


Thanks
Ken'ichi Ohmichi

Re: Determine version of kernel that produced vmcore

[Neil Horman]

On Tue, Jul 10, 2007 at 12:13:15PM +0900, Ken'ichi Ohmichi wrote:
> 
> Hi,
> 
> On Fri, 6 Jul 2007 17:58:04 +0300, Dan Aloni <da-x@monatomic.org> wrote:
> > On Fri, Jul 06, 2007 at 03:28:14PM +0200, Bernhard Walle wrote:
> > > Hello,
> > > 
> > > does anybody know a _reliable_ way to determine the version the kernel
> > > that produced a vmcore file? This means not scanning for a specific
> > > string or something like that which can fail on random memory.
> > > 
> > > Would it make sense to add a ELF PT_NOTE section in the vmcore?
> > > 
> > > Thanks for input!
> 
> (CONFIGFILE means makedumpfile's config file.)
> 
> makedumpfile checks kernel version by reading system_utsname.release
> from /proc/vmcore, as you know. If the release and OSRELEASE in CONFIGFILE
> don't match, makedumpfile fails.
> 
> Besides Dan's plan, I'm planning the change of CONFIGFILE for distributors.
> In the kernel building process, distributors need to make CONFIGFILE
> on an older kernel (ex. RHEL5 kernel is built on RHEL4), and OSRELEASE
> may be an older kernel. So OSRELEASE should be modified to the building
> kernel version by hand, but it is not smart.
> 
> To solve this problem, I'm proposing 2 plans.
> Could you give me your opinion ?
> 
> Plan 1:
>   A new option [--osrelease="string"] is added.
>   The OSRELEASE of CONFIGFILE is overwritten by "string".
>   In the kernel building process, distributors should specify "string"
>   as the building kernel version.
> 
> Plan2:
>   Remove the OSRELEASE from CONFIGFILE.
>   Instead of checking the OSRELEASE, makedumpfile only checks whether the
>   area of /proc/vmcore specified by the symbol "system_utsname" in CONFIGFILE
>   is the string "2.6.". If CONFIGFILE and /proc/vmcore don't match, the
>   "system_utsname" must not point to the correct area in most cases.
>   Old makedumpfile needs OSRELEASE, and it cannot work by new CONFIGFILE.
>   But I think there are not any problems because old makedumpfile will not
>   read new CONFIGFILE. Now, CONFIGFILE is used only by RHEL5's kdump initramfs,
>   the CONFIGFILE is generated during 1st-kernel running. Even if CONFIGFILE
>   will be updated, makedumpfile can read the CONFIGFILE because makedumpfile
>   should be updated with CONFIGFILE.
> 
> 
> I'd like to change the name of CONFIGFILE to mkdfinfo.
> 
Why not, instead of either plan above, just redefine OSRELEASE to be the version
of the kernel the config file was built against?  i.e. when you build a config
file, you need to specify a kernel to extract symbol information from, why not
grab the utsname from that kernel and use that to set OSRELEASE?  When you're
building a config file the running kernel on the system isn't really relevent
anyway.

Thanks & Regards
Neil

> 
> Thanks
> Ken'ichi Ohmichi

-- 
/***************************************************
 *Neil Horman
 *Software Engineer
 *Red Hat, Inc.
 *nhorman@redhat.com
 *gpg keyid: 1024D / 0x92A74FA1
 *http://pgp.mit.edu
 ***************************************************/

Re: Determine version of kernel that produced vmcore

[Ken'ichi Ohmichi]

Hi all,

2007/07/10 08:02:43 -0400, Neil Horman <nhorman@redhat.com> wrote:
>> Besides Dan's plan, I'm planning the change of CONFIGFILE for distributors.
>> In the kernel building process, distributors need to make CONFIGFILE
>> on an older kernel (ex. RHEL5 kernel is built on RHEL4), and OSRELEASE
>> may be an older kernel. So OSRELEASE should be modified to the building
>> kernel version by hand, but it is not smart.
>> 
>> To solve this problem, I'm proposing 2 plans.
>> Could you give me your opinion ?
>> 
>> Plan 1:
>>   A new option [--osrelease="string"] is added.
>>   The OSRELEASE of CONFIGFILE is overwritten by "string".
>>   In the kernel building process, distributors should specify "string"
>>   as the building kernel version.
>> 
>> Plan2:
>>   Remove the OSRELEASE from CONFIGFILE.
>>   Instead of checking the OSRELEASE, makedumpfile only checks whether the
>>   area of /proc/vmcore specified by the symbol "system_utsname" in CONFIGFILE
>>   is the string "2.6.". If CONFIGFILE and /proc/vmcore don't match, the
>>   "system_utsname" must not point to the correct area in most cases.
>>   Old makedumpfile needs OSRELEASE, and it cannot work by new CONFIGFILE.
>>   But I think there are not any problems because old makedumpfile will not
>>   read new CONFIGFILE. Now, CONFIGFILE is used only by RHEL5's kdump initramfs,
>>   the CONFIGFILE is generated during 1st-kernel running. Even if CONFIGFILE
>>   will be updated, makedumpfile can read the CONFIGFILE because makedumpfile
>>   should be updated with CONFIGFILE.
>> 
>> 
>> I'd like to change the name of CONFIGFILE to mkdfinfo.
>> 
>Why not, instead of either plan above, just redefine OSRELEASE to be the version
>of the kernel the config file was built against?  i.e. when you build a config
>file, you need to specify a kernel to extract symbol information from, why not
>grab the utsname from that kernel and use that to set OSRELEASE?  When you're
>building a config file the running kernel on the system isn't really relevent
>anyway.

Did you say "makedumpfile should get OSRELEASE from a vmlinux file,
and output it to a mkdfinfo file", right ?
If so, you are right. I'm creating the patch for it.

BTW, I'd like to remove PAGESIZE from a mkdfinfo file.
While 2nd-kernel is running, new makedumpfile comes to consider
2nd-kernel PAGESIZE as 1st-kernel PAGESIZE without getting PAGESIZE
from a mkdfinfo file.

In current implementation, makedumpfile considers 2nd-kernel PAGESIZE
as 1st-kernel PAGESIZE if a vmlinux file is specified instead of a
mkdfinfo file. On the other hand, makedumpfile can get 1st-kernel
PAGESIZE correctly if a mkdfinfo file is specified, because there
is the rule that a mkdfinfo file should be generated on 1st-kernel.

Now, I will change this rule for generating a mkdfinfo file on the
kernel-building environment. I feel considering 2nd-kernel PAGESIZE
as 1st-kernel PAGESIZE is better than considering the PAGESIZE of
kernel-building environment, because 1st-kernel and 2nd-kernel have
the same PAGESIZE in most cases such as relocatable kernel.


Please let me know your opinion.


Thanks
Ken'ichi Ohmichi

Re: Determine version of kernel that produced vmcore

[Bernhard Walle]

* Ken'ichi Ohmichi <oomichi@mxs.nes.nec.co.jp> [2007-07-13 13:05]:
> 
> BTW, I'd like to remove PAGESIZE from a mkdfinfo file.
> While 2nd-kernel is running, new makedumpfile comes to consider
> 2nd-kernel PAGESIZE as 1st-kernel PAGESIZE without getting PAGESIZE
> from a mkdfinfo file.

I don't think that's a good idea. IMO the kernel should be modified to
export the page size in a variable for that purpose. That would solve
all problems and dependencies, doesn't it?



Thanks,
   Bernhard

Re: Determine version of kernel that produced vmcore

[Vivek Goyal]

On Fri, Jul 13, 2007 at 03:15:50PM +0200, Bernhard Walle wrote:
> * Ken'ichi Ohmichi <oomichi@mxs.nes.nec.co.jp> [2007-07-13 13:05]:
> > 
> > BTW, I'd like to remove PAGESIZE from a mkdfinfo file.
> > While 2nd-kernel is running, new makedumpfile comes to consider
> > 2nd-kernel PAGESIZE as 1st-kernel PAGESIZE without getting PAGESIZE
> > from a mkdfinfo file.
> 
> I don't think that's a good idea. IMO the kernel should be modified to
> export the page size in a variable for that purpose. That would solve
> all problems and dependencies, doesn't it?
> 

Agreed. We need to export PAGESIZE from kernel instead of assuming that
second kernel as got same page size as first kernel.

Thanks
Vivek

Re: Determine version of kernel that produced vmcore

[Bernhard Walle]

* Vivek Goyal <vgoyal@in.ibm.com> [2007-07-16 06:19]:
> On Fri, Jul 13, 2007 at 03:15:50PM +0200, Bernhard Walle wrote:
> > * Ken'ichi Ohmichi <oomichi@mxs.nes.nec.co.jp> [2007-07-13 13:05]:
> > > 
> > > BTW, I'd like to remove PAGESIZE from a mkdfinfo file.
> > > While 2nd-kernel is running, new makedumpfile comes to consider
> > > 2nd-kernel PAGESIZE as 1st-kernel PAGESIZE without getting PAGESIZE
> > > from a mkdfinfo file.
> > 
> > I don't think that's a good idea. IMO the kernel should be modified to
> > export the page size in a variable for that purpose. That would solve
> > all problems and dependencies, doesn't it?
> > 
> 
> Agreed. We need to export PAGESIZE from kernel instead of assuming that
> second kernel as got same page size as first kernel.

So what about this? Do you think it has a chance to get included?
Should the variable not be inside mm/ but otherwhere?


Signed-off-by: Bernhard Walle <bwalle@suse.de>

---
 mm/mmap.c |   10 ++++++++++
 1 file changed, 10 insertions(+)

--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -35,6 +35,16 @@
 #define arch_mmap_check(addr, len, flags)	(0)
 #endif
 
+#ifdef CONFIG_KEXEC
+
+/*
+ * Although that variable is not needed for the kernel, initialise it here
+ * to have the page size available in the vmlinux binary.
+ */
+int page_size = PAGE_SIZE;
+
+#endif
+
 static void unmap_region(struct mm_struct *mm,
 		struct vm_area_struct *vma, struct vm_area_struct *prev,
 		unsigned long start, unsigned long end);

Re: Determine version of kernel that produced vmcore

[Vivek Goyal]

On Mon, Jul 16, 2007 at 01:57:07PM +0200, Bernhard Walle wrote:
> * Vivek Goyal <vgoyal@in.ibm.com> [2007-07-16 06:19]:
> > On Fri, Jul 13, 2007 at 03:15:50PM +0200, Bernhard Walle wrote:
> > > * Ken'ichi Ohmichi <oomichi@mxs.nes.nec.co.jp> [2007-07-13 13:05]:
> > > > 
> > > > BTW, I'd like to remove PAGESIZE from a mkdfinfo file.
> > > > While 2nd-kernel is running, new makedumpfile comes to consider
> > > > 2nd-kernel PAGESIZE as 1st-kernel PAGESIZE without getting PAGESIZE
> > > > from a mkdfinfo file.
> > > 
> > > I don't think that's a good idea. IMO the kernel should be modified to
> > > export the page size in a variable for that purpose. That would solve
> > > all problems and dependencies, doesn't it?
> > > 
> > 
> > Agreed. We need to export PAGESIZE from kernel instead of assuming that
> > second kernel as got same page size as first kernel.
> 
> So what about this? Do you think it has a chance to get included?
> Should the variable not be inside mm/ but otherwhere?
> 
> 
> Signed-off-by: Bernhard Walle <bwalle@suse.de>
> 
> ---
>  mm/mmap.c |   10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> --- a/mm/mmap.c
> +++ b/mm/mmap.c
> @@ -35,6 +35,16 @@
>  #define arch_mmap_check(addr, len, flags)	(0)
>  #endif
> 
> +#ifdef CONFIG_KEXEC
> +
> +/*
> + * Although that variable is not needed for the kernel, initialise it here
> + * to have the page size available in the vmlinux binary.
> + */
> +int page_size = PAGE_SIZE;
> +

Ok. Now there seems to be two ways for accessing such info.
- Through global variables
- Export through ELF notes.

Personally, I like the approach taken by Dan Aloni of exporting required
info through ELF notes. That seems to be more standard in the sense we 
are not dependent on somebody removing above variable tomorrow.

Dan, are you planning to put the modified patch for discussions on LKML?

Thanks
Vivek

Re: Determine version of kernel that produced vmcore

[Bernhard Walle]

[-- Attachment #1: Type: text/plain, Size: 648 bytes --]

* Vivek Goyal <vgoyal@in.ibm.com> [2007-07-16 14:25]:
> 
> Ok. Now there seems to be two ways for accessing such info.
> - Through global variables
> - Export through ELF notes.
> 
> Personally, I like the approach taken by Dan Aloni of exporting required
> info through ELF notes. That seems to be more standard in the sense we 
> are not dependent on somebody removing above variable tomorrow.
> 
> Dan, are you planning to put the modified patch for discussions on LKML?

You mean the proposal with the makedumpfile-compatible ELF note?
Although I like that idea, I think Ken'ichi Ohmichi was against it.


Thanks,
   Bernhard

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Determine version of kernel that produced vmcore

[Bernhard Walle]

* Vivek Goyal <vgoyal@in.ibm.com> [2007-07-16 14:25]:
> 
> Ok. Now there seems to be two ways for accessing such info.
> - Through global variables
> - Export through ELF notes.
> 
> Personally, I like the approach taken by Dan Aloni of exporting required
> info through ELF notes. That seems to be more standard in the sense we 
> are not dependent on somebody removing above variable tomorrow.
> 
> Dan, are you planning to put the modified patch for discussions on LKML?

You mean the proposal with the makedumpfile-compatible ELF note?
Although I like that idea, I think Ken'ichi Ohmichi was against it.


Thanks,
   Bernhard

Re: Determine version of kernel that produced vmcore

[Vivek Goyal]

On Mon, Jul 16, 2007 at 02:28:54PM +0200, Bernhard Walle wrote:
> * Vivek Goyal <vgoyal@in.ibm.com> [2007-07-16 14:25]:
> > 
> > Ok. Now there seems to be two ways for accessing such info.
> > - Through global variables
> > - Export through ELF notes.
> > 
> > Personally, I like the approach taken by Dan Aloni of exporting required
> > info through ELF notes. That seems to be more standard in the sense we 
> > are not dependent on somebody removing above variable tomorrow.
> > 
> > Dan, are you planning to put the modified patch for discussions on LKML?
> 
> You mean the proposal with the makedumpfile-compatible ELF note?
> Although I like that idea, I think Ken'ichi Ohmichi was against it.
> 

Yes, makedumpfile compatible ELF note one. Ken'ichi did not like the idea
too much, but I am not sure we discussed his objections in detail. I had
also responded with a mail with some of the advantages of ELF note based
approach. No response on that.

Thanks
Vivek

Re: Determine version of kernel that produced vmcore

[Ken'ichi Ohmichi]

Hi,

2007/07/16 18:06:33 +0530, Vivek Goyal <vgoyal@in.ibm.com> wrote:
>On Mon, Jul 16, 2007 at 02:28:54PM +0200, Bernhard Walle wrote:
>> * Vivek Goyal <vgoyal@in.ibm.com> [2007-07-16 14:25]:
>> > 
>> > Ok. Now there seems to be two ways for accessing such info.
>> > - Through global variables
>> > - Export through ELF notes.
>> > 
>> > Personally, I like the approach taken by Dan Aloni of exporting required
>> > info through ELF notes. That seems to be more standard in the sense we 
>> > are not dependent on somebody removing above variable tomorrow.
>> > 
>> > Dan, are you planning to put the modified patch for discussions on LKML?
>> 
>> You mean the proposal with the makedumpfile-compatible ELF note?
>> Although I like that idea, I think Ken'ichi Ohmichi was against it.
>> 
>
>Yes, makedumpfile compatible ELF note one. Ken'ichi did not like the idea
>too much, but I am not sure we discussed his objections in detail. I had
>also responded with a mail with some of the advantages of ELF note based
>approach. No response on that.

Sorry for my delayed response, I took a short vacation.

I'm not against with the proposal adding PT_NOTE segment to /proc/vmcore.
I don't like only point that adds the necessary information list
(some symbols, structures, etc.) for makedumpfile to kernel code.

The content of mkdfinfo file has been increasing whenever adding
features and correcting bugs. The content is increasing even now.
And, the feature addition is done not only for new kernels but also
for old upstream kernels. When the content of mkdfinfo file is taken 
into the kernel and fixed, the feature addition in the future is
remarkably limited. So makedumpfile needs mkdfinfo file outside of
the kernel.

I propose the solution including Dan's, Vivek's and Bernhard's opinions.
How about the following sequence for distributors ?
(It is not necessary to rebuild 2nd-kernel initrd even if 1st-kernel changes.
 A new option "--mkdfinfo" is added to kexec command.)

* On the kernel building system
  1. Create the kernel files.
     # cd linux-2.6.xx
     # make
  2. Create the mkdfinfo file from the vmlinux.
     # makedumpfile -g mkdfinfo-2.6.xx -x vmlinux
     (OSRELEASE is taken from the vmlinux.)
  3. Pack the kernel package with the kernel files and the mkdfinfo file.

* On each system
  4. Preload 2nd-kernel and the mkdfinfo file.
     # kexec -p vmlinux-kdump --mkdfinfo=mkdfinfo-2.6.xx ...
     - 2nd-kernel is preloaded.
     - 1st-kernel's PAGESIZE is taken by sysconf().
     - The mkdfinfo file and the above PAGESIZE is preloaded
       into /proc/vmcore's PT_NOTE segment.
  5. Panic happens, and the system switchs to 2nd-kernel.
  6. Create a filtered dump file.
     # makedumpfile -cd31 /proc/vmcore dumpfile
     (User doesn't need to specify the mkdfinfo file, because the
      contents of the mkdfinfo file is included into /proc/vmcore.)


Thanks
Ken'ichi Ohmichi

Re: Determine version of kernel that produced vmcore

[Bernhard Walle]

Hello,

* Ken'ichi Ohmichi <oomichi@mxs.nes.nec.co.jp> [2007-07-18 16:07]:
> I propose the solution including Dan's, Vivek's and Bernhard's opinions.
> How about the following sequence for distributors ?
> (It is not necessary to rebuild 2nd-kernel initrd even if 1st-kernel changes.
>  A new option "--mkdfinfo" is added to kexec command.)
> 
> * On the kernel building system
>   1. Create the kernel files.
>      # cd linux-2.6.xx
>      # make
>   2. Create the mkdfinfo file from the vmlinux.
>      # makedumpfile -g mkdfinfo-2.6.xx -x vmlinux
>      (OSRELEASE is taken from the vmlinux.)
>   3. Pack the kernel package with the kernel files and the mkdfinfo file.
> 
> * On each system
>   4. Preload 2nd-kernel and the mkdfinfo file.
>      # kexec -p vmlinux-kdump --mkdfinfo=mkdfinfo-2.6.xx ...
>      - 2nd-kernel is preloaded.
>      - 1st-kernel's PAGESIZE is taken by sysconf().
>      - The mkdfinfo file and the above PAGESIZE is preloaded
>        into /proc/vmcore's PT_NOTE segment.
>   5. Panic happens, and the system switchs to 2nd-kernel.
>   6. Create a filtered dump file.
>      # makedumpfile -cd31 /proc/vmcore dumpfile
>      (User doesn't need to specify the mkdfinfo file, because the
>       contents of the mkdfinfo file is included into /proc/vmcore.)

I like the proposal. The advantage over exporting the needed
information from the kernel directly is that makedumpfile (and the
contents of the mkdfinfo file [1]) can be changed without changing the
kernel which simplifies maintenance.

If now everybody agrees that this is the way to go, I can also offer
to implement the kernel and kexec userspace utility part
implementation of that stuff.

Maybe we should wait until Simon Horman is back here as he's the
current maintainer of kexec-tools(-testing).


Thanks,
   Bernhard

[1] didn't we agree to vmcoreinfo?

Re: Determine version of kernel that produced vmcore

[Ken'ichi Ohmichi]

Hi Bernhard,

2007/07/19 01:10:50 +0200, Bernhard Walle <bwalle@suse.de> wrote:
>[1] didn't we agree to vmcoreinfo?

I agree to vmcoreinfo.
I'll rename makedumpfile's config file to "vmcoreinfo".


Thanks
Ken'ichi Ohmichi

Re: Determine version of kernel that produced vmcore

[Vivek Goyal]

On Wed, Jul 18, 2007 at 11:07:40PM +0900, Ken'ichi Ohmichi wrote:
> 
> Hi,
> 
> 2007/07/16 18:06:33 +0530, Vivek Goyal <vgoyal@in.ibm.com> wrote:
> >On Mon, Jul 16, 2007 at 02:28:54PM +0200, Bernhard Walle wrote:
> >> * Vivek Goyal <vgoyal@in.ibm.com> [2007-07-16 14:25]:
> >> > 
> >> > Ok. Now there seems to be two ways for accessing such info.
> >> > - Through global variables
> >> > - Export through ELF notes.
> >> > 
> >> > Personally, I like the approach taken by Dan Aloni of exporting required
> >> > info through ELF notes. That seems to be more standard in the sense we 
> >> > are not dependent on somebody removing above variable tomorrow.
> >> > 
> >> > Dan, are you planning to put the modified patch for discussions on LKML?
> >> 
> >> You mean the proposal with the makedumpfile-compatible ELF note?
> >> Although I like that idea, I think Ken'ichi Ohmichi was against it.
> >> 
> >
> >Yes, makedumpfile compatible ELF note one. Ken'ichi did not like the idea
> >too much, but I am not sure we discussed his objections in detail. I had
> >also responded with a mail with some of the advantages of ELF note based
> >approach. No response on that.
> 
> Sorry for my delayed response, I took a short vacation.
> 
> I'm not against with the proposal adding PT_NOTE segment to /proc/vmcore.
> I don't like only point that adds the necessary information list
> (some symbols, structures, etc.) for makedumpfile to kernel code.
> 
> The content of mkdfinfo file has been increasing whenever adding
> features and correcting bugs. The content is increasing even now.
> And, the feature addition is done not only for new kernels but also
> for old upstream kernels. When the content of mkdfinfo file is taken 
> into the kernel and fixed, the feature addition in the future is
> remarkably limited. So makedumpfile needs mkdfinfo file outside of
> the kernel.
> 

Well, idea of filtering tool having flexibility to create its own ELF note
and later retrieve it and use it looks good. But at the same time you 
will have to standardize the contents of ELF note. One can not keep on
changing it at will.

> I propose the solution including Dan's, Vivek's and Bernhard's opinions.
> How about the following sequence for distributors ?
> (It is not necessary to rebuild 2nd-kernel initrd even if 1st-kernel changes.
>  A new option "--mkdfinfo" is added to kexec command.)
> 
> * On the kernel building system
>   1. Create the kernel files.
>      # cd linux-2.6.xx
>      # make
>   2. Create the mkdfinfo file from the vmlinux.
>      # makedumpfile -g mkdfinfo-2.6.xx -x vmlinux
>      (OSRELEASE is taken from the vmlinux.)
>   3. Pack the kernel package with the kernel files and the mkdfinfo file.
> 
> * On each system
>   4. Preload 2nd-kernel and the mkdfinfo file.
>      # kexec -p vmlinux-kdump --mkdfinfo=mkdfinfo-2.6.xx ...

Providing first kernel's debug data on commandline while loading second
kernel is little odd but I guess can be lived with.

>      - 2nd-kernel is preloaded.
>      - 1st-kernel's PAGESIZE is taken by sysconf().
>      - The mkdfinfo file and the above PAGESIZE is preloaded
>        into /proc/vmcore's PT_NOTE segment.

>   5. Panic happens, and the system switchs to 2nd-kernel.
>   6. Create a filtered dump file.
>      # makedumpfile -cd31 /proc/vmcore dumpfile
>      (User doesn't need to specify the mkdfinfo file, because the
>       contents of the mkdfinfo file is included into /proc/vmcore.)
> 

Overall proposal looks interesting. But it still requires a debug compiled
vmlinux for one to use filtering. For distros it is probably easy as they
can extract the debug info at build time and ship separate file with kernel
package (/boot/mkdfinfo-2.6.xxx). For people building their custom kernel,
they still need to build vmlinux with debuginfo. Personally, I wished
we could get rid of this requirement. I think may be down the line we can
start exporting it from kenrel once contents of ELF note are well defined.

Thanks
Vivek

Re: Determine version of kernel that produced vmcore

[Don Zickus]

On Wed, Jul 18, 2007 at 11:07:40PM +0900, Ken'ichi Ohmichi wrote:
> The content of mkdfinfo file has been increasing whenever adding
> features and correcting bugs. The content is increasing even now.
> And, the feature addition is done not only for new kernels but also
> for old upstream kernels. When the content of mkdfinfo file is taken 
> into the kernel and fixed, the feature addition in the future is
> remarkably limited. So makedumpfile needs mkdfinfo file outside of
> the kernel.
> 
> I propose the solution including Dan's, Vivek's and Bernhard's opinions.
> How about the following sequence for distributors ?
> (It is not necessary to rebuild 2nd-kernel initrd even if 1st-kernel changes.
>  A new option "--mkdfinfo" is added to kexec command.)
> 
> * On the kernel building system
>   1. Create the kernel files.
>      # cd linux-2.6.xx
>      # make
>   2. Create the mkdfinfo file from the vmlinux.
>      # makedumpfile -g mkdfinfo-2.6.xx -x vmlinux

I am not a big fan of this approach as it forces distros to require
kexec-tools when building a kernel.  Even Joe Hacker who wants a custom
kernel (and not interested in kexec) would have to not only download the
kernel.src.rpm but kexec-tools just to build a kernel that probably
doesn't have kexec enabled.

I had to deal with a similar experience when providing a build dependency
on the unifdef package when create the kernel-headers rpm for RHEL-5.  A
lot of people were confused and upset by this dependency.  Luckily
upstream resolved this by just putting similar code in the kernel/scripts
directory and thus removing the dependency (well not for RHEL-5).  I would
rather not have to deal with this again unless I know there is a more
permanent solution that would remove this dependency.

After talking to Dave Anderson about this more, I start to understand why
Ken'ichi prefers to implement the new features in userspace instead of the
kernel (it makes things automatically work with older kernels), but I
still am not a big fan of it.  I was hoping for a complete in kernel
solution (that way you never need the vmlinux file).  Perhaps makedumpfile
can support both vmlinux files (if provided) or interface with the kernel
(if the vmlinux is not provided?).

Just my input.  I might as well argue with Ken'ichi here than the
bugzilla. :-)

Cheers,
Don

Re: Determine version of kernel that produced vmcore

[Bernhard Walle]

* Don Zickus <dzickus@redhat.com> [2007-07-19 18:39]:
> 
> I am not a big fan of this approach as it forces distros to require
> kexec-tools when building a kernel.  Even Joe Hacker who wants a custom
> kernel (and not interested in kexec) would have to not only download the
> kernel.src.rpm but kexec-tools just to build a kernel that probably
> doesn't have kexec enabled.

He would have to download makedumpfile, not kexec-tools.


Thanks,
   Bernhard

Re: Determine version of kernel that produced vmcore

[Don Zickus]

On Thu, Jul 19, 2007 at 06:49:53PM +0200, Bernhard Walle wrote:
> * Don Zickus <dzickus@redhat.com> [2007-07-19 18:39]:
> > 
> > I am not a big fan of this approach as it forces distros to require
> > kexec-tools when building a kernel.  Even Joe Hacker who wants a custom
> > kernel (and not interested in kexec) would have to not only download the
> > kernel.src.rpm but kexec-tools just to build a kernel that probably
> > doesn't have kexec enabled.
> 
> He would have to download makedumpfile, not kexec-tools.

My apologies.  Red Hat packages makedumpfile under kexec-tools, hence my
confusion.

Cheers,
Don

Re: Determine version of kernel that produced vmcore

[Vivek Goyal]

On Thu, Jul 19, 2007 at 12:39:15PM -0400, Don Zickus wrote:
[..]
> 
> I am not a big fan of this approach as it forces distros to require
> kexec-tools when building a kernel.  Even Joe Hacker who wants a custom
> kernel (and not interested in kexec) would have to not only download the
> kernel.src.rpm but kexec-tools just to build a kernel that probably
> doesn't have kexec enabled.
> 
> I had to deal with a similar experience when providing a build dependency
> on the unifdef package when create the kernel-headers rpm for RHEL-5.  A
> lot of people were confused and upset by this dependency.  Luckily
> upstream resolved this by just putting similar code in the kernel/scripts
> directory and thus removing the dependency (well not for RHEL-5).  I would
> rather not have to deal with this again unless I know there is a more
> permanent solution that would remove this dependency.
> 

That's a good point Don. kernel rpm being dependent on kexec-tools rpm is
not a good idea.

> After talking to Dave Anderson about this more, I start to understand why
> Ken'ichi prefers to implement the new features in userspace instead of the
> kernel (it makes things automatically work with older kernels), but I
> still am not a big fan of it.  I was hoping for a complete in kernel
> solution (that way you never need the vmlinux file).  Perhaps makedumpfile
> can support both vmlinux files (if provided) or interface with the kernel
> (if the vmlinux is not provided?).
> 

I am also in favour of a complete kernel based solution. Export required
info from kernel and let kexec-tools parse that info, pass it to second
kernel and it will be appended to vmcore.

This will put some restrictions on that we can't keep on changing the
format of the info very frequently and some new features might not work
with older kernels. But I guess, kexec-tools can provide an override option
where dump filtering info can be passed on kexec-tools command line (as
suggested by ken'chi). If user passes this info on command line then
kexec-tools will not read the info exported from kernel. This way new
features can be made to work on older kernels. 

Thanks
Vivek

Re: Determine version of kernel that produced vmcore

[Ken'ichi Ohmichi]

Hi,

2007/07/23 10:31:47 +0530, Vivek Goyal <vgoyal@in.ibm.com> wrote:
>On Thu, Jul 19, 2007 at 12:39:15PM -0400, Don Zickus wrote:
>[..]
>> 
>> I am not a big fan of this approach as it forces distros to require
>> kexec-tools when building a kernel.  Even Joe Hacker who wants a custom
>> kernel (and not interested in kexec) would have to not only download the
>> kernel.src.rpm but kexec-tools just to build a kernel that probably
>> doesn't have kexec enabled.
>> 
>> I had to deal with a similar experience when providing a build dependency
>> on the unifdef package when create the kernel-headers rpm for RHEL-5.  A
>> lot of people were confused and upset by this dependency.  Luckily
>> upstream resolved this by just putting similar code in the kernel/scripts
>> directory and thus removing the dependency (well not for RHEL-5).  I would
>> rather not have to deal with this again unless I know there is a more
>> permanent solution that would remove this dependency.
>> 
>
>That's a good point Don. kernel rpm being dependent on kexec-tools rpm is
>not a good idea.

That's right. The kernel package should not be dependent on kexec-tools
package (including makedumpfile). I understand Dan Aloni's proposal
(including the dump filtering information in kernel) makes many
advantages. Ok, I agree with his proposal.

BTW, we should test makedumpfile by each linux(-rc1 ?) releases.
Now, I test the makedumpfile functions by comparing the crash utility's
output of /proc/vmcore and the one of the filtered dumpfile.
Often, the crash utility cannot read /proc/vmcore of the latest kernel.
Does anybody know the crash's option that the crash can run loosely for
the early test ?


>> After talking to Dave Anderson about this more, I start to understand why
>> Ken'ichi prefers to implement the new features in userspace instead of the
>> kernel (it makes things automatically work with older kernels), but I
>> still am not a big fan of it.  I was hoping for a complete in kernel
>> solution (that way you never need the vmlinux file).  Perhaps makedumpfile
>> can support both vmlinux files (if provided) or interface with the kernel
>> (if the vmlinux is not provided?).
>> 
>
>I am also in favour of a complete kernel based solution. Export required
>info from kernel and let kexec-tools parse that info, pass it to second
>kernel and it will be appended to vmcore.
>
>This will put some restrictions on that we can't keep on changing the
>format of the info very frequently and some new features might not work
>with older kernels. But I guess, kexec-tools can provide an override option
>where dump filtering info can be passed on kexec-tools command line (as
>suggested by ken'chi). If user passes this info on command line then
>kexec-tools will not read the info exported from kernel. This way new
>features can be made to work on older kernels. 

That sounds good.


Dan Aloni, I'd like to cooperate with you for implementing this feature.
If you have some patches other than 2007/07/10 patches, could you send
me them ?  I will update them.


Thanks
Ken'ichi Ohmichi

Re: Determine version of kernel that produced vmcore

[Dan Aloni]

On Mon, Jul 23, 2007 at 08:47:23PM +0900, Ken'ichi Ohmichi wrote:
> 
> Hi,
> 
> 2007/07/23 10:31:47 +0530, Vivek Goyal <vgoyal@in.ibm.com> wrote:
[..]
> >
> >I am also in favour of a complete kernel based solution. Export required
> >info from kernel and let kexec-tools parse that info, pass it to second
> >kernel and it will be appended to vmcore.
> >
> >This will put some restrictions on that we can't keep on changing the
> >format of the info very frequently and some new features might not work
> >with older kernels. But I guess, kexec-tools can provide an override option
> >where dump filtering info can be passed on kexec-tools command line (as
> >suggested by ken'chi). If user passes this info on command line then
> >kexec-tools will not read the info exported from kernel. This way new
> >features can be made to work on older kernels. 
> 
> That sounds good.

Note that currently kexec doesn't actually pass the ELF notes themselves
to the kernel, but only their physical addresses as extracted from /sys. 
If we want to be able to pass the notes themselves from userspace, a 
little more trickery will be needed here.

> Dan Aloni, I'd like to cooperate with you for implementing this feature.
> If you have some patches other than 2007/07/10 patches, could you send
> me them ?  I will update them.

Sure, though I haven't made new patches (was busy with other things). 
Feel free to post new versions of these patches, I'll take a look and 
cooperate with you on this.

-- 
Dan Aloni
XIV LTD, http://www.xivstorage.com
da-x (at) monatomic.org, dan (at) xiv.co.il

Re: Determine version of kernel that produced vmcore

[Vivek Goyal]

On Mon, Jul 23, 2007 at 04:02:39PM +0300, Dan Aloni wrote:
> On Mon, Jul 23, 2007 at 08:47:23PM +0900, Ken'ichi Ohmichi wrote:
> > 
> > Hi,
> > 
> > 2007/07/23 10:31:47 +0530, Vivek Goyal <vgoyal@in.ibm.com> wrote:
> [..]
> > >
> > >I am also in favour of a complete kernel based solution. Export required
> > >info from kernel and let kexec-tools parse that info, pass it to second
> > >kernel and it will be appended to vmcore.
> > >
> > >This will put some restrictions on that we can't keep on changing the
> > >format of the info very frequently and some new features might not work
> > >with older kernels. But I guess, kexec-tools can provide an override option
> > >where dump filtering info can be passed on kexec-tools command line (as
> > >suggested by ken'chi). If user passes this info on command line then
> > >kexec-tools will not read the info exported from kernel. This way new
> > >features can be made to work on older kernels. 
> > 
> > That sounds good.
> 
> Note that currently kexec doesn't actually pass the ELF notes themselves
> to the kernel, but only their physical addresses as extracted from /sys. 
> If we want to be able to pass the notes themselves from userspace, a 
> little more trickery will be needed here.
> 

That should be easy. Like so many other segments kexec prepares, one
more segment need to be created and required elf note data needs to be
put in this segment. 

Right now we create a PT_NOTE type ELF header for every cpu. Now we need
to create few more notes to pass on the OS release, config variables and
debug data to second kernel.

Should we create a separate note for passing on kernel config info something
like "linuxconfig" and every note will contain CONFIG_VARIABLE=VALUE string?

Probably something similar can be done for appropriate debugging data 
instead of putting all the info in one ELF note. Breaking the required
info into multiple generic notes will help it to be used in other things
also down the line. Just a thought....

Thanks
Vivek

Re: Determine version of kernel that produced vmcore

[Ken'ichi Ohmichi]

Hi Dan,

2007/07/23 16:02:39 +0300, Dan Aloni <da-x@monatomic.org> wrote:
>> Dan Aloni, I'd like to cooperate with you for implementing this feature.
>> If you have some patches other than 2007/07/10 patches, could you send
>> me them ?  I will update them.
>
>Sure, though I haven't made new patches (was busy with other things). 
>Feel free to post new versions of these patches, I'll take a look and 
>cooperate with you on this.

OK. Now, I am testing a new makedumpfile including some corrections
and some features to release it in July. I'll start making new patches
after the release.


Thanks
Ken'ichi Ohmichi

Re: Determine version of kernel that produced vmcore

[Vivek Goyal]

On Mon, Jul 16, 2007 at 01:57:07PM +0200, Bernhard Walle wrote:
> * Vivek Goyal <vgoyal@in.ibm.com> [2007-07-16 06:19]:
> > On Fri, Jul 13, 2007 at 03:15:50PM +0200, Bernhard Walle wrote:
> > > * Ken'ichi Ohmichi <oomichi@mxs.nes.nec.co.jp> [2007-07-13 13:05]:
> > > > 
> > > > BTW, I'd like to remove PAGESIZE from a mkdfinfo file.
> > > > While 2nd-kernel is running, new makedumpfile comes to consider
> > > > 2nd-kernel PAGESIZE as 1st-kernel PAGESIZE without getting PAGESIZE
> > > > from a mkdfinfo file.
> > > 
> > > I don't think that's a good idea. IMO the kernel should be modified to
> > > export the page size in a variable for that purpose. That would solve
> > > all problems and dependencies, doesn't it?
> > > 
> > 
> > Agreed. We need to export PAGESIZE from kernel instead of assuming that
> > second kernel as got same page size as first kernel.
> 
> So what about this? Do you think it has a chance to get included?
> Should the variable not be inside mm/ but otherwhere?
> 
> 
> Signed-off-by: Bernhard Walle <bwalle@suse.de>
> 
> ---
>  mm/mmap.c |   10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> --- a/mm/mmap.c
> +++ b/mm/mmap.c
> @@ -35,6 +35,16 @@
>  #define arch_mmap_check(addr, len, flags)	(0)
>  #endif
> 
> +#ifdef CONFIG_KEXEC
> +
> +/*
> + * Although that variable is not needed for the kernel, initialise it here
> + * to have the page size available in the vmlinux binary.
> + */
> +int page_size = PAGE_SIZE;
> +

Thinking more about it. These config variables are accessible in user
space in first kernel through sysconf() interface. In that case kexec
can pack required config variables into an ELF note and kernel does not
have to intervene.

Thanks
Vivek

Re: Determine version of kernel that produced vmcore

[Bernhard Walle]

* Vivek Goyal <vgoyal@in.ibm.com> [2007-07-17 05:50]:
> 
> Thinking more about it. These config variables are accessible in user
> space in first kernel through sysconf() interface. In that case kexec
> can pack required config variables into an ELF note and kernel does not
> have to intervene.

That's a good point. So we'd only need to aggree on a format for all
this information. I think the makedumpfile format is quite good --
instead of inventing another new format.


Thanks,
   Bernhard

Re: Determine version of kernel that produced vmcore

[Bernhard Walle]

Hi,

* Ken'ichi Ohmichi <oomichi@mxs.nes.nec.co.jp> [2007-07-10 05:13]:
> Plan 1:
>   A new option [--osrelease="string"] is added.
>   The OSRELEASE of CONFIGFILE is overwritten by "string".
>   In the kernel building process, distributors should specify "string"
>   as the building kernel version.
> 
> Plan2:
>   Remove the OSRELEASE from CONFIGFILE.
>   Instead of checking the OSRELEASE, makedumpfile only checks whether the
>   area of /proc/vmcore specified by the symbol "system_utsname" in CONFIGFILE
>   is the string "2.6.". If CONFIGFILE and /proc/vmcore don't match, the
>   "system_utsname" must not point to the correct area in most cases.
>   Old makedumpfile needs OSRELEASE, and it cannot work by new CONFIGFILE.
>   But I think there are not any problems because old makedumpfile will not
>   read new CONFIGFILE. Now, CONFIGFILE is used only by RHEL5's kdump initramfs,
>   the CONFIGFILE is generated during 1st-kernel running. Even if CONFIGFILE
>   will be updated, makedumpfile can read the CONFIGFILE because makedumpfile
>   should be updated with CONFIGFILE.

I would propose a third method. Some information cannot be extracted
from the vmlinux file, even with debugging information. That is
currently the page size.

So why not adding an option to makedumpfile which specifies the
.config of the kernel? If the .config is available, it can extract the
missing information (page size) from the .config and also use the
kernel version from this file.

If no config is available, the current process is right since using
the version of the running kernel ensures to some degree (not 100 %)
that the page size is correct.


Thanks,
   Bernhard

Re: Determine version of kernel that produced vmcore

[Vivek Goyal]

On Fri, Jul 06, 2007 at 03:28:14PM +0200, Bernhard Walle wrote:
> Hello,
> 
> does anybody know a _reliable_ way to determine the version the kernel
> that produced a vmcore file? This means not scanning for a specific
> string or something like that which can fail on random memory.
> 
> Would it make sense to add a ELF PT_NOTE section in the vmcore?
> 

To me it makes sense to append an ELF NOTE to vmcore which can uniquely
associate that vmcore with a particular vmlinux file (Some string
equivalent to the output of uname -a).

Interesting thing is that to find version of vmlinux, you will still be
doing <string | grep "some pattern">. Probably then we should append an
ELF note to vmlinux file also?

Thanks
Vivek

Re: Determine version of kernel that produced vmcore

[Bernhard Walle]

* Vivek Goyal <vgoyal@in.ibm.com> [2007-07-10 08:36]:
> > does anybody know a _reliable_ way to determine the version the kernel
> > that produced a vmcore file? This means not scanning for a specific
> > string or something like that which can fail on random memory.
> > 
> > Would it make sense to add a ELF PT_NOTE section in the vmcore?
> 
> To me it makes sense to append an ELF NOTE to vmcore which can uniquely
> associate that vmcore with a particular vmlinux file (Some string
> equivalent to the output of uname -a).

Well, if we embed the CONFIGFILE from makedumpfile in some magic way
we don't need it ... we'll see.

> Interesting thing is that to find version of vmlinux, you will still be
> doing <string | grep "some pattern">. Probably then we should append an
> ELF note to vmlinux file also?

Yes, that was my proposal some time ago, see
http://article.gmane.org/gmane.comp.boot-loaders.fastboot.general/3996
Any comments on that patch are appreciated. :)


Thanks,
   Bernhard