Stupid kexec/kdump question...

Stupid kexec/kdump question...

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 793 bytes --]

OK, I'm running a Fedora Core 6 (rawhide actually) box with -18-mm1 kernel.
I've installed kexec-tools and similar, and am trying to get the kernels
built following the hints in Documentation/kdump/kdump.txt, but a few
questions arise:

1) Other than the fact that the Fedora userspace looks for a ${kernelvers}kdump
kernel, is there any reason the kdump kernel has to match the running one, or
can an older kernel be used?

2) I'm presuming that a massively stripped down kernel (no sound support,
no netfilter, no etc) that just has what's needed to mount the dump location
is sufficient?

3) The docs recommend 'crashkernel=64M@16M', but that's 8% of my memory.
What will happen if I try '16M@16M' instead?  Just slower copying due to
a smaller buffer cache space, or something more evil?


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Stupid kexec/kdump question...

[Andrew Morton]

On Tue, 26 Sep 2006 11:25:06 -0400
Valdis.Kletnieks@vt.edu wrote:

> OK, I'm running a Fedora Core 6 (rawhide actually) box with -18-mm1 kernel.
> I've installed kexec-tools and similar, and am trying to get the kernels
> built following the hints in Documentation/kdump/kdump.txt, but a few
> questions arise:
> 
> 1) Other than the fact that the Fedora userspace looks for a ${kernelvers}kdump
> kernel, is there any reason the kdump kernel has to match the running one, or
> can an older kernel be used?
> 
> 2) I'm presuming that a massively stripped down kernel (no sound support,
> no netfilter, no etc) that just has what's needed to mount the dump location
> is sufficient?
> 
> 3) The docs recommend 'crashkernel=64M@16M', but that's 8% of my memory.
> What will happen if I try '16M@16M' instead?  Just slower copying due to
> a smaller buffer cache space, or something more evil?
> 

(cc added)

Re: [Fastboot] Stupid kexec/kdump question...

[Horms]

On Tue, Sep 26, 2006 at 10:10:29PM -0700, Andrew Morton wrote:
> On Tue, 26 Sep 2006 11:25:06 -0400
> Valdis.Kletnieks@vt.edu wrote:
> 
> > OK, I'm running a Fedora Core 6 (rawhide actually) box with -18-mm1 kernel.
> > I've installed kexec-tools and similar, and am trying to get the kernels
> > built following the hints in Documentation/kdump/kdump.txt, but a few
> > questions arise:
> > 
> > 1) Other than the fact that the Fedora userspace looks for a
> > ${kernelvers}kdump kernel, is there any reason the kdump kernel has
> > to match the running one, or can an older kernel be used?

The post-crash kernel is not realy dependant on the pre-crash kernel.
What is important is that either the kernel is relocatable
(which is being worked on for x86 and i386), or it is compiled to
run at a non-default address and that address corresponds
to the region reserved by the crashkernel command line parameter
passed to the pre-crash kernel.

The post-crash kernel will also need CONFIG_CRASH_DUMP 
and likely CONFIG_PROC_VMCORE

> > 2) I'm presuming that a massively stripped down kernel (no sound support,
> > no netfilter, no etc) that just has what's needed to mount the dump location
> > is sufficient?

Yes

> > 3) The docs recommend 'crashkernel=64M@16M', but that's 8% of my memory.
> > What will happen if I try '16M@16M' instead?  Just slower copying due to
> > a smaller buffer cache space, or something more evil?

There is a lower bound to how small you can make the space, which
is basically how little memory space your post-crash kernel needs.
16M is probably pushing it, but 32M should be more than possible.
Experimentation is really the order of the day here.

-- 
Horms
  H: http://www.vergenet.net/~horms/
  W: http://www.valinux.co.jp/en/

Re: [Fastboot] Stupid kexec/kdump question...

[Eric W. Biederman]

Horms <horms@verge.net.au> writes:

> On Tue, Sep 26, 2006 at 10:10:29PM -0700, Andrew Morton wrote:
>> On Tue, 26 Sep 2006 11:25:06 -0400
>> Valdis.Kletnieks@vt.edu wrote:
>> 
>> > OK, I'm running a Fedora Core 6 (rawhide actually) box with -18-mm1 kernel.
>> > I've installed kexec-tools and similar, and am trying to get the kernels
>> > built following the hints in Documentation/kdump/kdump.txt, but a few
>> > questions arise:
>> > 
>> > 1) Other than the fact that the Fedora userspace looks for a
>> > ${kernelvers}kdump kernel, is there any reason the kdump kernel has
>> > to match the running one, or can an older kernel be used?
>
> The post-crash kernel is not realy dependant on the pre-crash kernel.
> What is important is that either the kernel is relocatable
> (which is being worked on for x86 and i386), or it is compiled to
> run at a non-default address and that address corresponds
> to the region reserved by the crashkernel command line parameter
> passed to the pre-crash kernel.
>
> The post-crash kernel will also need CONFIG_CRASH_DUMP 
> and likely CONFIG_PROC_VMCORE

hip hip horray!  Getting mismatched kernels to be supported was a long
discussion.

I suspect the reason for the matching kernel from the distros is just
that everything is quite new so they don't want to wonder if you old
kernel has all of the appropriate support, and by that point they can
probably assume the shipped kernel works.

>> > 2) I'm presuming that a massively stripped down kernel (no sound support,
>> > no netfilter, no etc) that just has what's needed to mount the dump location
>> > is sufficient?
>
> Yes
>
>> > 3) The docs recommend 'crashkernel=64M@16M', but that's 8% of my memory.
>> > What will happen if I try '16M@16M' instead?  Just slower copying due to
>> > a smaller buffer cache space, or something more evil?
>
> There is a lower bound to how small you can make the space, which
> is basically how little memory space your post-crash kernel needs.
> 16M is probably pushing it, but 32M should be more than possible.
> Experimentation is really the order of the day here.

At that level I would say that below 32M is where you start dealing with
custom built programs, instead of slapping a bunch of utilities inside
a ramdisk.  I suspect with a little care you could get a few K user
space executable and fit everything inside of 4M.  But I don't know if anyone
is that ambitious yet.

Eric

Re: [Fastboot] Stupid kexec/kdump question...

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 2229 bytes --]

On Wed, 27 Sep 2006 02:00:07 MDT, Eric W. Biederman said:

> I suspect the reason for the matching kernel from the distros is just
> that everything is quite new so they don't want to wonder if you old
> kernel has all of the appropriate support, and by that point they can
> probably assume the shipped kernel works.

And careful reading of the Fedora scripts shows how to provide your
own custom kernel/initrd pair. Setting the right variable in the right
config file will use that - if it's unset, it will go with a kernel
named ${kernelversion}kdump.

> >> > 2) I'm presuming that a massively stripped down kernel (no sound support,
> >> > no netfilter, no etc) that just has what's needed to mount the dump location
> >> > is sufficient?
> >
> > Yes

I'm definitely submitting a doc patch to explain all this in more detail, as
soon as I get it figured out enough. ;)

> >> > 3) The docs recommend 'crashkernel=64M@16M', but that's 8% of my memory.
> >> > What will happen if I try '16M@16M' instead?  Just slower copying due to
> >> > a smaller buffer cache space, or something more evil?
> >
> > There is a lower bound to how small you can make the space, which
> > is basically how little memory space your post-crash kernel needs.
> > 16M is probably pushing it, but 32M should be more than possible.
> > Experimentation is really the order of the day here.
> 
> At that level I would say that below 32M is where you start dealing with
> custom built programs, instead of slapping a bunch of utilities inside
> a ramdisk.  I suspect with a little care you could get a few K user
> space executable and fit everything inside of 4M.  But I don't know if anyone
> is that ambitious yet.

Well, the stripped down kernel is right around 2M.  Unfortunately, I need
to run lvm.static, which is another 1.5M at least.  So unless busybox has
grown support for LVM, I'm looking at 8M at best.

Another stupid question - I see how the first kernel gets the 'crashkernel='
parameter and knows how much space there is.  But if you set it to 32M@16M,
how does the kdump kernel know it only has 32M?  Or does it just start at
the 16M line, and it's your job to make sure it doesn't go over the 48M line
and start corrupting the dump?


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: [Fastboot] Stupid kexec/kdump question...

[Vivek Goyal]

On Wed, Sep 27, 2006 at 08:51:50AM -0400, Valdis.Kletnieks@vt.edu wrote:
> On Wed, 27 Sep 2006 02:00:07 MDT, Eric W. Biederman said:
[..]
> > At that level I would say that below 32M is where you start dealing with
> > custom built programs, instead of slapping a bunch of utilities inside
> > a ramdisk.  I suspect with a little care you could get a few K user
> > space executable and fit everything inside of 4M.  But I don't know if anyone
> > is that ambitious yet.
> 
> Well, the stripped down kernel is right around 2M.  Unfortunately, I need
> to run lvm.static, which is another 1.5M at least.  So unless busybox has
> grown support for LVM, I'm looking at 8M at best.
> 
> Another stupid question - I see how the first kernel gets the 'crashkernel='
> parameter and knows how much space there is.  But if you set it to 32M@16M,
> how does the kdump kernel know it only has 32M?  Or does it just start at
> the 16M line, and it's your job to make sure it doesn't go over the 48M line
> and start corrupting the dump?
> 

Kdump kernel is booted with User defined memory map. For this purpose
kexec-tools passes memmap=exactmap option on command line to the second
kernel. Hence second kernel execution is bounded within reserved memory
region.

-Vivek

Re: [Fastboot] Stupid kexec/kdump question...

[Vivek Goyal]

On Wed, Sep 27, 2006 at 08:51:50AM -0400, Valdis.Kletnieks@vt.edu wrote:
> On Wed, 27 Sep 2006 02:00:07 MDT, Eric W. Biederman said:
> 
> > I suspect the reason for the matching kernel from the distros is just
> > that everything is quite new so they don't want to wonder if you old
> > kernel has all of the appropriate support, and by that point they can
> > probably assume the shipped kernel works.
> 
> And careful reading of the Fedora scripts shows how to provide your
> own custom kernel/initrd pair. Setting the right variable in the right
> config file will use that - if it's unset, it will go with a kernel
> named ${kernelversion}kdump.
> 
> > >> > 2) I'm presuming that a massively stripped down kernel (no sound support,
> > >> > no netfilter, no etc) that just has what's needed to mount the dump location
> > >> > is sufficient?
> > >
> > > Yes
> 
> I'm definitely submitting a doc patch to explain all this in more detail, as
> soon as I get it figured out enough. ;)
> 
> > >> > 3) The docs recommend 'crashkernel=64M@16M', but that's 8% of my memory.
> > >> > What will happen if I try '16M@16M' instead?  Just slower copying due to
> > >> > a smaller buffer cache space, or something more evil?
> > >
> > > There is a lower bound to how small you can make the space, which
> > > is basically how little memory space your post-crash kernel needs.
> > > 16M is probably pushing it, but 32M should be more than possible.
> > > Experimentation is really the order of the day here.
> > 
> > At that level I would say that below 32M is where you start dealing with
> > custom built programs, instead of slapping a bunch of utilities inside
> > a ramdisk.  I suspect with a little care you could get a few K user
> > space executable and fit everything inside of 4M.  But I don't know if anyone
> > is that ambitious yet.
> 
> Well, the stripped down kernel is right around 2M.  Unfortunately, I need
> to run lvm.static, which is another 1.5M at least.  So unless busybox has
> grown support for LVM, I'm looking at 8M at best.
> 

Distros don't want to build and ship an additional kernel just for dumping
purposes. They want to use production kernel as binary itself as dump
capture kernel. It reduces overheads for them significantly.

Currently we are reserving 64MB of memory for our testing purposes on 
i386 and x86_64. Any thing lesser than that, sometimes either system
hangs or OOM killer pitches in. Now people are working on preparing custom
initrd for dumping purposes so that dumping can be done from early user
space itself. This might lead to reduced memory usage and more reliability.
I think soon it should be available in fedora (busybox based custom initrd
for crashdumping purposes.)

-Vivek

Re: [Fastboot] Stupid kexec/kdump question...

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 721 bytes --]

On Wed, 27 Sep 2006 09:24:20 EDT, Vivek Goyal said:

> Currently we are reserving 64MB of memory for our testing purposes on 
> i386 and x86_64. Any thing lesser than that, sometimes either system
> hangs or OOM killer pitches in. Now people are working on preparing custom
> initrd for dumping purposes so that dumping can be done from early user
> space itself. This might lead to reduced memory usage and more reliability.
> I think soon it should be available in fedora (busybox based custom initrd
> for crashdumping purposes.)

I'm told the busybox-based initrd stuff has been pushed to Fedora Rawhide, but
it hasn't emerged yet - so most likely within the next 24 hours or so.  I'll
report back what happens... :)

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]