Question re: mem= usage and resultant vmcore

Question re: mem= usage and resultant vmcore

[Dave Anderson]

On an 4GB x86_64 kernel, with memory restricted with "mem=" like so:

   kernel /vmlinuz-2.6.18-36.el5 ro root=/dev/VolGroup00/LogVol00
   console=ttyS0,115200 mem=2000m crashkernel=128M@16M

The secondary kernel boots fine with this:

   Kernel command line: ro root=/dev/VolGroup00/LogVol00 console=ttyS0,115200
   mem=2000m  irqpoll maxcpus=1 memmap=exactmap memmap=640K@0K
   memmap=5048K@16384K memmap=125368K@22072K elfcorehdr=147440K
   memmap=76K#3406720K memmap=564K#3406796K

The /proc/vmcore shows 4GB:

   # ls -l /proc/vmcore
   -r-------- 1 root root 4164192032 Aug  1 10:57 /proc/vmcore
   #

I'm not sure whether that's supposed to reflect the "mem=2000m" size
or not?

Anyway, when copied to /var/crash, the "cp" command in the kdump init
file returns a 1, and the vmcore file is kept named as "vmcore-incomplete".

But the vmcore-incomplete looks to be the "right" size, and is functional
as a vmcore file:

   # crash /usr/lib/debug/lib/modules/2.6.18-36.el5/vmlinux vmcore-incomplete

   crash 4.0-4.3.1
   Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007  Red Hat, Inc.
   Copyright (C) 2004, 2005, 2006  IBM Corporation
   Copyright (C) 1999-2006  Hewlett-Packard Co
   Copyright (C) 2005, 2006  Fujitsu Limited
   Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
   Copyright (C) 2005  NEC Corporation
   Copyright (C) 1999, 2002  Silicon Graphics, Inc.
   Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
   This program is free software, covered by the GNU General Public License,
   and you are welcome to change it and/or distribute copies of it under
   certain conditions.  Enter "help copying" to see the conditions.
   This program has absolutely no warranty.  Enter "help warranty" for details.

   GNU gdb 6.1
   Copyright 2004 Free Software Foundation, Inc.
   GDB is free software, covered by the GNU General Public License, and you are
   welcome to change it and/or distribute copies of it under certain conditions.
   Type "show copying" to see the conditions.
   There is absolutely no warranty for GDB.  Type "show warranty" for details.
   This GDB was configured as "x86_64-unknown-linux-gnu"...

         KERNEL: /usr/lib/debug/lib/modules/2.6.18-36.el5/vmlinux
       DUMPFILE: vmcore-incomplete
           CPUS: 4
           DATE: Wed Aug  1 11:47:13 2007
         UPTIME: 00:02:15
   LOAD AVERAGE: 0.16, 0.14, 0.06
          TASKS: 118
       NODENAME: nec-em17.rhts.boston.redhat.com
        RELEASE: 2.6.18-36.el5
        VERSION: #1 SMP Fri Jul 20 14:26:46 EDT 2007
        MACHINE: x86_64  (2992 Mhz)
         MEMORY: 1.9 GB
          PANIC: "SysRq : Trigger a crashdump"
            PID: 3044
        COMMAND: "bash"
           TASK: ffff8100794f0100  [THREAD_INFO: ffff8100656dc000]
            CPU: 0
          STATE: TASK_RUNNING (SYSRQ)

   crash>

So two questions -- should the /proc/vmcore on the secondary kernel
reflect the restricted size, and what would cause the "cp" to
only copy the "correct" amount, while returning an error code?

Dave



_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Question re: mem= usage and resultant vmcore

[Vivek Goyal]

On Wed, Aug 01, 2007 at 12:29:59PM -0400, Dave Anderson wrote:
> 
> On an 4GB x86_64 kernel, with memory restricted with "mem=" like so:
> 
>    kernel /vmlinuz-2.6.18-36.el5 ro root=/dev/VolGroup00/LogVol00
>    console=ttyS0,115200 mem=2000m crashkernel=128M@16M
> 
> The secondary kernel boots fine with this:
> 
>    Kernel command line: ro root=/dev/VolGroup00/LogVol00 console=ttyS0,115200
>    mem=2000m  irqpoll maxcpus=1 memmap=exactmap memmap=640K@0K
>    memmap=5048K@16384K memmap=125368K@22072K elfcorehdr=147440K
>    memmap=76K#3406720K memmap=564K#3406796K
> 
> The /proc/vmcore shows 4GB:
> 
>    # ls -l /proc/vmcore
>    -r-------- 1 root root 4164192032 Aug  1 10:57 /proc/vmcore
>    #
> 
> I'm not sure whether that's supposed to reflect the "mem=2000m" size
> or not?
> 

Hi Dave,

/proc/iomem on i386 and x86_64 behave differently when passed with mem=
parameter. I think on i386, only memory specified by mem= is visible but
in case of x86_64, all the memory passed by BIOS to kernel is visible.

Kexec/kdump retrieve the physical memory info from /proc/iomem. We need
both the behaviours for two scenarios.

- For kexec, we want to see whole of the memory (irrespective of the fact
  how much current kernel is using), so that next kernel can potentially
  use all the memory to boot in.

- For kdump, we want to know about only the physical memory current kernel
  is using and not all the memory system has.

Here the issue is that despite the fact you have passed mem=2000m, /proc/iomem
will show 4G physical memory and kdump will create elf header for 4G of 
memory. That's why /proc/vmcore size is 4G. I am not sure why it did not
copy whole 4G on disk and stoppped after 2000m. To me it should have copied
whole of the 4G.

Bottom line, we need to do some work in this area.

- Make /proc/iomem behaviour consistent across i386 and x86_64. I think
  it can be changed to reflect the physical memory currently used by 
  kernel (based on mem=) parameter.

- Create another /proc interface, lets say /proc/physmem, which will reflect
  all the physical memory system has, irrespective of the fact what is being
  used by the kernel.

In this case kexec can make use of /proc/physmem and kdump can make use
of /proc/iomem and things will be fine.

Anybody interested in writing patches for this?

Thanks
Vivek

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Question re: mem= usage and resultant vmcore

[Dave Anderson]

Vivek Goyal wrote:
> On Wed, Aug 01, 2007 at 12:29:59PM -0400, Dave Anderson wrote:
> 
>>On an 4GB x86_64 kernel, with memory restricted with "mem=" like so:
>>
>>   kernel /vmlinuz-2.6.18-36.el5 ro root=/dev/VolGroup00/LogVol00
>>   console=ttyS0,115200 mem=2000m crashkernel=128M@16M
>>
>>The secondary kernel boots fine with this:
>>
>>   Kernel command line: ro root=/dev/VolGroup00/LogVol00 console=ttyS0,115200
>>   mem=2000m  irqpoll maxcpus=1 memmap=exactmap memmap=640K@0K
>>   memmap=5048K@16384K memmap=125368K@22072K elfcorehdr=147440K
>>   memmap=76K#3406720K memmap=564K#3406796K
>>
>>The /proc/vmcore shows 4GB:
>>
>>   # ls -l /proc/vmcore
>>   -r-------- 1 root root 4164192032 Aug  1 10:57 /proc/vmcore
>>   #
>>
>>I'm not sure whether that's supposed to reflect the "mem=2000m" size
>>or not?
>>
> 
> 
> Hi Dave,
> 
> /proc/iomem on i386 and x86_64 behave differently when passed with mem=
> parameter. I think on i386, only memory specified by mem= is visible but
> in case of x86_64, all the memory passed by BIOS to kernel is visible.
> 
> Kexec/kdump retrieve the physical memory info from /proc/iomem. We need
> both the behaviours for two scenarios.
> 
> - For kexec, we want to see whole of the memory (irrespective of the fact
>   how much current kernel is using), so that next kernel can potentially
>   use all the memory to boot in.
> 
> - For kdump, we want to know about only the physical memory current kernel
>   is using and not all the memory system has.
> 
> Here the issue is that despite the fact you have passed mem=2000m, /proc/iomem
> will show 4G physical memory and kdump will create elf header for 4G of 
> memory. That's why /proc/vmcore size is 4G. I am not sure why it did not
> copy whole 4G on disk and stoppped after 2000m. To me it should have copied
> whole of the 4G.

Yeah -- I haven't verified it, but I'm guessing that read_vmcore()
fails due to its call to map_offset_to_paddr(), which doesn't
find the physical memory beyond 2000m in the vmcore_list?

Interestingly enough, this may never have been noticed except for
the save_core() function in the /etc/init.d/kdump file in Red Hat's
kexec-tools package:

function save_core()
{
         coredir="/var/crash/`date +"%Y-%m-%d-%H:%M"`"

         mkdir -p $coredir
         cp /proc/vmcore $coredir/vmcore-incomplete
         exitcode=$?
         if [ $exitcode == 0 ]; then
                 mv $coredir/vmcore-incomplete $coredir/vmcore
                 $LOGGER "saved a vmcore to $coredir"
         else
                 $LOGGER "failed to save a vmcore to $coredir"
         fi
         return $exitcode
}

And even though the ELF header reflects the 4GB memory size,
the vmcore-incomplete file is "complete" w/respect to the
primary kernel.  In other words, even though the ELF header
advertises non-existent physical memory beyond the 2000m
limit, there's never a need to access it from the crash utility,
so it's kind of a benign bug.

Dave


> 
> Bottom line, we need to do some work in this area.
> 
> - Make /proc/iomem behaviour consistent across i386 and x86_64. I think
>   it can be changed to reflect the physical memory currently used by 
>   kernel (based on mem=) parameter.
> 
> - Create another /proc interface, lets say /proc/physmem, which will reflect
>   all the physical memory system has, irrespective of the fact what is being
>   used by the kernel.
> 
> In this case kexec can make use of /proc/physmem and kdump can make use
> of /proc/iomem and things will be fine.
> 
> Anybody interested in writing patches for this?
> 
> Thanks
> Vivek



_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Question re: mem= usage and resultant vmcore

[Dave Anderson]

> The /proc/vmcore shows 4GB:
> 
>   # ls -l /proc/vmcore
>   -r-------- 1 root root 4164192032 Aug  1 10:57 /proc/vmcore
>   #

I forgot to mention that the resultant, usable, vmcore-incomplete
files are consistently the same size, reflecting the mem=2000m
kernel option:

   # ls -l
   total 435020
   -r-------- 1 root root 1967558656 Aug  1 11:47 vmcore-incomplete
   #

So, is this a bug or a feature?

Dave



_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec