Re: Can't kexec/kdump on mpc85xx

[Makito SHIOKAWA <mshiokawa@miraclelinux.com>]

(2010/09/29 0:47), Matthew McClintock wrote:

> Can you try this patch below when building kexec-tools? I ran in to an issue with my cross compiler...

Second kernel has booted normally on both "kexec -l", "kexec -p" with your 
patch! I've been working for this long time and could never find out this 
issue, big thanks!

Now, next is to make CONFIG_CRASH_DUMP, CONFIG_PROC_VMCORE settable and get a 
dump from /proc/vmcore.
So, as a trial, I made these option settable on MPC8548CDS forcibly.

---
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -368,7 +368,7 @@ config KEXEC

  config CRASH_DUMP
         bool "Build a kdump crash kernel"
-       depends on PPC64 || 6xx
+       depends on PPC64 || 6xx || FSL_BOOKE
         select RELOCATABLE if PPC64
         help
           Build a kernel suitable for use as a kdump capture kernel.
--- a/arch/powerpc/kernel/crash_dump.c
+++ b/arch/powerpc/kernel/crash_dump.c
@@ -128,7 +128,7 @@ ssize_t copy_oldmem_page(unsigned long p
         if (!csize)
                 return 0;

-       csize = min(csize, PAGE_SIZE);
+       csize = min(csize, (size_t)PAGE_SIZE);
---

And, set CONFIG_CRASH_DUMP, CONFIG_PROC_VMCORE and modified 
CONFIG_KERNEL_START to 0xc0000000. (CONFIG_KERNEL_START is set to 0xc2000000 
with CONFIG_CRASH_DUMP enabled and first kernel didn't boot with that value)

---
--- .config.bak 2010-09-14 06:32:36.000000000 +0900
+++ .config     2010-09-29 07:25:14.000000000 +0900
@@ -1,7 +1,7 @@
  #
  # Automatically generated make config: don't edit
  # Linux kernel version: 2.6.35
-# Tue Sep  7 08:52:12 2010
+# Wed Sep 29 07:25:14 2010
  #
  # CONFIG_PPC64 is not set

@@ -285,6 +285,7 @@ CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
  CONFIG_ARCH_HAS_WALK_MEMORY=y
  CONFIG_ARCH_ENABLE_MEMORY_HOTREMOVE=y
  CONFIG_KEXEC=y
+CONFIG_CRASH_DUMP=y
  CONFIG_SPARSE_IRQ=y
  CONFIG_MAX_ACTIVE_REGIONS=32
  CONFIG_ARCH_FLATMEM_ENABLE=y
@@ -351,7 +352,7 @@ CONFIG_LOWMEM_CAM_NUM=3
  CONFIG_RELOCATABLE=y
  # CONFIG_PAGE_OFFSET_BOOL is not set
  CONFIG_PAGE_OFFSET=0xc0000000
-# CONFIG_KERNEL_START_BOOL is not set
+CONFIG_KERNEL_START_BOOL=y
  CONFIG_KERNEL_START=0xc0000000
  # CONFIG_PHYSICAL_START_BOOL is not set
  CONFIG_PHYSICAL_START=0x00000000
@@ -1290,6 +1291,7 @@ CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
  #
  CONFIG_PROC_FS=y
  CONFIG_PROC_KCORE=y
+CONFIG_PROC_VMCORE=y
  CONFIG_PROC_SYSCTL=y
  CONFIG_PROC_PAGE_MONITOR=y
  CONFIG_SYSFS=y
---

But, after booting with this first kernel, /proc/vmcore still doesn't exist 
and this time second kernel gets panic on "kexec -p". ("kexec -l" works fine)

---
# kexec -p --command-line="console=ttyS0,38400 ip=bootp root=/dev/nfs rw maxcpus
=1 noirqdistrib reset_devices" /boot/vmlinux
0000000000000000-0000000040000000 : 0
get base memory ranges:1
usable memory rgns size:1 base:0 size:30000000
exclude_range sorted exclude_range[0] start:0, end:41d000
setup_memory_ranges memory_range[0] start:41d001, end:30000000
CRASH MEMORY RANGES
0000000000000000-0000000000010000
0000000000010000-0000000020000000
0000000030000000-0000000040000000
Elf header: p_type = 4, p_offset = 0x1f880400 p_paddr = 0x1f880400 p_vaddr = 0x0
  p_filesz = 0x400 p_memsz = 0x400
vmcoreinfo header: p_type = 4, p_offset = 0x4064f4 p_paddr = 0x4064f4 p_vaddr =
0x0 p_filesz = 0x1000 p_memsz = 0x1000
Elf header: p_type = 1, p_offset = 0x2041d000 p_paddr = 0x0 p_vaddr = 0xc0000000
  p_filesz = 0x10000 p_memsz = 0x10000
Elf header: p_type = 1, p_offset = 0x10000 p_paddr = 0x10000 p_vaddr = 0xc001000
0 p_filesz = 0x1fff0000 p_memsz = 0x1fff0000
Elf header: p_type = 1, p_offset = 0x30000000 p_paddr = 0x30000000 p_vaddr = 0xf
fffffff p_filesz = 0x10000000 p_memsz = 0x10000000
Command line after adding elfcorehdr:  elfcorehdr=528564K
Command line after adding elfcorehdr:  elfcorehdr=528564K savemaxmem=1024M
sym: sha256_starts info: 12 other: 00 shndx: 1 value: 127c size: e0
sym: sha256_starts value: 2042f27c addr: 2042e01c
sym: sha256_regions info: 11 other: 00 shndx: 4 value: 0 size: 100
sym: sha256_regions value: 20434788 addr: 2042e022
sym: sha256_regions info: 11 other: 00 shndx: 4 value: 0 size: 100
sym: sha256_regions value: 20434788 addr: 2042e026
sym: sha256_regions info: 11 other: 00 shndx: 4 value: 0 size: 100
sym: sha256_regions value: 20434688 addr: 2042e02e
sym: sha256_regions info: 11 other: 00 shndx: 4 value: 0 size: 100
sym: sha256_regions value: 20434688 addr: 2042e032
sym: sha256_update info: 12 other: 00 shndx: 1 value: 5f3c size: 1b0
sym: sha256_update value: 20433f3c addr: 2042e078
sym: sha256_finish info: 12 other: 00 shndx: 1 value: 60ec size: 528
sym: sha256_finish value: 204340ec addr: 2042e0a8
sym: sha256_digest info: 11 other: 00 shndx: 4 value: 100 size: 20
sym: sha256_digest value: 20434788 addr: 2042e0b6
sym: sha256_digest info: 11 other: 00 shndx: 4 value: 100 size: 20
sym: sha256_digest value: 20434788 addr: 2042e0ba
sym:     memcmp info: 12 other: 00 shndx: 1 value: f38 size: bc
sym: memcmp value: 2042ef38 addr: 2042e0c0
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 20434614 addr: 2042e0d2
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 20434614 addr: 2042e0d6
sym:     printf info: 12 other: 00 shndx: 1 value: d10 size: a4
sym: printf value: 2042ed10 addr: 2042e0dc
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 20434634 addr: 2042e0e2
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 20434634 addr: 2042e0e6
sym:     printf info: 12 other: 00 shndx: 1 value: d10 size: a4
sym: printf value: 2042ed10 addr: 2042e0ec
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 20434644 addr: 2042e116
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 20434644 addr: 2042e11a
sym:     printf info: 12 other: 00 shndx: 1 value: d10 size: a4
sym: printf value: 2042ed10 addr: 2042e124
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 2043464c addr: 2042e142
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 2043464c addr: 2042e146
sym:     printf info: 12 other: 00 shndx: 1 value: d10 size: a4
sym: printf value: 2042ed10 addr: 2042e14c
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 20434650 addr: 2042e152
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 20434650 addr: 2042e156
sym:     printf info: 12 other: 00 shndx: 1 value: d10 size: a4
sym: printf value: 2042ed10 addr: 2042e15c
sym: sha256_digest info: 11 other: 00 shndx: 4 value: 100 size: 20
sym: sha256_digest value: 20434788 addr: 2042e172
sym: sha256_digest info: 11 other: 00 shndx: 4 value: 100 size: 20
sym: sha256_digest value: 20434788 addr: 2042e176
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 20434644 addr: 2042e182
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 20434644 addr: 2042e186
sym:     printf info: 12 other: 00 shndx: 1 value: d10 size: a4
sym: printf value: 2042ed10 addr: 2042e190
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 2043464c addr: 2042e1ae
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 2043464c addr: 2042e1b2
sym:     printf info: 12 other: 00 shndx: 1 value: d10 size: a4
sym: printf value: 2042ed10 addr: 2042e1b8
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 20434660 addr: 2042e1ee
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 20434660 addr: 2042e1f2
sym:     printf info: 12 other: 00 shndx: 1 value: d10 size: a4
sym: printf value: 2042ed10 addr: 2042e1f8
sym: setup_arch info: 12 other: 00 shndx: 1 value: 11e0 size: 1c
sym: setup_arch value: 2042f1e0 addr: 2042e1fc
sym: verify_sha256_digest info: 12 other: 00 shndx: 1 value: 0 size: 1d8
sym: verify_sha256_digest value: 2042e000 addr: 2042e200
sym: post_verification_setup_arch info: 12 other: 00 shndx: 1 value: 11fc size:
44
sym: post_verification_setup_arch value: 2042f1fc addr: 2042e204
sym:    putchar info: 12 other: 00 shndx: 1 value: 125c size: 20
sym: putchar value: 2042f25c addr: 2042e2b8
sym:    putchar info: 12 other: 00 shndx: 1 value: 125c size: 20
sym: putchar value: 2042f25c addr: 2042e3b0
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 20434674 addr: 2042e72a
sym:    .rodata info: 03 other: 00 shndx: 3 value: 0 size: 0
sym: .rodata value: 20434674 addr: 2042e72e
sym:    putchar info: 12 other: 00 shndx: 1 value: 125c size: 20
sym: putchar value: 2042f25c addr: 2042ebe4
sym:   vsprintf info: 12 other: 00 shndx: 1 value: 220 size: a4c
sym: vsprintf value: 2042e220 addr: 2042ecf4
sym:   vsprintf info: 12 other: 00 shndx: 1 value: 220 size: a4c
sym: vsprintf value: 2042e220 addr: 2042ed98
sym:      stack info: 11 other: 00 shndx: 5 value: 8 size: 4
sym: stack value: 204347f0 addr: 2042f106
sym:      stack info: 11 other: 00 shndx: 5 value: 8 size: 4
sym: stack value: 204347f0 addr: 2042f10a
sym:  purgatory info: 12 other: 00 shndx: 1 value: 1d8 size: 48
sym: purgatory value: 2042e1d8 addr: 2042f114
sym:  dt_offset info: 11 other: 00 shndx: 5 value: c size: 4
sym: dt_offset value: 204347f4 addr: 2042f12e
sym:  dt_offset info: 11 other: 00 shndx: 5 value: c size: 4
sym: dt_offset value: 204347f4 addr: 2042f132
sym:     kernel info: 11 other: 00 shndx: 5 value: 14 size: 4
sym: kernel value: 204347fc addr: 2042f14a
sym:     kernel info: 11 other: 00 shndx: 5 value: 14 size: 4
sym: kernel value: 204347fc addr: 2042f14e
sym: panic_kernel info: 11 other: 00 shndx: 5 value: 0 size: 4
sym: panic_kernel value: 204347e8 addr: 2042f212
sym: panic_kernel info: 11 other: 00 shndx: 5 value: 0 size: 4
sym: panic_kernel value: 204347e8 addr: 2042f21a
sym: crashdump_backup_memory info: 12 other: 00 shndx: 1 value: 1240 size: 1c
sym: crashdump_backup_memory value: 2042f240 addr: 2042f224
sym:     memcpy info: 12 other: 00 shndx: 1 value: ea8 size: 90
sym: memcpy value: 2042eea8 addr: 2043402c
sym: sha256_process info: 12 other: 00 shndx: 1 value: 135c size: 4be0
sym: sha256_process value: 2042f35c addr: 20434040
sym: sha256_process info: 12 other: 00 shndx: 1 value: 135c size: 4be0
sym: sha256_process value: 2042f35c addr: 20434078
sym:     memcpy info: 12 other: 00 shndx: 1 value: ea8 size: 90
sym: memcpy value: 2042eea8 addr: 204340c8
sym:      .data info: 03 other: 00 shndx: 4 value: 0 size: 0
sym: .data value: 204347a8 addr: 204341fa
sym:      .data info: 03 other: 00 shndx: 4 value: 0 size: 0
sym: .data value: 204347a8 addr: 204341fe
sym: sha256_update info: 12 other: 00 shndx: 1 value: 5f3c size: 1b0
sym: sha256_update value: 20433f3c addr: 20434204
sym: sha256_update info: 12 other: 00 shndx: 1 value: 5f3c size: 1b0
sym: sha256_update value: 20433f3c addr: 20434218
Modified cmdline:console=ttyS0,38400 ip=bootp root=/dev/nfs rw maxcpus=1 noirqdi
strib reset_devices elfcorehdr=528564K savemaxmem=1024M
reserve regions: 2
0: offset: 217fd000, size: 3000
1: offset: 20000000, size: 435000
debug.dtb written
# echo c > /proc/sysrq-trigger
SysRq : Trigger a crash
Unable to handle kernel paging request for data at address 0x00000000
Faulting instruction address: 0xc01b8044
Oops: Kernel access of bad area, sig: 11 [#1]
MPC85xx CDS
last sysfs file: /sys/devices/system/cpu/cpu0/crash_notes
NIP: c01b8044 LR: c01b830c CTR: c01b8030
REGS: dfbbfdf0 TRAP: 0300   Not tainted  (2.6.35)
MSR: 00021000 <ME,CE>  CR: 22002442  XER: 20000000
DEAR: 00000000, ESR: 00800000
TASK = df9a6e40[891] 'bash' THREAD: dfbbe000
GPR00: 00000001 dfbbfea0 df9a6e40 00000063 00000000 ffffffff c01bf3a0 00000000
GPR08: 00002d02 00000000 00000010 00000000 22002422 1009cc80 10080000 10095a10
GPR16: 10090000 10090000 bfd0ca70 00000000 100956b0 10090000 100956c0 bfd0ca60
GPR24: 00000000 00000007 c03e5490 00029000 c03e558c c03e0000 00000000 00000063
NIP [c01b8044] sysrq_handle_crash+0x14/0x20
LR [c01b830c] __handle_sysrq+0xbc/0x1c0
Call Trace:
[dfbbfea0] [c01b82f4] __handle_sysrq+0xa4/0x1c0 (unreliable)
[dfbbfed0] [c01b846c] write_sysrq_trigger+0x5c/0x70
[dfbbfee0] [c00f1cec] proc_reg_write+0x4c/0x70
[dfbbfef0] [c00a94d0] vfs_write+0xc0/0x1a0
[dfbbff10] [c00a96ac] sys_write+0x4c/0x90
[dfbbff40] [c0010594] ret_from_syscall+0x0/0x3c
--- Exception: c01 at 0xfe1d758
     LR = 0xfdb93ec
Instruction dump:
7d6b0078 7d60492d 40a2fff4 807e0018 7c7b1b78 4bfffefc 00000000 3d20c040
38000001 9009d12c 7c0004ac 39200000 <98090000> 4e800020 60000000 3803ffd0
Bye!
Using MPC85xx CDS machine description
Memory CAM mapping: 256 Mb, residual: 0Mb
Linux version 2.6.35 (root@ppcdev) (gcc version 4.4.1 (Sourcery G++ Lite 4.4-254
) ) #40 Wed Sep 29 07:31:35 JST 2010
bootconsole [udbg0] enabled
setup_arch: bootmem
mpc85xx_cds_setup_arch()
CDS Version = 0x64 in slot 2

Found FSL PCI host bridge at 0x00000000d0008000. Firmware bus number: 0->0
PCI host bridge /pci@d0008000 (primary) ranges:
  MEM 0x0000000080000000..0x00000000bfffffff -> 0x0000000080000000
   IO 0x00000000d2000000..0x00000000d5ffffff -> 0x0000000000000000
/pci@d0008000: PCICSRBAR @ 0xfff00000
arch: exit
Zone PFN ranges:
   DMA      0x00020000 -> 0x00030000
   Normal   empty
Movable zone start PFN for each node
early_node_map[1] active PFN ranges
     0: 0x00020000 -> 0x00030000
MMU: Allocated 1088 bytes of context maps for 255 contexts
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 65024
Kernel command line: console=ttyS0,38400 ip=bootp root=/dev/nfs rw maxcpus=1 noi
rqdistrib reset_devices elfcorehdr=528564K savemaxmem=1024M
PID hash table entries: 1024 (order: 0, 4096 bytes)
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Memory: 255492k/262144k available (3948k kernel code, 6652k reserved, 136k data,
  126k bss, 160k init)
Kernel virtual memory layout:
   * 0xfffdf000..0xfffff000  : fixmap
   * 0xfcffb000..0xfe000000  : early ioremap
   * 0xd1000000..0xfcffb000  : vmalloc & ioremap
SLUB: Genslabs=13, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Hierarchical RCU implementation.
         RCU-based detection of stalled CPUs is disabled.
         Verbose stalled-CPUs detection is disabled.
NR_IRQS:512 nr_irqs:512
mpic: Setting up MPIC " OpenPIC  " version 1.2 at d0040000, max 1 CPUs
mpic: ISU size: 80, shift: 7, mask: 7f
mpic: Initializing for 80 sources
clocksource: timebase mult[3c9b26d] shift[22] registered
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 512
NET: Registered protocol family 16

PCI: Probing PCI hardware
bio: create slab <bio-0> at 0
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
Switching to clocksource timebase
NET: Registered protocol family 2
IP route cache hash table entries: 2048 (order: 1, 8192 bytes)
TCP established hash table entries: 8192 (order: 4, 65536 bytes)
TCP bind hash table entries: 8192 (order: 3, 32768 bytes)
TCP: Hash tables configured (established 8192 bind 8192)
TCP reno registered
UDP hash table entries: 256 (order: 0, 4096 bytes)
UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
NET: Registered protocol family 1
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
i8259 legacy interrupt controller initialized
Unable to handle kernel paging request for data at address 0xbf880400
Faulting instruction address: 0xc00166a8
Oops: Kernel access of bad area, sig: 11 [#1]
MPC85xx CDS
last sysfs file:
NIP: c00166a8 LR: c000b9fc CTR: 00000080
REGS: cf82fd80 TRAP: 0300   Not tainted  (2.6.35)
MSR: 00029000 <EE,ME,CE>  CR: 24002442  XER: 20000000
DEAR: bf880400, ESR: 00000000
TASK = cf830000[1] 'swapper' THREAD: cf82e000
GPR00: 00000000 cf82fe30 cf830000 cf8ed800 bf8803fc 00000400 cf8ed7fc 00000080
GPR08: 00000007 20000000 2042d0d4 c03f4d70 ffffffff 1009cc80 10080000 20435000
GPR16: 000000d4 cf82fe80 00000400 00000000 100956b0 10090000 100956c0 00000000
GPR24: 00000000 cf82fe80 00000000 00000000 0001f880 cf8ed800 00000000 00000400
NIP [c00166a8] memcpy+0x1c/0x9c
LR [c000b9fc] copy_oldmem_vaddr+0xbc/0xe0
Call Trace:
[cf82fe30] [cf8ed800] 0xcf8ed800 (unreliable)
[cf82fe40] [c00fbb5c] read_from_oldmem+0x7c/0xe0
[cf82fe70] [c03c400c] parse_crash_elf32_headers+0x1a4/0x470
[cf82ff30] [c03c4834] vmcore_init+0x310/0x44c
[cf82ffb0] [c0001dcc] do_one_initcall+0x3c/0x1f0
[cf82ffe0] [c03b31e4] kernel_init+0xc4/0x168
[cf82fff0] [c000ffec] kernel_thread+0x4c/0x68
Instruction dump:
38c60001 4200fff0 4e800020 7c032040 418100a0 54a7e8ff 38c3fffc 3884fffc
41820028 70c00003 7ce903a6 40820054 <80e40004> 85040008 90e60004 95060008
---[ end trace 5e5d343531be722e ]---
Kernel panic - not syncing: Attempted to kill init!
Rebooting in 180 seconds..
---

It'll be very helpful again if you notice something.
Thanks,

-- 
Makito SHIOKAWA　<mshiokawa@miraclelinux.com>

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec