how to generate a smaller core with xm dump-core

how to generate a smaller core with xm dump-core

[Zhenzhong Duan]

[-- Attachment #1.1: Type: text/plain, Size: 956 bytes --]

Hi Maintainers,

We are facing issue collecting  coredump using the xm dump mechanism in
Dom0.

We face couple of such issues daily, where the VMs panic s and the SA
team is supposed to collect the core.

The actual problem is that where the VMs are having huge RAM like 32+ GB
RAMs dumping the core using *xm dump* generated core of almost the same
size.

And also when the RAM size is huge the time taken to dump is also huge.

Transferring this core dump of this huge size to another system for
analysis is really a pain. I am looking for some solution where we can
generate smaller core using *xm dump* or any other tool.

Something similar to *makedumpfile* which eliminates the zero and
unnecessary pages, which results in core of smaller size.

This has become a hot issue and we terribly need this feature.

Can you come across with anything or are you aware of any feature which
could cater our need….?

thanks
zduan

[-- Attachment #1.2: Type: text/html, Size: 1562 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: how to generate a smaller core with xm dump-core

[Ian Campbell]

On Thu, 2015-01-15 at 11:31 +0800, Zhenzhong Duan wrote:
> Hi Maintainers,
> 
> 
> We are facing issue collecting  coredump using the xm dump mechanism
> in Dom0.
> 
> We face couple of such issues daily, where the VMs panic s and the SA
> team is supposed to collect the core.
> 
> The actual problem is that where the VMs are having huge RAM like 32+
> GB RAMs dumping the core using xm dump generated core of almost the
> same size. 
> 
> And also when the RAM size is huge the time taken to dump is also
> huge.
> 
> Transferring this core dump of this huge size to another system for
> analysis is really a pain. I am looking for some solution where we can
> generate smaller core using xm dump or any other tool.
> 
> Something similar to makedumpfile which eliminates the zero and
> unnecessary pages, which results in core of smaller size.
> 
>  
> 
> This has become a hot issue and we terribly need this feature.
> 
> Can you come across with anything or are you aware of any feature
> which could cater our need….?

I'm not aware of any existing features or efforts in this area.

If things like zero pages can be omitted from a core file without making
it non-standard then I think we'd be happy to see such patches to the
core libxc functions which produce the core files.

That wouldn't help much with a heavily loaded VM which is making good
use of all its ram though.

I've not looked but it's not unlikely that the dumping code itself is
not very optimal, so it might be worth having a look to see if more
batching etc might help speed things up at the root. Ages ago there were
patches floating around which improved the performance of the privcmd
mmap ioctl (from Mats Peterson), which might help here too (assuming
core dumping uses those interfaces as I expect). AFAIK those patches
were abandoned when Mats moved onto other things, resurrecting them
would probably help here as well as with migration and other
foreign-memory-man heavy workloads.

Other random thoughts:
      * Automatically compress the core file, doesn't help with time to
        produce the dump (quite the opposite) but helps with copying it
        off box (but could also compress manually before moving or do it
        as a post processing step)
      * Have an option to produce some kind of "minicore", e.g. just
        register state and a few pages of the referenced stacks, perhaps
        a few pages of RAM either side of any register which happens to
        be a valid looking pointer.
              * Care would be needed to retain an acceptable probability
                that the resulting core will actually be useful.
      * Consider including kernel RAM only (which might still be huge
        though if it has a 1:1 mapping of all RAM).

I don't think any of those would be suitable for enabling by default but
they might be useful to have in the arsenal.

HTH,

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: how to generate a smaller core with xm dump-core

[Don Slutz]

On 01/15/15 05:20, Ian Campbell wrote:
> On Thu, 2015-01-15 at 11:31 +0800, Zhenzhong Duan wrote:
>> Hi Maintainers,
>>
>>
>> We are facing issue collecting  coredump using the xm dump mechanism
>> in Dom0.
>>
>> We face couple of such issues daily, where the VMs panic s and the SA
>> team is supposed to collect the core.
>>
>> The actual problem is that where the VMs are having huge RAM like 32+
>> GB RAMs dumping the core using xm dump generated core of almost the
>> same size.
>>
>> And also when the RAM size is huge the time taken to dump is also
>> huge.
>>
>> Transferring this core dump of this huge size to another system for
>> analysis is really a pain. I am looking for some solution where we can
>> generate smaller core using xm dump or any other tool.
>>
>> Something similar to makedumpfile which eliminates the zero and
>> unnecessary pages, which results in core of smaller size.
>>
>>   
>>
>> This has become a hot issue and we terribly need this feature.
>>
>> Can you come across with anything or are you aware of any feature
>> which could cater our need….?
> I'm not aware of any existing features or efforts in this area.
>
> If things like zero pages can be omitted from a core file without making
> it non-standard then I think we'd be happy to see such patches to the
> core libxc functions which produce the core files.
>
> That wouldn't help much with a heavily loaded VM which is making good
> use of all its ram though.
>
> I've not looked but it's not unlikely that the dumping code itself is
> not very optimal, so it might be worth having a look to see if more
> batching etc might help speed things up at the root. Ages ago there were
> patches floating around which improved the performance of the privcmd
> mmap ioctl (from Mats Peterson), which might help here too (assuming
> core dumping uses those interfaces as I expect). AFAIK those patches
> were abandoned when Mats moved onto other things, resurrecting them
> would probably help here as well as with migration and other
> foreign-memory-man heavy workloads.
>
> Other random thoughts:
>        * Automatically compress the core file, doesn't help with time to
>          produce the dump (quite the opposite) but helps with copying it
>          off box (but could also compress manually before moving or do it
>          as a post processing step)
>        * Have an option to produce some kind of "minicore", e.g. just
>          register state and a few pages of the referenced stacks, perhaps
>          a few pages of RAM either side of any register which happens to
>          be a valid looking pointer.
>                * Care would be needed to retain an acceptable probability
>                  that the resulting core will actually be useful.
>        * Consider including kernel RAM only (which might still be huge
>          though if it has a 1:1 mapping of all RAM).
>
> I don't think any of those would be suitable for enabling by default but
> they might be useful to have in the arsenal.
>
> HTH,
>
> Ian.
>

If these are Linux guests then the patch:

http://lists.xenproject.org/archives/html/xen-devel/2013-11/msg02351.html

Can be used to enable crash to access the crashed guest and collect
some basic info. I would also include the output of xen-hvmctx and/or 
xenctx.
-- This is the quick way I would do Ian's minicore .

Note: dump-core currently does not include xen-hvmctx output (nor does 
it include
xenctx -a output).

Using the results from this may allow you to not need a copy of every 
dump (
not to imply that 2 similar minicore's would insure that a copy of each core
was not needed).

I also think that makedumpfile can process a xen dump-core file and make it
smaller.

     -Don Slutz

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: how to generate a smaller core with xm dump-core

[Zhenzhong Duan]

在 2015/1/16 0:16, Don Slutz 写道:
> On 01/15/15 05:20, Ian Campbell wrote:
>> On Thu, 2015-01-15 at 11:31 +0800, Zhenzhong Duan wrote:
>>> Hi Maintainers,
>>>
>>>
>>> We are facing issue collecting  coredump using the xm dump mechanism
>>> in Dom0.
>>>
>>> We face couple of such issues daily, where the VMs panic s and the SA
>>> team is supposed to collect the core.
>>>
>>> The actual problem is that where the VMs are having huge RAM like 32+
>>> GB RAMs dumping the core using xm dump generated core of almost the
>>> same size.
>>>
>>> And also when the RAM size is huge the time taken to dump is also
>>> huge.
>>>
>>> Transferring this core dump of this huge size to another system for
>>> analysis is really a pain. I am looking for some solution where we can
>>> generate smaller core using xm dump or any other tool.
>>>
>>> Something similar to makedumpfile which eliminates the zero and
>>> unnecessary pages, which results in core of smaller size.
>>>
>>>
>>> This has become a hot issue and we terribly need this feature.
>>>
>>> Can you come across with anything or are you aware of any feature
>>> which could cater our need….?
>> I'm not aware of any existing features or efforts in this area.
>>
>> If things like zero pages can be omitted from a core file without making
>> it non-standard then I think we'd be happy to see such patches to the
>> core libxc functions which produce the core files.
>>
>> That wouldn't help much with a heavily loaded VM which is making good
>> use of all its ram though.
>>
>> I've not looked but it's not unlikely that the dumping code itself is
>> not very optimal, so it might be worth having a look to see if more
>> batching etc might help speed things up at the root. Ages ago there were
>> patches floating around which improved the performance of the privcmd
>> mmap ioctl (from Mats Peterson), which might help here too (assuming
>> core dumping uses those interfaces as I expect). AFAIK those patches
>> were abandoned when Mats moved onto other things, resurrecting them
>> would probably help here as well as with migration and other
>> foreign-memory-man heavy workloads.
>>
>> Other random thoughts:
>>        * Automatically compress the core file, doesn't help with time to
>>          produce the dump (quite the opposite) but helps with copying it
>>          off box (but could also compress manually before moving or 
>> do it
>>          as a post processing step)
>>        * Have an option to produce some kind of "minicore", e.g. just
>>          register state and a few pages of the referenced stacks, 
>> perhaps
>>          a few pages of RAM either side of any register which happens to
>>          be a valid looking pointer.
>>                * Care would be needed to retain an acceptable 
>> probability
>>                  that the resulting core will actually be useful.
>>        * Consider including kernel RAM only (which might still be huge
>>          though if it has a 1:1 mapping of all RAM).
>>
>> I don't think any of those would be suitable for enabling by default but
>> they might be useful to have in the arsenal.
>>
>> HTH,
>>
>> Ian.
>>
>
> If these are Linux guests then the patch:
>
> http://lists.xenproject.org/archives/html/xen-devel/2013-11/msg02351.html
>
> Can be used to enable crash to access the crashed guest and collect
> some basic info. I would also include the output of xen-hvmctx and/or 
> xenctx.
> -- This is the quick way I would do Ian's minicore .
>
> Note: dump-core currently does not include xen-hvmctx output (nor does 
> it include
> xenctx -a output).
>
> Using the results from this may allow you to not need a copy of every 
> dump (
> not to imply that 2 similar minicore's would insure that a copy of 
> each core
> was not needed).
>
> I also think that makedumpfile can process a xen dump-core file and 
> make it
> smaller.
Looks similar as gdbsx.
This patch will help if I could reproduce the issue locally.
But I can't access customer's env in most situation and they will not 
wait me to do online debug remotely.

Yes, makedumpfile may help in size rather than time here.
I am thinking if port some code from makedumpfile is possible and 
acceptable.

zduan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: how to generate a smaller core with xm dump-core

[Ian Campbell]

On Fri, 2015-01-16 at 16:39 +0800, Zhenzhong Duan wrote:
> I am thinking if port some code from makedumpfile is possible and 
> acceptable.

makedumpfile appears to be GPL, whereas libxc is LGPL, so if you wanted
to port code directly you would need permissions from the copyright
holders to re-license.

I think we would be OK with you reimplementing some of the techniques
within tools/libxc/xc_core*.c though, so long as you don't copy actual
code. Obviously just improving that code without reference to
makedumpfile is fine too.

I don't think we need or want yet another tool for doing this in the
code base though.

Another option would be for you to add Xen support to makedumpfile,
you'd want to speak to the maintainers of the tool about that though.

Ian.

Re: how to generate a smaller core with xm dump-core

[Daniel Kiper]

On Fri, Jan 16, 2015 at 10:14:12AM +0000, Ian Campbell wrote:
> On Fri, 2015-01-16 at 16:39 +0800, Zhenzhong Duan wrote:
> > I am thinking if port some code from makedumpfile is possible and
> > acceptable.
>
> makedumpfile appears to be GPL, whereas libxc is LGPL, so if you wanted
> to port code directly you would need permissions from the copyright
> holders to re-license.
>
> I think we would be OK with you reimplementing some of the techniques
> within tools/libxc/xc_core*.c though, so long as you don't copy actual
> code. Obviously just improving that code without reference to
> makedumpfile is fine too.
>
> I don't think we need or want yet another tool for doing this in the
> code base though.
>
> Another option would be for you to add Xen support to makedumpfile,
> you'd want to speak to the maintainers of the tool about that though.

In general most of Xen related functionality is in place. There are some
issues when you try to use non common things. Once I though about extrating
makedumpfile's core functionality to a library (so file) in makedumpfile
project itself (IIRC, somebody dicussed this topic on kexec list, but
in another context, a few months ago) and then linking xl stuff with it.
Now it is more hand waving and I have this thing on my TODO list but right
now I have more important things to do. So, this is a quite far future work...

Daniel

Re: how to generate a smaller core with xm dump-core

[Don Slutz]

On 01/16/15 03:39, Zhenzhong Duan wrote:
> 在 2015/1/16 0:16, Don Slutz 写道:
>> On 01/15/15 05:20, Ian Campbell wrote:
>>> On Thu, 2015-01-15 at 11:31 +0800, Zhenzhong Duan wrote:
>>>> Hi Maintainers,
...
>> If these are Linux guests then the patch:
>>
>> http://lists.xenproject.org/archives/html/xen-devel/2013-11/msg02351.html
>>
>> Can be used to enable crash to access the crashed guest and collect
>> some basic info. I would also include the output of xen-hvmctx and/or
>> xenctx.
>> -- This is the quick way I would do Ian's minicore .
>>
>> Note: dump-core currently does not include xen-hvmctx output (nor does
>> it include
>> xenctx -a output).
>>
>> Using the results from this may allow you to not need a copy of every
>> dump (
>> not to imply that 2 similar minicore's would insure that a copy of
>> each core
>> was not needed).
>>
>> I also think that makedumpfile can process a xen dump-core file and
>> make it
>> smaller.
> Looks similar as gdbsx.

Sigh, I somehow missing this email till now.

It is like gdbsx, but for crash not gdb.

> This patch will help if I could reproduce the issue locally.
> But I can't access customer's env in most situation and they will not
> wait me to do online debug remotely.

Clearly I did not get my message across. I was not trying to say
"reproduce the issue locally".  What I was trying to say was that
this patch (to generate code) that can be used in a script to generate a
"minicore".  Also a script could use this "minicore" to decide if a full
core was needed.

crash is good at providing a summary of a core and can give high level
info about the state of linux.  I do not know of scripts that use just
gdb that can get this data.

This would be a way to save time and storage.

(Info on crash at http://people.redhat.com/anderson/ )

For example:

[root@hyper-0-21-51 C63-min-tools]# xl list
Name                                        ID   Mem VCPUs      State
Time(s)
Domain-0                                     0  2048     4     r-----
11563.1
C63-min-tools                                4  2080     1     ---sc-
   32.3
[root@hyper-0-21-51 C63-min-tools]# /usr/lib/xen/bin/xen-crashd 4 4444&
[1] 30786
[root@hyper-0-21-51 C63-min-tools]# 30 Jan 15 22:37:53.363 socket ready
on port 4444 after 1 bind call
crash localhost:4444
/home/don/C63-min-tools/kernel-debuginfo-2.6.32-279/usr/lib/debug/lib/modules/2.6.32-279.el6.x86_64/vmlinux

crash 7.0.9
Copyright (C) 2002-2014  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.

30 Jan 15 22:38:03.373 Accepted a connection.
WARNING: daemon cannot access /proc/version

GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

      KERNEL:
/home/don/C63-min-tools/kernel-debuginfo-2.6.32-279/usr/lib/debug/lib/modules/2.6.32-279.el6.x86_64/vmlinux
    DUMPFILE: /dev/mem@localhost  (remote live system)
        CPUS: 1
        DATE: Fri Jan 30 22:16:13 2015
      UPTIME: 00:21:06
LOAD AVERAGE: 0.07, 0.05, 0.02
       TASKS: 96
    NODENAME: C63-min-tools.tc5.don.cloudswitch.com
     RELEASE: 2.6.32-279.el6.x86_64
     VERSION: #1 SMP Fri Jun 22 12:19:21 UTC 2012
     MACHINE: x86_64  (2000 Mhz)
      MEMORY: 2 GB
         PID: 0
     COMMAND: "swapper"
        TASK: ffffffff81a8d020  [THREAD_INFO: ffffffff81a00000]
         CPU: 0
       STATE: TASK_RUNNING

crash> dmesg | tail -100
udev: renamed network interface rename17 to eth17
udev: renamed network interface eth5 to rename7
udev: renamed network interface eth9 to rename11
udev: renamed network interface rename14 to eth14
udev: renamed network interface eth4 to rename6
udev: renamed network interface eth8 to rename10
udev: renamed network interface eth2 to rename4
udev: renamed network interface eth6 to rename8
udev: renamed network interface eth1 to rename3
udev: renamed network interface eth0 to eth18
sd 0:0:0:0: Attached scsi generic sg0 type 0
sd 0:0:1:0: Attached scsi generic sg1 type 0
sr 1:0:0:0: Attached scsi generic sg2 type 5
sr 1:0:1:0: Attached scsi generic sg3 type 5
piix4_smbus 0000:00:07.3: SMBus Host Controller at 0xb100, revision 0
shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
EXT4-fs (xvda1): mounted filesystem with ordered data mode. Opts:
SELinux: initialized (dev xvda1, type ext4), uses xattr
Adding 2064376k swap on /dev/mapper/vg_c63mintools-lv_swap.  Priority:-1
extents:1 across:2064376k SS
SELinux: initialized (dev binfmt_misc, type binfmt_misc), uses
genfs_contexts
vmci module is older than RHEL 6.2 ... applying fixups
VMCI: Major device number is: 249
vsock module is older than RHEL 6.2 ... applying fixups
ip6_tables: (C) 2000-2006 Netfilter Core Team
nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
ip_tables: (C) 2000-2006 Netfilter Core Team
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
SELinux: initialized (dev rpc_pipefs, type rpc_pipefs), uses genfs_contexts
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
bnx2fc: Broadcom NetXtreme II FCoE Driver bnx2fc v1.0.11 (Apr 24, 2012)
eth0: no IPv6 routers present
eth1: no IPv6 routers present
SysRq : Trigger a crash
BUG: unable to handle kernel NULL pointer dereference at (null)
IP: [<ffffffff8132e6b6>] sysrq_handle_crash+0x16/0x20
PGD 7928a067 PUD 79b04067 PMD 0
Oops: 0002 [#1] SMP
last sysfs file: /sys/devices/vbd-51712/block/xvda/dev
CPU 0
Modules linked in: bnx2fc fcoe libfcoe libfc scsi_transport_fc 8021q
scsi_tgt garp stp llc sunrpc ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4
iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6
xt_state nf_conntrack ip6table_filter ip6_tables vsock(U) vmci(U) shpchp
i2c_piix4 i2c_core sg ext4 mbcache jbd2 vmw_pvscsi mptspi mptscsih
mptbase scsi_transport_spi sd_mod crc_t10dif sr_mod cdrom xen_netfront
xen_blkfront pata_acpi ata_generic ata_piix e1000 vmxnet3 dm_mirror
dm_region_hash dm_log dm_mod be2iscsi bnx2i cnic uio ipv6 cxgb4i cxgb4
cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp qla4xxx iscsi_boot_sysfs
libiscsi scsi_transport_iscsi [last unloaded: scsi_wait_scan]

Pid: 1803, comm: bash Not tainted 2.6.32-279.el6.x86_64 #1 Xen HVM domU
VMwareless
RIP: 0010:[<ffffffff8132e6b6>]  [<ffffffff8132e6b6>]
sysrq_handle_crash+0x16/0x20
RSP: 0018:ffff8800370f7e18  EFLAGS: 00010096
RAX: 0000000000000010 RBX: 0000000000000063 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000063
RBP: ffff8800370f7e18 R08: 0000000000000000 R09: ffffffff8163ab80
R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000000
R13: ffffffff81afb760 R14: 0000000000000286 R15: 0000000000000007
FS:  00007fb4234a4700(0000) GS:ffff88000c400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000037075000 CR4: 00000000000406f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process bash (pid: 1803, threadinfo ffff8800370f6000, task ffff88007c62a040)
Stack:
 ffff8800370f7e68 ffffffff8132e972 ffff88007c62a040 ffff880000000000
<d> 0000000d80df4018 0000000000000002 ffff88007d14cd40 00007fb4234a2000
<d> 0000000000000002 fffffffffffffffb ffff8800370f7e98 ffffffff8132ea2e
Call Trace:
 [<ffffffff8132e972>] __handle_sysrq+0x132/0x1a0
 [<ffffffff8132ea2e>] write_sysrq_trigger+0x4e/0x50
 [<ffffffff811e0abe>] proc_reg_write+0x7e/0xc0
 [<ffffffff8117b068>] vfs_write+0xb8/0x1a0
 [<ffffffff810d69e2>] ? audit_syscall_entry+0x272/0x2a0
 [<ffffffff8117ba81>] sys_write+0x51/0x90
 [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b
Code: d0 88 81 63 44 fd 81 c9 c3 66 66 66 2e 0f 1f 84 00 00 00 00 00 55
48 89 e5 0f 1f 44 00 00 c7 05 1d 8d 76 00 01 00 00 00 0f ae f8 <c6> 04
25 00 00 00 00 01 c9 c3 55 48 89 e5 0f 1f 44 00 00 8d 47
RIP  [<ffffffff8132e6b6>] sysrq_handle_crash+0x16/0x20
 RSP <ffff8800370f7e18>
CR2: 0000000000000000
---[ end trace 0593342d2727c9fd ]---
Kernel panic - not syncing: Fatal exception
Pid: 1803, comm: bash Tainted: G      D    ---------------
2.6.32-279.el6.x86_64 #1
Call Trace:
 [<ffffffff814fd11a>] ? panic+0xa0/0x168
 [<ffffffff815012b4>] ? oops_end+0xe4/0x100
 [<ffffffff81043bab>] ? no_context+0xfb/0x260
 [<ffffffff81043e35>] ? __bad_area_nosemaphore+0x125/0x1e0
 [<ffffffff815032e6>] ? notifier_call_chain+0x16/0x80
 [<ffffffff81043f5e>] ? bad_area+0x4e/0x60
 [<ffffffff81044710>] ? __do_page_fault+0x3d0/0x480
 [<ffffffff8106b8e5>] ? __call_console_drivers+0x75/0x90
 [<ffffffff81097e2f>] ? up+0x2f/0x50
 [<ffffffff8106b94a>] ? _call_console_drivers+0x4a/0x80
 [<ffffffff8106bfdf>] ? release_console_sem+0x1cf/0x220
 [<ffffffff8150326e>] ? do_page_fault+0x3e/0xa0
 [<ffffffff81500625>] ? page_fault+0x25/0x30
 [<ffffffff8132e6b6>] ? sysrq_handle_crash+0x16/0x20
 [<ffffffff8132e972>] ? __handle_sysrq+0x132/0x1a0
 [<ffffffff8132ea2e>] ? write_sysrq_trigger+0x4e/0x50
 [<ffffffff811e0abe>] ? proc_reg_write+0x7e/0xc0
 [<ffffffff8117b068>] ? vfs_write+0xb8/0x1a0
 [<ffffffff810d69e2>] ? audit_syscall_entry+0x272/0x2a0
 [<ffffffff8117ba81>] ? sys_write+0x51/0x90
 [<ffffffff8100b0f2>] ? system_call_fastpath+0x16/0x1b
crash> runq
CPU 0 RUNQUEUE: ffff88000c416680
  CURRENT: PID: 1803   TASK: ffff88007c62a040  COMMAND: "bash"
  RT PRIO_ARRAY: ffff88000c416808
     [no tasks queued]
  CFS RB_ROOT: ffff88000c416718
     [no tasks queued]
crash> net -a
NEIGHBOUR        IP ADDRESS      HW TYPE    HW ADDRESS         DEVICE  STATE
ffff8800374f6d80 172.16.51.1     ETHER      00:22:99:c8:81:67  eth0
REACHABLE
ffff8800374f6e80 0.0.0.0         UNKNOWN    00 00 00 00 00 00  lo      NOARP
crash>

etc.

   -Don Slutz


> 
> Yes, makedumpfile may help in size rather than time here.
> I am thinking if port some code from makedumpfile is possible and
> acceptable.
> 
> zduan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel