Re: how to generate a smaller core with xm dump-core

[Zhenzhong Duan <zhenzhong.duan@oracle.com>]

在 2015/1/16 0:16, Don Slutz 写道:
> On 01/15/15 05:20, Ian Campbell wrote:
>> On Thu, 2015-01-15 at 11:31 +0800, Zhenzhong Duan wrote:
>>> Hi Maintainers,
>>>
>>>
>>> We are facing issue collecting  coredump using the xm dump mechanism
>>> in Dom0.
>>>
>>> We face couple of such issues daily, where the VMs panic s and the SA
>>> team is supposed to collect the core.
>>>
>>> The actual problem is that where the VMs are having huge RAM like 32+
>>> GB RAMs dumping the core using xm dump generated core of almost the
>>> same size.
>>>
>>> And also when the RAM size is huge the time taken to dump is also
>>> huge.
>>>
>>> Transferring this core dump of this huge size to another system for
>>> analysis is really a pain. I am looking for some solution where we can
>>> generate smaller core using xm dump or any other tool.
>>>
>>> Something similar to makedumpfile which eliminates the zero and
>>> unnecessary pages, which results in core of smaller size.
>>>
>>>
>>> This has become a hot issue and we terribly need this feature.
>>>
>>> Can you come across with anything or are you aware of any feature
>>> which could cater our need….?
>> I'm not aware of any existing features or efforts in this area.
>>
>> If things like zero pages can be omitted from a core file without making
>> it non-standard then I think we'd be happy to see such patches to the
>> core libxc functions which produce the core files.
>>
>> That wouldn't help much with a heavily loaded VM which is making good
>> use of all its ram though.
>>
>> I've not looked but it's not unlikely that the dumping code itself is
>> not very optimal, so it might be worth having a look to see if more
>> batching etc might help speed things up at the root. Ages ago there were
>> patches floating around which improved the performance of the privcmd
>> mmap ioctl (from Mats Peterson), which might help here too (assuming
>> core dumping uses those interfaces as I expect). AFAIK those patches
>> were abandoned when Mats moved onto other things, resurrecting them
>> would probably help here as well as with migration and other
>> foreign-memory-man heavy workloads.
>>
>> Other random thoughts:
>>        * Automatically compress the core file, doesn't help with time to
>>          produce the dump (quite the opposite) but helps with copying it
>>          off box (but could also compress manually before moving or 
>> do it
>>          as a post processing step)
>>        * Have an option to produce some kind of "minicore", e.g. just
>>          register state and a few pages of the referenced stacks, 
>> perhaps
>>          a few pages of RAM either side of any register which happens to
>>          be a valid looking pointer.
>>                * Care would be needed to retain an acceptable 
>> probability
>>                  that the resulting core will actually be useful.
>>        * Consider including kernel RAM only (which might still be huge
>>          though if it has a 1:1 mapping of all RAM).
>>
>> I don't think any of those would be suitable for enabling by default but
>> they might be useful to have in the arsenal.
>>
>> HTH,
>>
>> Ian.
>>
>
> If these are Linux guests then the patch:
>
> http://lists.xenproject.org/archives/html/xen-devel/2013-11/msg02351.html
>
> Can be used to enable crash to access the crashed guest and collect
> some basic info. I would also include the output of xen-hvmctx and/or 
> xenctx.
> -- This is the quick way I would do Ian's minicore .
>
> Note: dump-core currently does not include xen-hvmctx output (nor does 
> it include
> xenctx -a output).
>
> Using the results from this may allow you to not need a copy of every 
> dump (
> not to imply that 2 similar minicore's would insure that a copy of 
> each core
> was not needed).
>
> I also think that makedumpfile can process a xen dump-core file and 
> make it
> smaller.
Looks similar as gdbsx.
This patch will help if I could reproduce the issue locally.
But I can't access customer's env in most situation and they will not 
wait me to do online debug remotely.

Yes, makedumpfile may help in size rather than time here.
I am thinking if port some code from makedumpfile is possible and 
acceptable.

zduan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel