Re: kdump with xen-unstable on efi machine

[Juergen Gross <jgross@suse.com>]

On 11/27/2014 01:19 AM, Daniel Kiper wrote:
> On Wed, Nov 26, 2014 at 03:49:54PM +0100, Juergen Gross wrote:
>> On 11/26/2014 03:30 PM, Konrad Rzeszutek Wilk wrote:
>>> On Wed, Nov 26, 2014 at 03:01:51PM +0100, Juergen Gross wrote:
>>>> On 11/26/2014 01:41 PM, Andrew Cooper wrote:
>>>>> On 26/11/14 12:15, Juergen Gross wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I tried to enable kdump on my test-machine with actual xen-unstable
>>>>>> booting via EFI.
>>>>>>
>>>>>> The kdump kernel is not being loaded.
>>>>>>
>>>>>> I'm seeing the memory being reserved:
>>>>>>
>>>>>> (XEN) EFI RAM map:
>>>>>> (XEN)  0000000000000000 - 00000000000a0000 (usable)
>>>>>> (XEN)  0000000000100000 - 000000004bc00000 (usable)
>>>>>> (XEN)  000000004bc00000 - 000000005bc00000 (reserved)
>>>>>> (XEN)  000000005bc00000 - 000000005bfec000 (usable)
>>>>>> (XEN)  000000005bfec000 - 000000005c000000 (ACPI NVS)
>>>>>> (XEN)  000000005c000000 - 000000006a429000 (usable)
>>>>>> (XEN)  000000006a429000 - 000000006a42c000 (reserved)
>>>>>> (XEN)  000000006a42c000 - 000000006a7a2000 (usable)
>>>>>> (XEN)  000000006a7a2000 - 000000006a7a8000 (reserved)
>>>>>> (XEN)  000000006a7a8000 - 000000006a987000 (usable)
>>>>>> (XEN)  000000006a987000 - 000000006a98d000 (reserved)
>>>>>> (XEN)  000000006a98d000 - 000000006aa63000 (usable)
>>>>>> (XEN)  000000006aa63000 - 000000006aa73000 (reserved)
>>>>>> (XEN)  000000006aa73000 - 000000006ac60000 (usable)
>>>>>> (XEN)  000000006ac60000 - 000000006ac61000 (reserved)
>>>>>> (XEN)  000000006ac61000 - 000000006ac9b000 (ACPI data)
>>>>>> (XEN)  000000006ac9b000 - 000000006acac000 (reserved)
>>>>>> (XEN)  000000006acac000 - 000000006acad000 (usable)
>>>>>> (XEN)  000000006acad000 - 000000006acae000 (reserved)
>>>>>> (XEN)  000000006acae000 - 000000007189c000 (usable)
>>>>>> (XEN)  000000007189c000 - 0000000071946000 (reserved)
>>>>>> (XEN)  0000000071946000 - 0000000072d76000 (ACPI NVS)
>>>>>> (XEN)  0000000072d76000 - 0000000072db2000 (ACPI data)
>>>>>> (XEN)  0000000072db2000 - 0000000072edc000 (usable)
>>>>>> (XEN)  0000000080000000 - 0000000090000000 (reserved)
>>>>>> (XEN)  0000000100000000 - 0000002080000000 (usable)
>>>>>> (XEN) Kdump: 256MB (262144kB) at 0x206dff4000
>>>>>>
>>>>>> I'd expect this area being visible in the efi or e820 map presented to
>>>>>> dom0, but I can't see anything:
>>>>>
>>>>> This is expected.  The dom0 kernel now has nothing at all do with
>>>>> loading crash kernel.  Loading happens via hypercalls straight from the
>>>>> kexec utility.
>>>>>
>>>>> You need kexec-tools 2.0.4 (I think) or later, compiled with Xen
>>>>> support, but it should JustWork.
>>>>
>>>> Should. I have kexec 2.0.5 with Xen support. Doesn't work:
>>>>
>>>> Excerpt form strace:
>>>>
>>>> "sysctl operation failed -- need to rebuild the user-space tool set?\n"
>>>>
>>>> My personal translation: kexec is tightly coupled to the Xen version
>>>> (this one was built against Xen 4.4.1 AFAIK).
>>>
>>> Odd, the hypercall interface did not change in Xen 4.5 for kexec?
>>>
>>> Perhaps it is making some other hypercalls that are tied in
>>> to the version of Xen (like sysctl ones?).
>>
>> The error message above suggests that, yes. :-)
>>
>> Grepping for xc_ in kexec sources finds e.g. xc_get_max_cpus() which
>> in turn calls xc_physinfo() doing a sysctl.
>>
>>>
>>> I presume with recompiling it works?
>>
>> Didn't check up to now, but I think it should.
>
> Are you sure that kexec-tools configure script discovered
> Xen headers and development libraries? Please check that.
> "ldd kexec" is your friend.

Aah, here is the problem: The kexec installed on my test system has been
linked statically with libxenctrl to be usable on non-Xen systems, too.

The proper solution, however, would have been to check for running on
Xen dynamically and then load the current library via dlopen().

For now I'll just rebuild kexec.

I put above on my todo list. As I expect to modify kexec in the future
for support of the linear mapped p2m list I'll do the dlopen() stuff
then.

>
> Do not forget to use kexec-tools version 2.0.5 or newer.
>
>>>>
>>>> Perhaps we should add kexec to the tools directory?
>>>
>>> Gosh no.
>>
>> Oops, did I forget the smiley? ;-)
>>
>> I think we should look what kexec is really needing and put this in a
>> stable interface set (perhaps an own library?). This might require some
>
> David did the work.

Okay.

>
>> new sub functions of e.g. the KEXEC hypercall, but this is better than
>> making kexec depending on the Xen version.
>
> Maybe we need some things which are specific for EFI platforms. I am
> going to investigate that after finishing EFI + multiboot2 work.
> Probably it will happen at the beginning of next year.

I'll come back to you when I'm going to change kexec.


Juergen