ARM64 kexec/kdump timeline

ARM64 kexec/kdump timeline

[Timur Tabi]

On 05/21/2015 03:15 PM, Geoff Levand wrote:
> I feel kexec was ready to merge nine months ago.  Users have only sent
> private e-mails like yourself, so there has been no public inquiries,
> and so, the arm64 maintainers think kexec is not of interest to users.

Does kexec on ARM64 support ACPI?  In some of our initial tests, it 
appears that the kexec'd kernel won't boot because it's expecting a 
device tree, and we don't have one.

ARM64 servers are supposed to use ACPI instead of a device tree.

-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.

ARM64 kexec/kdump timeline

[Ard Biesheuvel]

On 27 May 2015 at 05:08, Timur Tabi <timur@codeaurora.org> wrote:
> On 05/21/2015 03:15 PM, Geoff Levand wrote:
>>
>> I feel kexec was ready to merge nine months ago.  Users have only sent
>> private e-mails like yourself, so there has been no public inquiries,
>> and so, the arm64 maintainers think kexec is not of interest to users.
>
>
> Does kexec on ARM64 support ACPI?  In some of our initial tests, it appears
> that the kexec'd kernel won't boot because it's expecting a device tree, and
> we don't have one.
>
> ARM64 servers are supposed to use ACPI instead of a device tree.
>

The latest arm64 kexec patches support booting via UEFI, and when
booting via UEFI, the ACPI root pointer is retrieved from a UEFI
configuration table if the FDT contains no system description. So ACPI
support should not depend at all on how kexec is implemented (unless
your FDT does contain a system description, in which case you should
pass along the 'acpi=xxx' param that was given to the cold booted
kernel)

-- 
Ard.

ARM64 kexec/kdump timeline

[Geoff Levand]

Hi,

On Tue, 2015-05-26 at 22:08 -0500, Timur Tabi wrote:
> Does kexec on ARM64 support ACPI?  In some of our initial tests, it 
> appears that the kexec'd kernel won't boot because it's expecting a 
> device tree, and we don't have one.

I tested kexec re-boot on the foundation model with ACPI and it worked
OK.

I guess either your second stage kernel is not configured correctly, or
you did not invoke kexec correctly.  Check your dtb and command line
parameters.

-Geoff

ARM64 kexec/kdump timeline

[Arnd Bergmann]

On Tuesday 26 May 2015 22:08:40 Timur Tabi wrote:
> On 05/21/2015 03:15 PM, Geoff Levand wrote:
> > I feel kexec was ready to merge nine months ago.  Users have only sent
> > private e-mails like yourself, so there has been no public inquiries,
> > and so, the arm64 maintainers think kexec is not of interest to users.
> 
> Does kexec on ARM64 support ACPI?  In some of our initial tests, it 
> appears that the kexec'd kernel won't boot because it's expecting a 
> device tree, and we don't have one.

ACPI support has just been merged and is still experimental. You
should be able to boot your system by passing a DT blob at the
initial boot that matches your hardware. Can you try if that
makes kexec work?

	Arnd

ARM64 kexec/kdump timeline

[Timur Tabi]

On 05/27/2015 11:39 AM, Arnd Bergmann wrote:
> ACPI support has just been merged and is still experimental. You
> should be able to boot your system by passing a DT blob at the
> initial boot that matches your hardware. Can you try if that
> makes kexec work?

If I had an initial DT blob that matched by hardware, I wouldn't need 
ACPI support!  This is an ARM64 Server system.  There is no device tree 
for the hardware.  Everything is in ACPI.

What does x86 do?  They don't have device trees, but they do use ACPI, 
so how does kexec work there?

-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.

ARM64 kexec/kdump timeline

[Hanjun Guo]

On 2015?05?27? 17:38, Ard Biesheuvel wrote:
> On 27 May 2015 at 05:08, Timur Tabi <timur@codeaurora.org> wrote:
>> On 05/21/2015 03:15 PM, Geoff Levand wrote:
>>>
>>> I feel kexec was ready to merge nine months ago.  Users have only sent
>>> private e-mails like yourself, so there has been no public inquiries,
>>> and so, the arm64 maintainers think kexec is not of interest to users.
>>
>>
>> Does kexec on ARM64 support ACPI?  In some of our initial tests, it appears
>> that the kexec'd kernel won't boot because it's expecting a device tree, and
>> we don't have one.
>>
>> ARM64 servers are supposed to use ACPI instead of a device tree.
>>
>
> The latest arm64 kexec patches support booting via UEFI, and when
> booting via UEFI, the ACPI root pointer is retrieved from a UEFI
> configuration table if the FDT contains no system description. So ACPI
> support should not depend at all on how kexec is implemented (unless
> your FDT does contain a system description, in which case you should
> pass along the 'acpi=xxx' param that was given to the cold booted
> kernel)

Thanks for the clarify. Geoff also tested kexec on FVP with ACPI
enabled and it works as far as I know.

Timur, can you provide more detailed information then we can debug
into it?

Thanks
Hanjun

ARM64 kexec/kdump timeline

[Hanjun Guo]

On 2015?05?28? 07:14, Timur Tabi wrote:
> On 05/27/2015 11:39 AM, Arnd Bergmann wrote:
>> ACPI support has just been merged and is still experimental. You
>> should be able to boot your system by passing a DT blob at the
>> initial boot that matches your hardware. Can you try if that
>> makes kexec work?
>
> If I had an initial DT blob that matched by hardware, I wouldn't need
> ACPI support!  This is an ARM64 Server system.  There is no device tree
> for the hardware.  Everything is in ACPI.
>
> What does x86 do?  They don't have device trees, but they do use ACPI,
> so how does kexec work there?

So the key point is that we need to get the root pointer to the
ACPI tables (RSDP), which is in the UEFI configuration table as
Ard said, I think this part is the same for ARM64 and x86.

ACPI provide an early param "acpi_rsdp" for kexec use as a option,
you can refer to Documentation/kernel-parameters.txt.

Thanks
Hanjun

ARM64 kexec/kdump timeline

[Ard Biesheuvel]

On 28 May 2015 at 05:52, Hanjun Guo <hanjun.guo@linaro.org> wrote:
> On 2015?05?28? 07:14, Timur Tabi wrote:
>>
>> On 05/27/2015 11:39 AM, Arnd Bergmann wrote:
>>>
>>> ACPI support has just been merged and is still experimental. You
>>> should be able to boot your system by passing a DT blob at the
>>> initial boot that matches your hardware. Can you try if that
>>> makes kexec work?
>>
>>
>> If I had an initial DT blob that matched by hardware, I wouldn't need
>> ACPI support!  This is an ARM64 Server system.  There is no device tree
>> for the hardware.  Everything is in ACPI.
>>
>> What does x86 do?  They don't have device trees, but they do use ACPI,
>> so how does kexec work there?
>
>
> So the key point is that we need to get the root pointer to the
> ACPI tables (RSDP), which is in the UEFI configuration table as
> Ard said, I think this part is the same for ARM64 and x86.
>
> ACPI provide an early param "acpi_rsdp" for kexec use as a option,
> you can refer to Documentation/kernel-parameters.txt.
>

Yes, but acpi_rsdp= really shouldn't be needed if the kexec'ed kernel
boots via UEFI.

So I think the issue may be (but Timur really needs to confirm) that
the DT passed via kexec does not contain all the UEFI specific nodes
in /chosen that are present in /sys/firmware/fdt.

@Timur: could you try passing the contents of /sys/firmware/fdt verbatim?

ARM64 kexec/kdump timeline

[Geoff Levand]

On Thu, 2015-05-28 at 11:22 +0800, Hanjun Guo wrote:
> Timur, can you provide more detailed information then we can debug
> into it?

Please post the output of kexec with the --debug option set, and also
the output of the kernel, maybe with 'loglevel=9'.

-Geoff

ARM64 kexec/kdump timeline

[Pratyush Anand]

On Thursday 28 May 2015 04:44 AM, Timur Tabi wrote:
> On 05/27/2015 11:39 AM, Arnd Bergmann wrote:
>> ACPI support has just been merged and is still experimental. You
>> should be able to boot your system by passing a DT blob at the
>> initial boot that matches your hardware. Can you try if that
>> makes kexec work?
>
> If I had an initial DT blob that matched by hardware, I wouldn't need
> ACPI support!  This is an ARM64 Server system.  There is no device tree
> for the hardware.  Everything is in ACPI.

So what error do you see when you execute kexec.

We had seen a failure with check_cpu_nodes(), when booting with ACPI 
without any DTB.

If you see the similar failure, then can you pl try following:

diff --git a/kexec/arch/arm64/kexec-arm64.c b/kexec/arch/arm64/kexec-arm64.c
index 7b219097dfff..8e085212d8a5 100644
--- a/kexec/arch/arm64/kexec-arm64.c
+++ b/kexec/arch/arm64/kexec-arm64.c
@@ -640,7 +640,7 @@ int arm64_load_other_segments(struct kexec_info *info,
         result =  check_cpu_nodes(&dtb_1, &dtb_2);

         if (result)
-               return result;
+               fprintf(stderr, "kexec: Warning: No device tree 
available.\n");

         /*
          * Put the DTB after the kernel with an alignment of 128 KiB, 
giving

~Pratyush

ARM64 kexec/kdump timeline

[Timur Tabi]

On Mon, Jun 1, 2015 at 1:25 AM, Pratyush Anand <panand@redhat.com> wrote:

> We had seen a failure with check_cpu_nodes(), when booting with ACPI without
> any DTB.
>
> If you see the similar failure, then can you pl try following:
>
> diff --git a/kexec/arch/arm64/kexec-arm64.c b/kexec/arch/arm64/kexec-arm64.c
> index 7b219097dfff..8e085212d8a5 100644
> --- a/kexec/arch/arm64/kexec-arm64.c
> +++ b/kexec/arch/arm64/kexec-arm64.c
> @@ -640,7 +640,7 @@ int arm64_load_other_segments(struct kexec_info *info,
>         result =  check_cpu_nodes(&dtb_1, &dtb_2);
>
>         if (result)
> -               return result;
> +               fprintf(stderr, "kexec: Warning: No device tree
> available.\n");
>
>         /*
>          * Put the DTB after the kernel with an alignment of 128 KiB, giving

I finally got around to testing this.  With this change, the tool just
displays the error message and continues working.  So +1 for this
patch.

It would be better, of course, if the tool could parse the ACPI tables
and generate a compatible device tree with CPU nodes.  Do you have any
plans to support ACPI with this tool?  We're working on that
internally, but it's mostly hacks and work-arounds for now.

-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.

ARM64 kexec/kdump timeline

[Azriel Samson]

Hi,

When booting the crash kernel with ACPI, how do you ensure that the ACPI
tables fall within the kernel's memmap region?
I was able to boot with ACPI when the crash kernel's memory overlapped
with the ACPI tables(based on the "crashkernel" and "mem" boot arguments).

Is there a better way to make the ACPI tables available to the crash kernel?

> On Mon, Jun 1, 2015 at 1:25 AM, Pratyush Anand <panand@redhat.com> wrote:
>
>> We had seen a failure with check_cpu_nodes(), when booting with ACPI
>> without
>> any DTB.
>>
>> If you see the similar failure, then can you pl try following:
>>
>> diff --git a/kexec/arch/arm64/kexec-arm64.c
>> b/kexec/arch/arm64/kexec-arm64.c
>> index 7b219097dfff..8e085212d8a5 100644
>> --- a/kexec/arch/arm64/kexec-arm64.c
>> +++ b/kexec/arch/arm64/kexec-arm64.c
>> @@ -640,7 +640,7 @@ int arm64_load_other_segments(struct kexec_info
>> *info,
>>         result =  check_cpu_nodes(&dtb_1, &dtb_2);
>>
>>         if (result)
>> -               return result;
>> +               fprintf(stderr, "kexec: Warning: No device tree
>> available.\n");
>>
>>         /*
>>          * Put the DTB after the kernel with an alignment of 128 KiB,
>> giving
>
> I finally got around to testing this.  With this change, the tool just
> displays the error message and continues working.  So +1 for this
> patch.
>
> It would be better, of course, if the tool could parse the ACPI tables
> and generate a compatible device tree with CPU nodes.  Do you have any
> plans to support ACPI with this tool?  We're working on that
> internally, but it's mostly hacks and work-arounds for now.
>
> --
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> a Linux Foundation Collaborative Project.
>


-- 
Thanks,
Azriel Samson
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,?
a Linux Foundation Collaborative Project

ARM64 kexec/kdump timeline

[Azriel Samson]

Hi,

On 7/13/2015 3:03 AM, AKASHI, Takahiro wrote:
> Sorry for not replying soon.
>
> On 9 July 2015 at 08:03, Azriel Samson <asamson@codeaurora.org
> <mailto:asamson@codeaurora.org>> wrote:
>  >
>  > Hi,
>  >
>  > When booting the crash kernel with ACPI, how do you ensure that the ACPI
>  > tables fall within the kernel's memmap region?
>  > I was able to boot with ACPI when the crash kernel's memory overlapped
>  > with the ACPI tables(based on the "crashkernel" and "mem" boot
> arguments).
>
> I don't think this is a right way to use kdump in ACPI environment on arm64
> because "mem" parameter only limits the maximum size of usable memory
> and so the crash dump data (/proc/vmcore) can get easily corrupted if
> the reserved
> memory region that the crash dump kernel can use also contains any
> portion of
> memory which belong to the crashed kernel.
>
> In the current implementation of arm64 kdump, kexec command manages to
>   exclude such memory regions by modifying the device tree passed on to
> the crash dump kernel.
>
>  > Is there a better way to make the ACPI tables available to the crash
> kernel?
>
> So ACPI tables is not the only problem.
> x86 seems to use "memmap" kernel parameter instead.(?)

I was able to test kdump with the following patch from Mark Salter which 
is needed for supporting ACPI tables that lie outside of kernel RAM:
https://git.fedorahosted.org/cgit/kernel-arm64.git/commit/?h=devel&id=a75bcec5679a2ff82f23c5cc335c77da158a6dc5


>
> Thanks,
> -Takahiro AKASHI
>

-- 
Thanks,
Azriel Samson
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project