[PATCH 18/19] arm64: kdump: update a kernel doc

[dyoung@redhat.com (Dave Young)]

On 01/20/16 at 03:07pm, AKASHI Takahiro wrote:
> On 01/20/2016 11:49 AM, Dave Young wrote:
> >On 01/19/16 at 02:01pm, Mark Rutland wrote:
> >>On Tue, Jan 19, 2016 at 09:45:53PM +0800, Dave Young wrote:
> >>>On 01/19/16 at 12:51pm, Mark Rutland wrote:
> >>>>On Tue, Jan 19, 2016 at 08:28:48PM +0800, Dave Young wrote:
> >>>>>On 01/19/16 at 02:35pm, AKASHI Takahiro wrote:
> >>>>>>On 01/19/2016 10:43 AM, Dave Young wrote:
> >>>>>>>X86 takes another way in latest kexec-tools and kexec_file_load, that is
> >>>>>>>recreating E820 table and pass it to kexec/kdump kernel, if the entries
> >>>>>>>are over E820 limitation then turn to use setup_data list for remain
> >>>>>>>entries.
> >>>>>>
> >>>>>>Thanks. I will visit x86 code again.
> >>>>>>
> >>>>>>>I think it is X86 specific. Personally I think device tree property is
> >>>>>>>better.
> >>>>>>
> >>>>>>Do you think so?
> >>>>>
> >>>>>I'm not sure it is the best way. For X86 we run into problem with
> >>>>>memmap= design, one example is pci domain X (X>1) need the pci memory
> >>>>>ranges being passed to kdump kernel. When we passed reserved ranges in /proc/iomem
> >>>>>to 2nd kernel we find that cmdline[] array is not big enough.
> >>>>
> >>>>I'm not sure how PCI ranges relate to the memory map used for normal
> >>>>memory (i.e. RAM), though I'm probably missing some caveat with the way
> >>>>ACPI and UEFI describe PCI. Why does memmap= affect PCI memory?
> >>>
> >>>Here is the old patch which was rejected in kexec-tools:
> >>>http://lists.infradead.org/pipermail/kexec/2013-February/007924.html
> >>>
> >>>>
> >>>>If the kernel got the rest of its system topology from DT, the PCI
> >>>>regions would be described there.
> >>>
> >>>Yes, if kdump kernel use same DT as 1st kernel.
> >>
> >>Other than for testing purposes, I don't see why you'd pass the kdump
> >>kernel a DTB inconsistent with that the 1st kernel was passsed (other
> >>than some proerties under /chosen).
> >>
> >>We added /sys/firmware/fdt specifically to allow the kexec tools to get
> >>the exact DTB the first kernel used. There's no reason for tools to have
> >>to make something up.
> >
> >Agreed, but kexec-tools has an option to pass in any dtb files. Who knows
> >how one will use it unless dropping the option and use /sys/firmware/fdt
> >unconditionally.
> 
> As a matter of fact, specifying proper command line parameters as well as
> dtb is partly users' responsibility for kdump to work correctly.
> (especially for BE kernel)

Right.

> 
> >If we choose to implement kexec_file_load only in kernel, the interfaces
> >provided are kernel, initrd and cmdline. We can always use same dtb.
> 
> I would say that we can always use the same dtb even with kexec_load
> from user's perspective. Right?
> (The difference is whether changes are made by kernel itself or kexec-tools.)

Right.

> 
> >>
> >>>>>Do you think for arm64 only usable memory is necessary to let kdump kernel
> >>>>>know? I'm curious about how arm64 kernel get all memory layout from boot loader,
> >>>>>via UEFI memmap?
> >>>>
> >>>>When booted via EFI, we use the EFI memory map. The EFI stub handles
> >>>>acquring the relevant information and passing that to the first kernel
> >>>>in the DTB (see Documentation/arm/uefi.txt).
> >>>
> >>>Ok, thanks for the pointer. So in dt we are just have uefi memmap infomation
> >>>instead of memory nodes details..
> >>
> >>When booted via EFI, yes.
> >>
> >>For NUMA topology in !ACPI kernels, we might need to also retain and
> >>parse memory nodes, but only for toplogy information. The kernel would
> >>still only use memory as described by the EFI memory map.
> >>
> >>There's a horrible edge case I've spotted if performing a chain of
> >>cross-endian kexecs: LE -> BE -> LE, as the BE kernel would have to
> >>respect the EFI memory map so as to avoid corrupting it for the
> >>subsequent LE kernel. Other than this I believe everything should just
> >>work.
> >
> >Firmware do not know kernel endianniess, kernel should respect firmware
> >maps and adapt to it, it sounds like a generic issue not specfic to kexec.
> 
> On arm64, a kernel image header has a bit field to specify the image's endianness.
> Anyway, our current implementation replies on a user-supplied dtb to start BE kernel.

Ok, I means uefi memmap are same, not specific to LE or BE.

> 
> >>
> >>>>A kexec'd kernel should simply inherit that. So long as the DTB and/or
> >>>>UEFI tables in memory are the same, it would be the same as a cold boot.
> >>>
> >>>For kexec all memory ranges are same, for kdump we need use original reserved
> >>>range with crashkernel= as usable memory and all other orignal usable ranges
> >>>are not usable anymore.
> >>
> >>Sure. This is what I believe we should expose with an additional
> >>property under /chosen, while keeping everything else pristine.
> >>
> >>The crash kernel can then limit itself to that region, while it would
> >>have the information of the full memory map (which it could log and/or
> >>use to drive other dumping).
> >
> >In this way kernel should be aware it is a kdump booting, it is doable though
> >I feel it is better for kdump kernel in a black box with infomations it
> >can use just like the 1st kernel. Things here is where we choose to cook
> >the memory infomation in boot loader or in kernel itself.
> >
> >>
> >>>Is it possible to modify uefi memmap for kdump case?
> >>
> >>Technically it would be possible, however I don't think it's necessary,
> >>and I think it would be disadvantageous to do so.
> >>
> >>Describing the range(s) the crash kernel can use in separate properties
> >>under /chosen has a number of advantages.
> >
> >Ok, I got the points. We have a is_kdump_kernel() by checking if there is
> >elfcorehdr_addr kernel cmdline. This is mainly for some drivers which
> >do not work well in kdump kernel some uncertain reasons. But ideally I
> >think kernel should handle things just like in 1st kernel and avoid to use
> >it.
> 
> So I'm not still sure about what are advantages of a property under /chosen
> over "memmap=" kernel parameter.
> Both are simple and can have the same effect with minimizing changes to dtb.
> (But if, in the latter case, we have to provide *all* the memory-related information
> through "memmap=" parameters, it would be much complicated.)

Maybe I did not say it clearly, I prefer kexec syscall/tool to modifiy dtb
or uefi-memmap so that we do not need any extra kernel cmdline.

For x86 we would like to drop the memmap= usage in kexec-tools but we can not
do that for a compatibility problem about calgary iommu. So that currently
kexec-tools supports both recreating E820 maps and passing memmap=.

We should think it carefully because it will be hard to remove once we support it.
IMO handling it in code is better than using an external interface.

> 
> -Takahiro AKASHI
> 
> >>
> >>>>In the !EFI case, we use the memory nodes in the DTB. Only in this case
> >>>>could usable-memory properties in memory nodes make sense. I'd prefer a
> >>>>uniform property under /chosen for both cases.
> >>>
> >>>We stil use same DTB, need to modify the DT and update the usable and unusable
> >>>nodes for kdump?
> >>
> >>We'd have a (slightly) modified DTB that contained additional properties
> >>describing the range(s) reserved for use by the crash kernel.
> >>
> >>Other than those properties under /chosen (e.g. the command line, initrd
> >>pointers if any), it would be the original DTB.
> >>
> >>Thanks,
> >>Mark.
> >
> >Thanks
> >Dave
> >

Thanks
Dave