kexec on arm fails, possibly due to memdup_user

kexec on arm fails, possibly due to memdup_user

[Amr Bekhit]

I'm trying to get kexec working on my Qualcom IPQ4019-based ARM
system. My system is built using yocto mickledore, but I'm using the
kernel from openwrt, so that's 5.15.110 (config:
https://pastebin.com/KgP2sJLf). I've compiled in kexec support in the
kernel and included the kexec tools. I'm then trying to load and
execute a zImage, but the load fails. I've added some debug messages
to the kexec_load function in the kernel to try and figure out where
it's failing (see https://pastebin.com/G6AEKakw). Here is the output
of the kexec commands, with the kernel debug messages interdispersed:

~ # kexec --version
kexec-tools 2.0.26
~ # kexec -d -l /data/zImage
Try gzip decompression.
Try LZMA decompression.
lzma_decompress_file: read on /data/zImage of 65536 bytes failed
kernel: 0xb6bea090 kernel_size: 0x324f00
MEMORY RANGES
0000000080000000-0000000087dfffff (0)
0000000088000000-000000009fffffff (0)
zImage header: 0x016f2818 0x00000000 0x00324f00
zImage size 0x324f00, file size 0x324f00
  offset 0x00004a08 tag 0x5a534c4b size 24
zImage requires 0x00335f00 bytes
Decompressed kernel sizes:
 text+data 0x00b90b00 bss 0x0003b8bc total 0x00bcc3bc
Resulting kernel space: 0x00ec6a00
Kernel: address=0x80008000 size=0x00ec6a00
DT    : address=0x80ed0000 size=0x00007c0d
kexec_load: entry = 0x80008000 flags = 0x280000
[   36.388603] __do_sys_kexec_load kexec_load_check: 0
nr_segments = 2
segment[0].buf   = 0xb6bea090
segment[0].bufsz = 0x324f00
segment[0].mem   = 0x80008000
segment[0].memsz = 0x325000
segment[1].buf   = 0xb6be20b0
segment[1].bufsz = 0x7c0d
segment[1].mem   = 0x80ed0000
[   36.432305] __do_sys_kexec_load calling memdup_user...

[   36.459002] __do_sys_kexec_load do_kexec_load: -22
kexec_load failed: Invalid argument
entry       = 0x80008000 flags = 0x280000
nr_segments = 2
segment[0].buf   = 0xb6bea090
segment[0].bufsz = 0x324f00
segment[0].mem   = 0x80008000
segment[0].memsz = 0x325000
segment[1].buf   = 0xb6be20b0
segment[1].bufsz = 0x7c0d
segment[1].mem   = 0x80ed0000
segment[1].memsz = 0x8000

It appears that execution reaches the call to memdup_user, but then
something seems to go amiss there as neither the debug message
indicating an error in memdup_user nor the debug message before
do_kexec_load are called for some reason. I've verified that the
zImage I'm trying to load boots fine in U-boot.

Any thoughts would be appreciated.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: kexec on arm fails, possibly due to memdup_user

[Russell King (Oracle)]

On Sat, Jul 01, 2023 at 08:25:46PM +0100, Amr Bekhit wrote:
> I'm trying to get kexec working on my Qualcom IPQ4019-based ARM
> system. My system is built using yocto mickledore, but I'm using the
> kernel from openwrt, so that's 5.15.110 (config:
> https://pastebin.com/KgP2sJLf). I've compiled in kexec support in the
> kernel and included the kexec tools. I'm then trying to load and
> execute a zImage, but the load fails. I've added some debug messages
> to the kexec_load function in the kernel to try and figure out where
> it's failing (see https://pastebin.com/G6AEKakw). Here is the output
> of the kexec commands, with the kernel debug messages interdispersed:

It could be:

        /*
         * Validate that if the current HW supports SMP, then the SW supports
         * and implements CPU hotplug for the current HW. If not, we won't be
         * able to kexec reliably, so fail the prepare operation.
         */
        if (num_possible_cpus() > 1 && platform_can_secondary_boot() &&
            !platform_can_cpu_hotplug())
                return -EINVAL;

or:

                if (!memblock_is_region_memory(idmap_to_phys(current_segment->mem),
                                               current_segment->memsz))
                        return -EINVAL;

in machine_kexec_prepare().

If it's the former, then it means your platform has more tha one CPU,
but has no way to take the secondary CPUs offline.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: kexec on arm fails, possibly due to memdup_user

[Amr Bekhit]

> It could be:
>
>         /*
>          * Validate that if the current HW supports SMP, then the SW supports
>          * and implements CPU hotplug for the current HW. If not, we won't be
>          * able to kexec reliably, so fail the prepare operation.
>          */
>         if (num_possible_cpus() > 1 && platform_can_secondary_boot() &&
>             !platform_can_cpu_hotplug())
>                 return -EINVAL;

Thanks - this was exactly the issue. memdup_user was a red herring
caused by some missing printk's due to me not adding newlines. It's
now clear that for my CPU (IPQ4019), the cpu_kill function is not
implemented in the smp_operations struct.

Looking online, I was able to see that kexec does work on my cpu as
long as nr_cpus=1 is specified in the kernel command line (see
https://patchwork.kernel.org/project/linux-arm-msm/patch/CAAGQ2nQNQ-aFkcrQHNA6H5TZ1tTovtfO_0Ohfndn9jXy13Hc6A@mail.gmail.com/#22064531).

I've tried this myself, and indeed setting this option allows the
kexec load to perform successfully. However, when I run kexec -e,
after the "Bye!" message the system just hangs and reboots after a
while, presumably due to a watchdog timeout:

~ # kexec -d --dtb=/data/image-qcom-ipq4019-nerd.dtb -l /data/zImage-openwrt
Try gzip decompression.
Try LZMA decompression.
lzma_decompress_file: read on /data/zImage-openwrt of 65536 bytes failed
kernel: 0xb6b4b090 kernel_size: 0x325e10
MEMORY RANGES
0000000080000000-0000000087dfffff (0)
0000000088000000-000000009fffffff (0)
zImage header: 0x016f2818 0x00000000 0x00325e10
zImage size 0x325e10, file size 0x325e10
  offset 0x00004a08 tag 0x5a534c4b size 24
zImage requires 0x00336e10 bytes
Decompressed kernel sizes:
 text+data 0x00b90b00 bss 0x0003b8bc total 0x00bcc3bc
Resulting kernel space: 0x00ec7910
Kernel: address=0x80008000 size=0x00ec7910
DT    : address=0x80ed1000 size=0x00004c0d
kexec_load: entry = 0x80008000 flags = 0x280000
nr_segments = 2
segment[0].buf   = 0xb6b4b090
segment[0].bufsz = 0x325e10
segment[0].mem   = 0x80008000
segment[0].memsz = 0x326000
segment[1].buf   = 0xb6b460b0
segment[1].bufsz = 0x4c0d
segment[1].mem   = 0x80ed1000
segment[1].memsz = 0x5000
~ # kexec -e
[   47.431861] kexec_core: Starting new kernel
[   47.431908] Bye!

Any pointers as to what could be wrong at this point?

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: kexec on arm fails, possibly due to memdup_user

[Russell King (Oracle)]

On Tue, Jul 04, 2023 at 06:48:26PM +0100, Amr Bekhit wrote:
> > It could be:
> >
> >         /*
> >          * Validate that if the current HW supports SMP, then the SW supports
> >          * and implements CPU hotplug for the current HW. If not, we won't be
> >          * able to kexec reliably, so fail the prepare operation.
> >          */
> >         if (num_possible_cpus() > 1 && platform_can_secondary_boot() &&
> >             !platform_can_cpu_hotplug())
> >                 return -EINVAL;
> 
> Thanks - this was exactly the issue. memdup_user was a red herring
> caused by some missing printk's due to me not adding newlines. It's
> now clear that for my CPU (IPQ4019), the cpu_kill function is not
> implemented in the smp_operations struct.
> 
> Looking online, I was able to see that kexec does work on my cpu as
> long as nr_cpus=1 is specified in the kernel command line (see
> https://patchwork.kernel.org/project/linux-arm-msm/patch/CAAGQ2nQNQ-aFkcrQHNA6H5TZ1tTovtfO_0Ohfndn9jXy13Hc6A@mail.gmail.com/#22064531).
> 
> I've tried this myself, and indeed setting this option allows the
> kexec load to perform successfully. However, when I run kexec -e,
> after the "Bye!" message the system just hangs and reboots after a
> while, presumably due to a watchdog timeout:
> 
> ~ # kexec -d --dtb=/data/image-qcom-ipq4019-nerd.dtb -l /data/zImage-openwrt
...
> Any pointers as to what could be wrong at this point?

Try without --dtb

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: kexec on arm fails, possibly due to memdup_user

[Amr Bekhit]

> > Any pointers as to what could be wrong at this point?
>
> Try without --dtb

No luck unfortunately:

~ # kexec -d -l /data/zImage-yocto
Try gzip decompression.
Try LZMA decompression.
lzma_decompress_file: read on /data/zImage-yocto of 65536 bytes failed
kernel: 0xb6c10090 kernel_size: 0x300e78
MEMORY RANGES
0000000080000000-0000000087dfffff (0)
0000000088000000-000000009fffffff (0)
zImage header: 0x016f2818 0x00000000 0x00300e78
zImage size 0x300e78, file size 0x300e78
  offset 0x00006490 tag 0x5a534c4b size 24
zImage requires 0x00311e78 bytes
Decompressed kernel sizes:
 text+data 0x00b90a80 bss 0x0003b93c total 0x00bcc3bc
Resulting kernel space: 0x00ea28f8
Kernel: address=0x80008000 size=0x00ea28f8
DT    : address=0x80eac000 size=0x00007c0d
kexec_load: entry = 0x80008000 flags = 0x280000
nr_segments = 2
segment[0].buf   = 0xb6c10090
segment[0].bufsz = 0x300e78
segment[0].mem   = 0x80008000
segment[0].memsz = 0x301000
segment[1].buf   = 0xb6c080b0
segment[1].bufsz = 0x7c0d
segment[1].mem   = 0x80eac000
segment[1].memsz = 0x8000
~ # kexec -d -e
[   33.703708] kexec_core: Starting new kernel
[   33.703753] Bye!

I also tried setting the command-line parameter, just in case the
kernel is actually booting but for some reason maybe the console is
not working:

~ # kexec -d -l /data/zImage-yocto --command-line="rootfstype=ubifs
ubi.mtd=12 root=ubi0:rootfs1 console=ttyMSM0,115200"
Try gzip decompression.
Try LZMA decompression.
lzma_decompress_file: read on /data/zImage-yocto of 65536 bytes failed
kernel: 0xb6b4c090 kernel_size: 0x300e78
MEMORY RANGES
0000000080000000-0000000087dfffff (0)
0000000088000000-000000009fffffff (0)
zImage header: 0x016f2818 0x00000000 0x00300e78
zImage size 0x300e78, file size 0x300e78
  offset 0x00006490 tag 0x5a534c4b size 24
zImage requires 0x00311e78 bytes
Decompressed kernel sizes:
 text+data 0x00b90a80 bss 0x0003b93c total 0x00bcc3bc
Resulting kernel space: 0x00ea28f8
Kernel: address=0x80008000 size=0x00ea28f8
DT    : address=0x80eac000 size=0x00007c49
kexec_load: entry = 0x80008000 flags = 0x280000
nr_segments = 2
segment[0].buf   = 0xb6b4c090
segment[0].bufsz = 0x300e78
segment[0].mem   = 0x80008000
segment[0].memsz = 0x301000
segment[1].buf   = 0xb6b440b0
segment[1].bufsz = 0x7c49
segment[1].mem   = 0x80eac000
segment[1].memsz = 0x8000
~ # kexec -d -e
[   92.123415] kexec_core: Starting new kernel
[   92.123461] Bye!

Finally, I also tried adding the "reset_devices" kernel parameter.

~ # kexec -d -l /data/zImage-yocto --command-line="rootfstype=ubifs
ubi.mtd=12 root=ubi0:rootfs1 console=ttyMSM0,115200 reset_devices"
Try gzip decompression.
Try LZMA decompression.
lzma_decompress_file: read on /data/zImage-yocto of 65536 bytes failed
kernel: 0xb6bfd090 kernel_size: 0x300e78
MEMORY RANGES
0000000080000000-0000000087dfffff (0)
0000000088000000-000000009fffffff (0)
zImage header: 0x016f2818 0x00000000 0x00300e78
zImage size 0x300e78, file size 0x300e78
  offset 0x00006490 tag 0x5a534c4b size 24
zImage requires 0x00311e78 bytes
Decompressed kernel sizes:
 text+data 0x00b90a80 bss 0x0003b93c total 0x00bcc3bc
Resulting kernel space: 0x00ea28f8
Kernel: address=0x80008000 size=0x00ea28f8
DT    : address=0x80eac000 size=0x00007c59
kexec_load: entry = 0x80008000 flags = 0x280000
nr_segments = 2
segment[0].buf   = 0xb6bfd090
segment[0].bufsz = 0x300e78
segment[0].mem   = 0x80008000
segment[0].memsz = 0x301000
segment[1].buf   = 0xb6bf50b0
segment[1].bufsz = 0x7c59
segment[1].mem   = 0x80eac000
segment[1].memsz = 0x8000
~ # kexec -d -e
[   96.913783] kexec_core: Starting new kernel
[   96.913829] Bye!

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel