From: Vijay Balakrishna <vijayb@linux.microsoft.com>
To: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>,
Catalin Marinas <catalin.marinas@arm.com>
Cc: Linux kernel mailing list <linux-kernel@vger.kernel.org>,
linux-arm-kernel@lists.infradead.org
Subject: [arm64] kernel boot slowdown in v5.10.19 -> v5.10.42 update
Date: Mon, 24 Jan 2022 15:03:48 -0800 [thread overview]
Message-ID: <9a34ee9b-0ede-30a6-0898-d32fe81d5b0c@linux.microsoft.com> (raw)
We noticed 150ms kernel boot slowdown back in June, 2021, when moving
from v5.10.19 to v5.10.42. This on a 8GB SoC. Only recently we
investigated this issue and found the regression is introduced by a
change in map_mem() (paging_init() -> map_mem() -> __map_memblock(), in
particular "map all the memory banks" for loop) by patch
2687275a5843d1089687f08fc64eb3f3b026a169
arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required
above is a follow up to
0a30c53573b07d5561457e41fb0ab046cd857da5
arm64: mm: Move reserve_crashkernel() into mem_init())
which deferred crashkernel reservation into mem_init().
The ~150ms slowdown disappears on booting without "crashkernel=.." on
kernel command-line.
Please let us know if it known/understood or addressed already or being
looked into. Any insight is appreciated.
Thanks,
Vijay
console output collected with debug prints --
[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[ 0.000000] Linux version
5.10.42.6-xxx-standard-2008.3.0-local-snapshot-01210432
(oe-user@oe-host) (clang version 10.[ 0.000001] sched_clock: 64 bits
at 125MHz, resolution 8ns, wraps every 4398046511100ns
[ 0.000003] 2.560781952 MSFT: kernel boot start
[ 0.000484] Machine model: xxx
[ 0.009379] earlycon: uart8250 at MMIO32 0x0000000068a10000 (options '')
[ 0.009383] printk: bootconsole [uart8250] enabled
[ 0.009828] before paging_init()
[ 0.009977] before map_mem()
[ 0.009978] after memblock_mark_nomap()
[ 0.152388] after __map_memblock() for loop
[ 0.152391] after __map_memblock() kernel_start, kernel_end
[ 0.152393] after memblock_clear_nomap
[ 0.152394] after map_mem()
[ 0.152401] after paging_init()
[ 0.153273] NUMA: No NUMA configuration found
[ 0.153278] NUMA: Faking a node at [mem
0x0000000080000000-0x000000097fffffff]
[ 0.153282] NUMA: NODE_DATA [mem 0x97efec5c0-0x97efedfff]
[ 0.153316] Zone ranges:
[ 0.153319] Normal [mem 0x0000000080000000-0x000000097fffffff]
[ 0.153323] Device empty
[ 0.153325] Movable zone start for each node
[ 0.153326] Early memory node ranges
[ 0.153328] node 0: [mem 0x0000000080000000-0x000000008adfffff]
[ 0.153330] node 0: [mem 0x000000008ae00000-0x000000008f223fff]
[ 0.153331] node 0: [mem 0x000000008f224000-0x00000000ffffffff]
[ 0.153333] node 0: [mem 0x0000000880000000-0x00000008800fffff]
[ 0.153334] node 0: [mem 0x0000000880100000-0x000000097fffffff]
[ 0.153337] Initmem setup node 0 [mem
0x0000000080000000-0x000000097fffffff]
[ 0.153340] On node 0 totalpages: 1572864
[ 0.153342] Normal zone: 24576 pages used for memmap
[ 0.153343] Normal zone: 0 pages reserved
[ 0.153345] Normal zone: 1572864 pages, LIFO batch:63
[ 0.154507] crashkernel reserved: 0x0000000968e00000 -
0x0000000978e00000 (256 MB)
Console output from PXE boot without "crashkernel=256M" boot parameter --
[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[ 0.000000] Linux version
5.10.42.6-xxx-standard-2008.3.0-local-snapshot-01210432
(oe-user@oe-host) (clang version 10.[ 0.000001] sched_clock: 64 bits
at 125MHz, resolution 8ns, wraps every 4398046511100ns
[ 0.000003] 14.765446920 MSFT: kernel boot start
[ 0.000484] Machine model: xxx
[ 0.009400] earlycon: uart8250 at MMIO32 0x0000000068a10000 (options '')
[ 0.009404] printk: bootconsole [uart8250] enabled
[ 0.009835] before paging_init()
[ 0.009986] before map_mem()
[ 0.009987] after memblock_mark_nomap()
[ 0.010154] after __map_memblock() for loop
[ 0.010156] after __map_memblock() kernel_start, kernel_end
[ 0.010158] after memblock_clear_nomap
[ 0.010159] after map_mem()
[ 0.010163] after paging_init()
[ 0.011002] NUMA: No NUMA configuration found
[ 0.011005] NUMA: Faking a node at [mem
0x0000000080000000-0x000000097fffffff]
[ 0.011009] NUMA: NODE_DATA [mem 0x97fbc65c0-0x97fbc7fff]
[ 0.011037] Zone ranges:
[ 0.011038] Normal [mem 0x0000000080000000-0x000000097fffffff]
[ 0.011041] Device empty
[ 0.011043] Movable zone start for each node
[ 0.011044] Early memory node ranges
[ 0.011046] node 0: [mem 0x0000000080000000-0x000000008adfffff]
[ 0.011048] node 0: [mem 0x000000008ae00000-0x000000008f223fff]
[ 0.011049] node 0: [mem 0x000000008f224000-0x00000000ffffffff]
[ 0.011050] node 0: [mem 0x0000000880000000-0x00000008800fffff]
[ 0.011052] node 0: [mem 0x0000000880100000-0x000000097fffffff]
[ 0.011054] Initmem setup node 0 [mem
0x0000000080000000-0x000000097fffffff]
[ 0.011056] On node 0 totalpages: 1572864
[ 0.011058] Normal zone: 24576 pages used for memmap
[ 0.011060] Normal zone: 0 pages reserved
[ 0.011061] Normal zone: 1572864 pages, LIFO batch:63
next reply other threads:[~2022-01-24 23:47 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-24 23:03 Vijay Balakrishna [this message]
2022-01-28 12:06 ` [arm64] kernel boot slowdown in v5.10.19 -> v5.10.42 update Catalin Marinas
2022-01-28 18:03 ` Vijay Balakrishna
2022-02-09 21:47 ` Vijay Balakrishna
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=9a34ee9b-0ede-30a6-0898-d32fe81d5b0c@linux.microsoft.com \
--to=vijayb@linux.microsoft.com \
--cc=catalin.marinas@arm.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=nsaenzjulienne@suse.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox