Re: Arm64 crash while reading memory sysfs

[Mike Rapoport <rppt@linux.ibm.com>]

On Wed, May 26, 2021 at 08:16:14PM -0400, Qian Cai wrote:
> 
> On 5/26/2021 1:24 PM, Mike Rapoport wrote:
> > On Wed, May 26, 2021 at 12:09:14PM +0000, Qian Cai (QUIC) wrote:
> >>>
> >>> On Tue, May 25, 2021 at 03:25:59PM +0000, Qian Cai (QUIC) wrote:
> >>>> Reverting the patchset "arm64: drop pfn_valid_within() and simplify pfn_valid()" [1] from today's linux-next fixed a crash while
> >>> reading files under /sys/devices/system/memory.
> > 
> > Does the issue persist of you only revert the latest patch in the series?
> > In next-20210525 it would be commit 
> > 89fb47db72f2 ("arm64-drop-pfn_valid_within-and-simplify-pfn_valid-fix")
> > and commit
> > dfe215e9bac2 ("arm64: drop pfn_valid_within() and simplify pfn_valid()").
> 
> Reverting those two commits alone is enough to fix the issue.
> 
> > 
> >>> Can you please send the beginning of the boot log, up to the
> >>> 	 "Memory: xK/yK available ..."
> >>> line?
> >>
> >> [    0.000000] NUMA: Failed to initialise from firmware
> >> [    0.000000] NUMA: Faking a node at [mem 0x0000000090000000-0x0000009fffffffff]
> >> [    0.000000] NUMA: NODE_DATA [mem 0x9ffefbabc0-0x9ffefbffff]
> >> [    0.000000] Zone ranges:
> >> [    0.000000]   Normal   [mem 0x0000000090000000-0x0000009fffffffff]
> >> [    0.000000] Movable zone start for each node
> >> [    0.000000] Early memory node ranges
> >> [    0.000000]   node   0: [mem 0x0000000090000000-0x0000000091ffffff]
> >> [    0.000000]   node   0: [mem 0x0000000092000000-0x00000000928fffff]
> >> [    0.000000]   node   0: [mem 0x0000000092900000-0x00000000fffbffff]
> >> [    0.000000]   node   0: [mem 0x00000000fffc0000-0x00000000ffffffff]
> >> [    0.000000]   node   0: [mem 0x0000000880000000-0x0000000fffffffff]
> >> [    0.000000]   node   0: [mem 0x0000008800000000-0x0000009ff5aeffff]
> >> [    0.000000]   node   0: [mem 0x0000009ff5af0000-0x0000009ff5b2ffff]
> >> [    0.000000]   node   0: [mem 0x0000009ff5b30000-0x0000009ff5baffff]
> >> [    0.000000]   node   0: [mem 0x0000009ff5bb0000-0x0000009ff7deffff]
> >> [    0.000000]   node   0: [mem 0x0000009ff7df0000-0x0000009ff7e5ffff]
> >> [    0.000000]   node   0: [mem 0x0000009ff7e60000-0x0000009ff7ffffff]
> >> [    0.000000]   node   0: [mem 0x0000009ff8000000-0x0000009fffffffff]
> >> [    0.000000] Initmem setup node 0 [mem 0x0000000090000000-0x0000009fffffffff]
> >> [    0.000000] mem auto-init: stack:off, heap alloc:on, heap free:off
> >> [    0.000000] Memory: 777216K/133955584K available (17920K kernel code, 118786K rwdata, 4416K rodata, 6080K init, 67276K bss, 17379072K reserved, 0K cma-reserved)
> > 
> > The available and reserved sizes look weird. Can you post the log with
> > memblock=debug and mminit_loglevel=4 added to the kernel command line?
> 
> http://www.lsbug.org/tmp/dmesg.txt

It seems cut in the middle and even then it's too long to be useful.

Let's drop memblock=debug for now and add this instead:

diff --git a/mm/memblock.c b/mm/memblock.c
index afaefa8fc6ab..3f888bef1994 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -2055,6 +2055,8 @@ void __init memblock_free_all(void)
 {
 	unsigned long pages;
 
+	__memblock_dump_all();
+
 	free_unused_memmap();
 	reset_all_zones_managed_pages();
 
> >>>> [1] https://lore.kernel.org/kvmarm/20210511100550.28178-1-rppt@kernel.org/
> >>>>
> >>>> [  247.669668][ T1443] kernel BUG at include/linux/mm.h:1383!
> >>>> [  247.675987][ T1443] Internal error: Oops - BUG: 0 [#1] SMP
> >>>> [  247.681472][ T1443] Modules linked in: loop processor efivarfs ip_tables x_tables ext4 mbcache jbd2 dm_mod igb i2c_algo_bit
> >>> nvme mlx5_core i2c_core nvme_core firmware_class
> >>>> [  247.696894][ T1443] CPU: 15 PID: 1443 Comm: ranbug Not tainted 5.13.0-rc3-next-20210524+ #11
> >>>> [  247.705326][ T1443] Hardware name: MiTAC RAPTOR EV-883832-X3-0001/RAPTOR, BIOS 1.6 06/28/2020
> >>>> [  247.713842][ T1443] pstate: 60000005 (nZCv daif -PAN -UAO -TCO BTYPE=--)
> >>>> [  247.720536][ T1443] pc : test_pages_in_a_zone+0x23c/0x300
> >>>> [  247.725935][ T1443] lr : test_pages_in_a_zone+0x23c/0x300
> > 
> > Do we know what PFN triggers it? Can you please run with this patch:
> 
> Nothing useful showed up with this patch. Yes, I double-checked that the patch was applied.

Sorry, I've missed that the BUG is apparently triggered for pfn + i. Can
you please try this instead:


diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 70620d0dd923..d0e42e09ad84 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1447,6 +1447,13 @@ struct zone *test_pages_in_a_zone(unsigned long start_pfn,
 			if (zone && !zone_spans_pfn(zone, pfn + i))
 				return NULL;
 			page = pfn_to_page(pfn + i);
+
+			if (!pfn_valid(pfn + i))
+				pr_info("%s: pfn %lx is not valid\n", __func__, pfn + i);
+			else if (PagePoisoned(page))
+				dump_page(page, "");
+
+
 			if (zone && page_zone(page) != zone)
 				return NULL;
 			zone = page_zone(page);
 
-- 
Sincerely yours,
Mike.