[PATCH v2] arch_numa: Restore nid checks before registering a memblock with a node

[PATCH v2] arch_numa: Restore nid checks before registering a memblock with a node

[Marc Zyngier]

Commit 767507654c22 ("arch_numa: switch over to numa_memblks")
significantly cleaned up the NUMA registration code, but also
dropped a significant check that was refusing to accept to
configure a memblock with an invalid nid.

On "quality hardware" such as my ThunderX machine, this results
in a kernel that dies immediately:

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x431f0a10]
[    0.000000] Linux version 6.12.0-00013-g8920d74cf8db (maz@valley-girl) (gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #3872 SMP PREEMPT Wed Nov 27 15:25:49 GMT 2024
[    0.000000] KASLR disabled due to lack of seed
[    0.000000] Machine model: Cavium ThunderX CN88XX board
[    0.000000] efi: EFI v2.4 by American Megatrends
[    0.000000] efi: ESRT=0xffce0ff18 SMBIOS 3.0=0xfffb0000 ACPI 2.0=0xffec60000 MEMRESERVE=0xffc905d98
[    0.000000] esrt: Reserving ESRT space from 0x0000000ffce0ff18 to 0x0000000ffce0ff50.
[    0.000000] earlycon: pl11 at MMIO 0x000087e024000000 (options '115200n8')
[    0.000000] printk: legacy bootconsole [pl11] enabled
[    0.000000] NODE_DATA(0) allocated [mem 0xff6754580-0xff67566bf]
[    0.000000] Unable to handle kernel paging request at virtual address 0000000000001d40
[    0.000000] Mem abort info:
[    0.000000]   ESR = 0x0000000096000004
[    0.000000]   EC = 0x25: DABT (current EL), IL = 32 bits
[    0.000000]   SET = 0, FnV = 0
[    0.000000]   EA = 0, S1PTW = 0
[    0.000000]   FSC = 0x04: level 0 translation fault
[    0.000000] Data abort info:
[    0.000000]   ISV = 0, ISS = 0x00000004, ISS2 = 0x00000000
[    0.000000]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
[    0.000000]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
[    0.000000] [0000000000001d40] user address but active_mm is swapper
[    0.000000] Internal error: Oops: 0000000096000004 [#1] PREEMPT SMP
[    0.000000] Modules linked in:
[    0.000000] CPU: 0 UID: 0 PID: 0 Comm: swapper Not tainted 6.12.0-00013-g8920d74cf8db #3872
[    0.000000] Hardware name: Cavium ThunderX CN88XX board (DT)
[    0.000000] pstate: a00000c5 (NzCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[    0.000000] pc : sparse_init_nid+0x54/0x428
[    0.000000] lr : sparse_init+0x118/0x240
[    0.000000] sp : ffff800081da3cb0
[    0.000000] x29: ffff800081da3cb0 x28: 0000000fedbab10c x27: 0000000000000001
[    0.000000] x26: 0000000ffee250f8 x25: 0000000000000001 x24: ffff800082102cd0
[    0.000000] x23: 0000000000000001 x22: 0000000000000000 x21: 00000000001fffff
[    0.000000] x20: 0000000000000001 x19: 0000000000000000 x18: ffffffffffffffff
[    0.000000] x17: 0000000001b00000 x16: 0000000ffd130000 x15: 0000000000000000
[    0.000000] x14: 00000000003e0000 x13: 00000000000001c8 x12: 0000000000000014
[    0.000000] x11: ffff800081e82860 x10: ffff8000820fb2c8 x9 : ffff8000820fb490
[    0.000000] x8 : 0000000000ffed20 x7 : 0000000000000014 x6 : 00000000001fffff
[    0.000000] x5 : 00000000ffffffff x4 : 0000000000000000 x3 : 0000000000000000
[    0.000000] x2 : 0000000000000000 x1 : 0000000000000040 x0 : 0000000000000007
[    0.000000] Call trace:
[    0.000000]  sparse_init_nid+0x54/0x428
[    0.000000]  sparse_init+0x118/0x240
[    0.000000]  bootmem_init+0x70/0x1c8
[    0.000000]  setup_arch+0x184/0x270
[    0.000000]  start_kernel+0x74/0x670
[    0.000000]  __primary_switched+0x80/0x90
[    0.000000] Code: f865d804 d37df060 cb030000 d2800003 (b95d4084)
[    0.000000] ---[ end trace 0000000000000000 ]---
[    0.000000] Kernel panic - not syncing: Attempted to kill the idle task!
[    0.000000] ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]---

while previous kernel versions were able to recognise how brain-damaged
the machine is, and only build a fake node.

Use the memblock_validate_numa_coverage() helper to restore some sanity
and a "working" system.

Fixes: 767507654c22 ("arch_numa: switch over to numa_memblks")
Suggested-by: Mike Rapoport <rppt@kernel.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Zi Yan <ziy@nvidia.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: stable@vger.kernel.org
---
 drivers/base/arch_numa.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/base/arch_numa.c b/drivers/base/arch_numa.c
index e187016764265..c63a72a1fed64 100644
--- a/drivers/base/arch_numa.c
+++ b/drivers/base/arch_numa.c
@@ -208,6 +208,10 @@ static int __init numa_register_nodes(void)
 {
 	int nid;
 
+	/* Check the validity of the memblock/node mapping */
+	if (!memblock_validate_numa_coverage(1))
+		return -EINVAL;
+
 	/* Finally register nodes. */
 	for_each_node_mask(nid, numa_nodes_parsed) {
 		unsigned long start_pfn, end_pfn;
-- 
2.39.2

Re: [PATCH v2] arch_numa: Restore nid checks before registering a memblock with a node

[Mike Rapoport]

Hi Marc,

On Sun, Dec 01, 2024 at 09:27:02AM +0000, Marc Zyngier wrote:
> Commit 767507654c22 ("arch_numa: switch over to numa_memblks")
> significantly cleaned up the NUMA registration code, but also
> dropped a significant check that was refusing to accept to
> configure a memblock with an invalid nid.

... 
 
> while previous kernel versions were able to recognise how brain-damaged
> the machine is, and only build a fake node.
> 
> Use the memblock_validate_numa_coverage() helper to restore some sanity
> and a "working" system.
> 
> Fixes: 767507654c22 ("arch_numa: switch over to numa_memblks")
> Suggested-by: Mike Rapoport <rppt@kernel.org>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will@kernel.org>
> Cc: Zi Yan <ziy@nvidia.com>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Cc: David Hildenbrand <david@redhat.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: stable@vger.kernel.org
> ---
>  drivers/base/arch_numa.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/drivers/base/arch_numa.c b/drivers/base/arch_numa.c
> index e187016764265..c63a72a1fed64 100644
> --- a/drivers/base/arch_numa.c
> +++ b/drivers/base/arch_numa.c
> @@ -208,6 +208,10 @@ static int __init numa_register_nodes(void)
>  {
>  	int nid;
>  
> +	/* Check the validity of the memblock/node mapping */
> +	if (!memblock_validate_numa_coverage(1))

I've changed this to memblock_validate_numa_coverage(0) and applied along
with my patch that changed memblock_validate_numa_coverage() to work with
0:

https://git.kernel.org/pub/scm/linux/kernel/git/rppt/memblock.git/log/?h=thunderx-fix

Can you please verify that it works on your "quality hardware"?

> +		return -EINVAL;
> +
>  	/* Finally register nodes. */
>  	for_each_node_mask(nid, numa_nodes_parsed) {
>  		unsigned long start_pfn, end_pfn;
> -- 
> 2.39.2

-- 
Sincerely yours,
Mike.

Re: [PATCH v2] arch_numa: Restore nid checks before registering a memblock with a node

[Marc Zyngier]

Hi Mike,

On Sun, 01 Dec 2024 19:32:22 +0000,
Mike Rapoport <rppt@kernel.org> wrote:
> 
> Hi Marc,
> 
> On Sun, Dec 01, 2024 at 09:27:02AM +0000, Marc Zyngier wrote:
> > Commit 767507654c22 ("arch_numa: switch over to numa_memblks")
> > significantly cleaned up the NUMA registration code, but also
> > dropped a significant check that was refusing to accept to
> > configure a memblock with an invalid nid.
> 
> ... 
>  
> > while previous kernel versions were able to recognise how brain-damaged
> > the machine is, and only build a fake node.
> > 
> > Use the memblock_validate_numa_coverage() helper to restore some sanity
> > and a "working" system.
> > 
> > Fixes: 767507654c22 ("arch_numa: switch over to numa_memblks")
> > Suggested-by: Mike Rapoport <rppt@kernel.org>
> > Signed-off-by: Marc Zyngier <maz@kernel.org>
> > Cc: Catalin Marinas <catalin.marinas@arm.com>
> > Cc: Will Deacon <will@kernel.org>
> > Cc: Zi Yan <ziy@nvidia.com>
> > Cc: Dan Williams <dan.j.williams@intel.com>
> > Cc: David Hildenbrand <david@redhat.com>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: stable@vger.kernel.org
> > ---
> >  drivers/base/arch_numa.c | 4 ++++
> >  1 file changed, 4 insertions(+)
> > 
> > diff --git a/drivers/base/arch_numa.c b/drivers/base/arch_numa.c
> > index e187016764265..c63a72a1fed64 100644
> > --- a/drivers/base/arch_numa.c
> > +++ b/drivers/base/arch_numa.c
> > @@ -208,6 +208,10 @@ static int __init numa_register_nodes(void)
> >  {
> >  	int nid;
> >  
> > +	/* Check the validity of the memblock/node mapping */
> > +	if (!memblock_validate_numa_coverage(1))
> 
> I've changed this to memblock_validate_numa_coverage(0) and applied along
> with my patch that changed memblock_validate_numa_coverage() to work with
> 0:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/rppt/memblock.git/log/?h=thunderx-fix
> 
> Can you please verify that it works on your "quality hardware"?

Commit 427c6179e159b in your tree still has memblock_validate_numa_coverage(1).
Forgot to push out the updated version?

Flipping this to 0 locally, I have verified that this still allows the
old thing to trudge along:

root@duodenum:~# uname -a
Linux duodenum 6.12.0-12115-g427c6179e159-dirty #3896 SMP PREEMPT Sun Dec  1 19:43:13 GMT 2024 aarch64

Thanks again,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH v2] arch_numa: Restore nid checks before registering a memblock with a node

[Mike Rapoport]

On Sun, Dec 01, 2024 at 07:49:44PM +0000, Marc Zyngier wrote:
> Hi Mike,
> 
> On Sun, 01 Dec 2024 19:32:22 +0000,
> Mike Rapoport <rppt@kernel.org> wrote:
> > 
> > Hi Marc,
> > 
> > On Sun, Dec 01, 2024 at 09:27:02AM +0000, Marc Zyngier wrote:
> > > Commit 767507654c22 ("arch_numa: switch over to numa_memblks")
> > > significantly cleaned up the NUMA registration code, but also
> > > dropped a significant check that was refusing to accept to
> > > configure a memblock with an invalid nid.
> > 
> > ... 
> >  
> > > while previous kernel versions were able to recognise how brain-damaged
> > > the machine is, and only build a fake node.
> > > 
> > > Use the memblock_validate_numa_coverage() helper to restore some sanity
> > > and a "working" system.
> > > 
> > > Fixes: 767507654c22 ("arch_numa: switch over to numa_memblks")
> > > Suggested-by: Mike Rapoport <rppt@kernel.org>
> > > Signed-off-by: Marc Zyngier <maz@kernel.org>
> > > Cc: Catalin Marinas <catalin.marinas@arm.com>
> > > Cc: Will Deacon <will@kernel.org>
> > > Cc: Zi Yan <ziy@nvidia.com>
> > > Cc: Dan Williams <dan.j.williams@intel.com>
> > > Cc: David Hildenbrand <david@redhat.com>
> > > Cc: Andrew Morton <akpm@linux-foundation.org>
> > > Cc: stable@vger.kernel.org
> > > ---
> > >  drivers/base/arch_numa.c | 4 ++++
> > >  1 file changed, 4 insertions(+)
> > > 
> > > diff --git a/drivers/base/arch_numa.c b/drivers/base/arch_numa.c
> > > index e187016764265..c63a72a1fed64 100644
> > > --- a/drivers/base/arch_numa.c
> > > +++ b/drivers/base/arch_numa.c
> > > @@ -208,6 +208,10 @@ static int __init numa_register_nodes(void)
> > >  {
> > >  	int nid;
> > >  
> > > +	/* Check the validity of the memblock/node mapping */
> > > +	if (!memblock_validate_numa_coverage(1))
> > 
> > I've changed this to memblock_validate_numa_coverage(0) and applied along
> > with my patch that changed memblock_validate_numa_coverage() to work with
> > 0:
> > 
> > https://git.kernel.org/pub/scm/linux/kernel/git/rppt/memblock.git/log/?h=thunderx-fix
> > 
> > Can you please verify that it works on your "quality hardware"?
> 
> Commit 427c6179e159b in your tree still has memblock_validate_numa_coverage(1).
> Forgot to push out the updated version?

Argh, indeed. 
 
> Flipping this to 0 locally, I have verified that this still allows the
> old thing to trudge along:
> 
> root@duodenum:~# uname -a
> Linux duodenum 6.12.0-12115-g427c6179e159-dirty #3896 SMP PREEMPT Sun Dec  1 19:43:13 GMT 2024 aarch64

Thanks for testing!
 
> Thanks again,
> 
> 	M.

-- 
Sincerely yours,
Mike.