[PATCH v2] of, numa: Return NUMA_NO_NODE from disable of_node_to_nid() if nid not possible.

[PATCH v2] of, numa: Return NUMA_NO_NODE from disable of_node_to_nid() if nid not possible.

[David Daney]

From: David Daney <david.daney@cavium.com>

On arm64 NUMA kernels we can pass "numa=off" on the command line to
disable NUMA.  A side effect of this is that kmalloc_node() calls to
non-zero nodes will crash the system with an OOPS:

[    0.000000] ITS@0x0000901000020000: allocated 2097152 Devices @10002000000 (flat, esz 8, psz 64K, shr 1)
[    0.000000] Unable to handle kernel NULL pointer dereference at virtual address 00001680
[    0.000000] pgd = fffffc0009470000
[    0.000000] [00001680] *pgd=0000010ffff90003, *pud=0000010ffff90003, *pmd=0000010ffff90003, *pte=0000000000000000
[    0.000000] Internal error: Oops: 96000006 [#1] SMP
.
.
.
[    0.000000] [<fffffc00081c8950>] __alloc_pages_nodemask+0xa4/0xe68
[    0.000000] [<fffffc000821fa70>] new_slab+0xd0/0x564
[    0.000000] [<fffffc0008221e24>] ___slab_alloc+0x2e4/0x514
[    0.000000] [<fffffc0008239498>] __slab_alloc+0x48/0x58
[    0.000000] [<fffffc0008222c20>] __kmalloc_node+0xd0/0x2dc
[    0.000000] [<fffffc0008115374>] __irq_domain_add+0x7c/0x164
[    0.000000] [<fffffc0008b461dc>] its_probe+0x784/0x81c
[    0.000000] [<fffffc0008b462bc>] its_init+0x48/0x1b0
[    0.000000] [<fffffc0008b4543c>] gic_init_bases+0x228/0x360
[    0.000000] [<fffffc0008b456bc>] gic_of_init+0x148/0x1cc
[    0.000000] [<fffffc0008b5aec8>] of_irq_init+0x184/0x298
[    0.000000] [<fffffc0008b43f9c>] irqchip_init+0x14/0x38
[    0.000000] [<fffffc0008b12d60>] init_IRQ+0xc/0x30
[    0.000000] [<fffffc0008b10a3c>] start_kernel+0x240/0x3b8
[    0.000000] [<fffffc0008b101c4>] __primary_switched+0x30/0x6c
[    0.000000] Code: 912ec2a0 b9403809 0a0902fb 37b007db (f9400300)
.
.
.

This is caused by code like this in kernel/irq/irqdomain.c

    domain = kzalloc_node(sizeof(*domain) + (sizeof(unsigned int) * size),
                  GFP_KERNEL, of_node_to_nid(of_node));

When NUMA is disabled, the concept of a node is really undefined, so
of_node_to_nid() should unconditionally return NUMA_NO_NODE.

Fix by returning NUMA_NO_NODE when the nid is not in the set of
possible nodes.

Reported-by: Gilbert Netzer <noname@pdc.kth.se>
Signed-off-by: David Daney <david.daney@cavium.com>
---
 drivers/of/of_numa.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c
index f63d4b0d..a53982a 100644
--- a/drivers/of/of_numa.c
+++ b/drivers/of/of_numa.c
@@ -176,7 +176,12 @@ int of_node_to_nid(struct device_node *device)
 			np->name);
 	of_node_put(np);
 
-	if (!r)
+	/*
+	 * If numa=off passed on command line, or with a defective
+	 * device tree, the nid may not be in the set of possible
+	 * nodes.  Check for this case and return NUMA_NO_NODE.
+	 */
+	if (!r && nid < MAX_NUMNODES && node_possible(nid))
 		return nid;
 
 	return NUMA_NO_NODE;
-- 
1.8.3.1

Re: [PATCH v2] of, numa: Return NUMA_NO_NODE from disable of_node_to_nid() if nid not possible.

[Rob Herring]

On Fri, Oct 28, 2016 at 4:15 PM, David Daney <ddaney.cavm-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> From: David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
>
> On arm64 NUMA kernels we can pass "numa=off" on the command line to
> disable NUMA.  A side effect of this is that kmalloc_node() calls to
> non-zero nodes will crash the system with an OOPS:
>
> [    0.000000] ITS@0x0000901000020000: allocated 2097152 Devices @10002000000 (flat, esz 8, psz 64K, shr 1)
> [    0.000000] Unable to handle kernel NULL pointer dereference at virtual address 00001680
> [    0.000000] pgd = fffffc0009470000
> [    0.000000] [00001680] *pgd=0000010ffff90003, *pud=0000010ffff90003, *pmd=0000010ffff90003, *pte=0000000000000000
> [    0.000000] Internal error: Oops: 96000006 [#1] SMP
> .
> .
> .
> [    0.000000] [<fffffc00081c8950>] __alloc_pages_nodemask+0xa4/0xe68
> [    0.000000] [<fffffc000821fa70>] new_slab+0xd0/0x564
> [    0.000000] [<fffffc0008221e24>] ___slab_alloc+0x2e4/0x514
> [    0.000000] [<fffffc0008239498>] __slab_alloc+0x48/0x58
> [    0.000000] [<fffffc0008222c20>] __kmalloc_node+0xd0/0x2dc
> [    0.000000] [<fffffc0008115374>] __irq_domain_add+0x7c/0x164
> [    0.000000] [<fffffc0008b461dc>] its_probe+0x784/0x81c
> [    0.000000] [<fffffc0008b462bc>] its_init+0x48/0x1b0
> [    0.000000] [<fffffc0008b4543c>] gic_init_bases+0x228/0x360
> [    0.000000] [<fffffc0008b456bc>] gic_of_init+0x148/0x1cc
> [    0.000000] [<fffffc0008b5aec8>] of_irq_init+0x184/0x298
> [    0.000000] [<fffffc0008b43f9c>] irqchip_init+0x14/0x38
> [    0.000000] [<fffffc0008b12d60>] init_IRQ+0xc/0x30
> [    0.000000] [<fffffc0008b10a3c>] start_kernel+0x240/0x3b8
> [    0.000000] [<fffffc0008b101c4>] __primary_switched+0x30/0x6c
> [    0.000000] Code: 912ec2a0 b9403809 0a0902fb 37b007db (f9400300)
> .
> .
> .
>
> This is caused by code like this in kernel/irq/irqdomain.c
>
>     domain = kzalloc_node(sizeof(*domain) + (sizeof(unsigned int) * size),
>                   GFP_KERNEL, of_node_to_nid(of_node));
>
> When NUMA is disabled, the concept of a node is really undefined, so
> of_node_to_nid() should unconditionally return NUMA_NO_NODE.
>
> Fix by returning NUMA_NO_NODE when the nid is not in the set of
> possible nodes.
>
> Reported-by: Gilbert Netzer <noname-QVTQ4TwcpgDLoDKTGw+V6w@public.gmane.org>
> Signed-off-by: David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>

Does this need to go in 4.9? stable? If so, since what kernel version?

Rob
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] of, numa: Return NUMA_NO_NODE from disable of_node_to_nid() if nid not possible.

[David Daney]

On 11/02/2016 08:37 PM, Rob Herring wrote:
> On Fri, Oct 28, 2016 at 4:15 PM, David Daney <ddaney.cavm@gmail.com> wrote:
>> From: David Daney <david.daney@cavium.com>
>>
>> On arm64 NUMA kernels we can pass "numa=off" on the command line to
>> disable NUMA.  A side effect of this is that kmalloc_node() calls to
>> non-zero nodes will crash the system with an OOPS:
>>
>> [    0.000000] ITS@0x0000901000020000: allocated 2097152 Devices @10002000000 (flat, esz 8, psz 64K, shr 1)
>> [    0.000000] Unable to handle kernel NULL pointer dereference at virtual address 00001680
>> [    0.000000] pgd = fffffc0009470000
>> [    0.000000] [00001680] *pgd=0000010ffff90003, *pud=0000010ffff90003, *pmd=0000010ffff90003, *pte=0000000000000000
>> [    0.000000] Internal error: Oops: 96000006 [#1] SMP
>> .
>> .
>> .
>> [    0.000000] [<fffffc00081c8950>] __alloc_pages_nodemask+0xa4/0xe68
>> [    0.000000] [<fffffc000821fa70>] new_slab+0xd0/0x564
>> [    0.000000] [<fffffc0008221e24>] ___slab_alloc+0x2e4/0x514
>> [    0.000000] [<fffffc0008239498>] __slab_alloc+0x48/0x58
>> [    0.000000] [<fffffc0008222c20>] __kmalloc_node+0xd0/0x2dc
>> [    0.000000] [<fffffc0008115374>] __irq_domain_add+0x7c/0x164
>> [    0.000000] [<fffffc0008b461dc>] its_probe+0x784/0x81c
>> [    0.000000] [<fffffc0008b462bc>] its_init+0x48/0x1b0
>> [    0.000000] [<fffffc0008b4543c>] gic_init_bases+0x228/0x360
>> [    0.000000] [<fffffc0008b456bc>] gic_of_init+0x148/0x1cc
>> [    0.000000] [<fffffc0008b5aec8>] of_irq_init+0x184/0x298
>> [    0.000000] [<fffffc0008b43f9c>] irqchip_init+0x14/0x38
>> [    0.000000] [<fffffc0008b12d60>] init_IRQ+0xc/0x30
>> [    0.000000] [<fffffc0008b10a3c>] start_kernel+0x240/0x3b8
>> [    0.000000] [<fffffc0008b101c4>] __primary_switched+0x30/0x6c
>> [    0.000000] Code: 912ec2a0 b9403809 0a0902fb 37b007db (f9400300)
>> .
>> .
>> .
>>
>> This is caused by code like this in kernel/irq/irqdomain.c
>>
>>     domain = kzalloc_node(sizeof(*domain) + (sizeof(unsigned int) * size),
>>                   GFP_KERNEL, of_node_to_nid(of_node));
>>
>> When NUMA is disabled, the concept of a node is really undefined, so
>> of_node_to_nid() should unconditionally return NUMA_NO_NODE.
>>
>> Fix by returning NUMA_NO_NODE when the nid is not in the set of
>> possible nodes.
>>
>> Reported-by: Gilbert Netzer <noname@pdc.kth.se>
>> Signed-off-by: David Daney <david.daney@cavium.com>
>
> Does this need to go in 4.9?

That would be my preference.

> stable? If so, since what kernel version?
>

v4.7 and later would be nice.

I guess if you merge it, you could add the Cc: stable@ tag


Thanks for looking at this,
David Daney

Re: [PATCH v2] of, numa: Return NUMA_NO_NODE from disable of_node_to_nid() if nid not possible.

[Rob Herring]

On Thu, Nov 3, 2016 at 10:11 AM, David Daney <ddaney@caviumnetworks.com> wrote:
> On 11/02/2016 08:37 PM, Rob Herring wrote:
>>
>> On Fri, Oct 28, 2016 at 4:15 PM, David Daney <ddaney.cavm@gmail.com>
>> wrote:
>>>
>>> From: David Daney <david.daney@cavium.com>
>>>
>>> On arm64 NUMA kernels we can pass "numa=off" on the command line to
>>> disable NUMA.  A side effect of this is that kmalloc_node() calls to
>>> non-zero nodes will crash the system with an OOPS:
>>>
>>> [    0.000000] ITS@0x0000901000020000: allocated 2097152 Devices
>>> @10002000000 (flat, esz 8, psz 64K, shr 1)
>>> [    0.000000] Unable to handle kernel NULL pointer dereference at
>>> virtual address 00001680
>>> [    0.000000] pgd = fffffc0009470000
>>> [    0.000000] [00001680] *pgd=0000010ffff90003, *pud=0000010ffff90003,
>>> *pmd=0000010ffff90003, *pte=0000000000000000
>>> [    0.000000] Internal error: Oops: 96000006 [#1] SMP
>>> .
>>> .
>>> .
>>> [    0.000000] [<fffffc00081c8950>] __alloc_pages_nodemask+0xa4/0xe68
>>> [    0.000000] [<fffffc000821fa70>] new_slab+0xd0/0x564
>>> [    0.000000] [<fffffc0008221e24>] ___slab_alloc+0x2e4/0x514
>>> [    0.000000] [<fffffc0008239498>] __slab_alloc+0x48/0x58
>>> [    0.000000] [<fffffc0008222c20>] __kmalloc_node+0xd0/0x2dc
>>> [    0.000000] [<fffffc0008115374>] __irq_domain_add+0x7c/0x164
>>> [    0.000000] [<fffffc0008b461dc>] its_probe+0x784/0x81c
>>> [    0.000000] [<fffffc0008b462bc>] its_init+0x48/0x1b0
>>> [    0.000000] [<fffffc0008b4543c>] gic_init_bases+0x228/0x360
>>> [    0.000000] [<fffffc0008b456bc>] gic_of_init+0x148/0x1cc
>>> [    0.000000] [<fffffc0008b5aec8>] of_irq_init+0x184/0x298
>>> [    0.000000] [<fffffc0008b43f9c>] irqchip_init+0x14/0x38
>>> [    0.000000] [<fffffc0008b12d60>] init_IRQ+0xc/0x30
>>> [    0.000000] [<fffffc0008b10a3c>] start_kernel+0x240/0x3b8
>>> [    0.000000] [<fffffc0008b101c4>] __primary_switched+0x30/0x6c
>>> [    0.000000] Code: 912ec2a0 b9403809 0a0902fb 37b007db (f9400300)
>>> .
>>> .
>>> .
>>>
>>> This is caused by code like this in kernel/irq/irqdomain.c
>>>
>>>     domain = kzalloc_node(sizeof(*domain) + (sizeof(unsigned int) *
>>> size),
>>>                   GFP_KERNEL, of_node_to_nid(of_node));
>>>
>>> When NUMA is disabled, the concept of a node is really undefined, so
>>> of_node_to_nid() should unconditionally return NUMA_NO_NODE.
>>>
>>> Fix by returning NUMA_NO_NODE when the nid is not in the set of
>>> possible nodes.
>>>
>>> Reported-by: Gilbert Netzer <noname@pdc.kth.se>
>>> Signed-off-by: David Daney <david.daney@cavium.com>
>>
>>
>> Does this need to go in 4.9?
>
>
> That would be my preference.

Given how late this is now, my having nothing else for 4.9 and that
his has never worked, I've applied for 4.10, but I did tag for stable.

Rob