[PATCH] nvme-multipath: fix flex array size in struct nvme_ns_head

[PATCH] nvme-multipath: fix flex array size in struct nvme_ns_head

[Nilay Shroff]

struct nvme_ns_head contains a flexible array member, current_path[],
which is indexed using the NUMA node ID:
head->current_path[numa_node_id()]

The structure is currently allocated as:
size = sizeof(struct nvme_ns_head) +
       (num_possible_nodes() * sizeof(struct nvme_ns *));
head = kzalloc(size, GFP_KERNEL);

This allocation assumes that NUMA node IDs are sequential and densely
packed from 0 .. num_possible_nodes() - 1. While this assumption holds
on many systems, it is not always true on some architectures such as
powerpc.

On some powerpc systems, NUMA node IDs can be sparse. For example:
NUMA:
  NUMA node(s):              6
  NUMA node0 CPU(s):         80-159
  NUMA node8 CPU(s):         0-79
  NUMA node252 CPU(s):
  NUMA node253 CPU(s):
  NUMA node254 CPU(s):
  NUMA node255 CPU(s):

That is, the possible/online NUMA node IDs are: 0, 8, 252, 253, 254, 255
In this case: num_possible_nodes() = 6

So memory is allocated for only 6 entries in current_path[]. However,
the array is later indexed using the actual NUMA node ID. As a result,
accesses such as:
head->current_path[8] or
head->current_path[252]
goes out of bounds, leading to the following KASAN splat:

==================================================================
BUG: KASAN: slab-out-of-bounds in nvme_mpath_revalidate_paths+0x22c/0x290 [nvme_core]
Write of size 8 at addr c00020003bda35b8 by task kworker/u641:2/1997

CPU: 1 UID: 0 PID: 1997 Comm: kworker/u641:2 Not tainted 7.1.0-rc5-dirty #14 PREEMPT(lazy)
Hardware name: 8335-GTH POWER9 0x4e1202 opal:skiboot-v6.5.3-35-g1851b2a06 PowerNV
Workqueue: async async_run_entry_fn
Call Trace:
[c000200037fa7510] [c0000000021c23d4] dump_stack_lvl+0x88/0xdc (unreliable)
[c000200037fa7540] [c0000000009fda90] print_report+0x22c/0x67c
[c000200037fa7630] [c0000000009fd508] kasan_report+0x108/0x220
[c000200037fa7740] [c0000000009fff48] __asan_store8+0xe8/0x120
[c000200037fa7760] [c008000018e76474] nvme_mpath_revalidate_paths+0x22c/0x290 [nvme_core]
[c000200037fa7800] [c008000018e6556c] nvme_update_ns_info+0x4a4/0x5e0 [nvme_core]
[c000200037fa7a50] [c008000018e66270] nvme_alloc_ns+0x6d8/0x1a70 [nvme_core]
[c000200037fa7c20] [c008000018e679fc] nvme_scan_ns+0x3f4/0x630 [nvme_core]
[c000200037fa7d10] [c00000000031f22c] async_run_entry_fn+0x9c/0x3a0
[c000200037fa7db0] [c0000000002fa544] process_one_work+0x414/0xa10
[c000200037fa7ec0] [c0000000002fbf00] worker_thread+0x320/0x640
[c000200037fa7f80] [c00000000030d0f8] kthread+0x278/0x290
[c000200037fa7fe0] [c00000000000ded8] start_kernel_thread+0x14/0x18

Allocated by task 1997 on cpu 1 at 35.928317s:

The buggy address belongs to the object at c00020003bda3000
 which belongs to the cache kmalloc-rnd-15-2k of size 2048
The buggy address is located 16 bytes to the right of
 allocated 1448-byte region [c00020003bda3000, c00020003bda35a8)

The buggy address belongs to the physical page:

Memory state around the buggy address:
 c00020003bda3480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 c00020003bda3500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>c00020003bda3580: 00 00 00 00 00 fc fc fc fc fc fc fc fc fc fc fc
                                        ^
 c00020003bda3600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
 c00020003bda3680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
==================================================================

Fix this by allocating the flexible array using nr_node_ids instead
of num_possible_nodes(). Since nr_node_ids represents the maximum
possible NUMA node IDs, indexing current_path[] using numa_node_id()
becomes safe even on systems with sparse node IDs.

Fixes: f333444708f8 ("nvme: take node locality into account when selecting a path")
Signed-off-by: Nilay Shroff <nilay@linux.ibm.com>
---
 drivers/nvme/host/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c3032d6ad6b1..96809227a0e2 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3926,7 +3926,7 @@ static struct nvme_ns_head *nvme_alloc_ns_head(struct nvme_ctrl *ctrl,
 	int ret = -ENOMEM;
 
 #ifdef CONFIG_NVME_MULTIPATH
-	size += num_possible_nodes() * sizeof(struct nvme_ns *);
+	size += nr_node_ids * sizeof(struct nvme_ns *);
 #endif
 
 	head = kzalloc(size, GFP_KERNEL);
-- 
2.53.0

Re: [PATCH] nvme-multipath: fix flex array size in struct nvme_ns_head

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH] nvme-multipath: fix flex array size in struct nvme_ns_head

[John Garry]

> 
> Fix this by allocating the flexible array using nr_node_ids instead
> of num_possible_nodes(). Since nr_node_ids represents the maximum
> possible NUMA node IDs, indexing current_path[] using numa_node_id()
> becomes safe even on systems with sparse node IDs.
> 
> Fixes: f333444708f8 ("nvme: take node locality into account when selecting a path")
> Signed-off-by: Nilay Shroff<nilay@linux.ibm.com>


Reviewed-by: John Garry <john.g.garry@oracle.com>

Re: [PATCH] nvme-multipath: fix flex array size in struct nvme_ns_head

[Mukesh Kumar Chaurasiya]

On Wed, May 27, 2026 at 11:50:00AM +0530, Nilay Shroff wrote:
> struct nvme_ns_head contains a flexible array member, current_path[],
> which is indexed using the NUMA node ID:
> head->current_path[numa_node_id()]
> 
> The structure is currently allocated as:
> size = sizeof(struct nvme_ns_head) +
>        (num_possible_nodes() * sizeof(struct nvme_ns *));
> head = kzalloc(size, GFP_KERNEL);
> 
> This allocation assumes that NUMA node IDs are sequential and densely
> packed from 0 .. num_possible_nodes() - 1. While this assumption holds
> on many systems, it is not always true on some architectures such as
> powerpc.
> 
> On some powerpc systems, NUMA node IDs can be sparse. For example:
> NUMA:
>   NUMA node(s):              6
>   NUMA node0 CPU(s):         80-159
>   NUMA node8 CPU(s):         0-79
>   NUMA node252 CPU(s):
>   NUMA node253 CPU(s):
>   NUMA node254 CPU(s):
>   NUMA node255 CPU(s):
> 
> That is, the possible/online NUMA node IDs are: 0, 8, 252, 253, 254, 255
> In this case: num_possible_nodes() = 6
> 
> So memory is allocated for only 6 entries in current_path[]. However,
> the array is later indexed using the actual NUMA node ID. As a result,
> accesses such as:
> head->current_path[8] or
> head->current_path[252]
> goes out of bounds, leading to the following KASAN splat:
> 
> ==================================================================
> BUG: KASAN: slab-out-of-bounds in nvme_mpath_revalidate_paths+0x22c/0x290 [nvme_core]
> Write of size 8 at addr c00020003bda35b8 by task kworker/u641:2/1997
> 
> CPU: 1 UID: 0 PID: 1997 Comm: kworker/u641:2 Not tainted 7.1.0-rc5-dirty #14 PREEMPT(lazy)
> Hardware name: 8335-GTH POWER9 0x4e1202 opal:skiboot-v6.5.3-35-g1851b2a06 PowerNV
> Workqueue: async async_run_entry_fn
> Call Trace:
> [c000200037fa7510] [c0000000021c23d4] dump_stack_lvl+0x88/0xdc (unreliable)
> [c000200037fa7540] [c0000000009fda90] print_report+0x22c/0x67c
> [c000200037fa7630] [c0000000009fd508] kasan_report+0x108/0x220
> [c000200037fa7740] [c0000000009fff48] __asan_store8+0xe8/0x120
> [c000200037fa7760] [c008000018e76474] nvme_mpath_revalidate_paths+0x22c/0x290 [nvme_core]
> [c000200037fa7800] [c008000018e6556c] nvme_update_ns_info+0x4a4/0x5e0 [nvme_core]
> [c000200037fa7a50] [c008000018e66270] nvme_alloc_ns+0x6d8/0x1a70 [nvme_core]
> [c000200037fa7c20] [c008000018e679fc] nvme_scan_ns+0x3f4/0x630 [nvme_core]
> [c000200037fa7d10] [c00000000031f22c] async_run_entry_fn+0x9c/0x3a0
> [c000200037fa7db0] [c0000000002fa544] process_one_work+0x414/0xa10
> [c000200037fa7ec0] [c0000000002fbf00] worker_thread+0x320/0x640
> [c000200037fa7f80] [c00000000030d0f8] kthread+0x278/0x290
> [c000200037fa7fe0] [c00000000000ded8] start_kernel_thread+0x14/0x18
> 
> Allocated by task 1997 on cpu 1 at 35.928317s:
> 
> The buggy address belongs to the object at c00020003bda3000
>  which belongs to the cache kmalloc-rnd-15-2k of size 2048
> The buggy address is located 16 bytes to the right of
>  allocated 1448-byte region [c00020003bda3000, c00020003bda35a8)
> 
> The buggy address belongs to the physical page:
> 
> Memory state around the buggy address:
>  c00020003bda3480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>  c00020003bda3500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >c00020003bda3580: 00 00 00 00 00 fc fc fc fc fc fc fc fc fc fc fc
>                                         ^
>  c00020003bda3600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
>  c00020003bda3680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
> ==================================================================
> 
> Fix this by allocating the flexible array using nr_node_ids instead
> of num_possible_nodes(). Since nr_node_ids represents the maximum
> possible NUMA node IDs, indexing current_path[] using numa_node_id()
> becomes safe even on systems with sparse node IDs.
> 
> Fixes: f333444708f8 ("nvme: take node locality into account when selecting a path")
> Signed-off-by: Nilay Shroff <nilay@linux.ibm.com>
> ---
>  drivers/nvme/host/core.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index c3032d6ad6b1..96809227a0e2 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -3926,7 +3926,7 @@ static struct nvme_ns_head *nvme_alloc_ns_head(struct nvme_ctrl *ctrl,
>  	int ret = -ENOMEM;
>  
>  #ifdef CONFIG_NVME_MULTIPATH
> -	size += num_possible_nodes() * sizeof(struct nvme_ns *);
> +	size += nr_node_ids * sizeof(struct nvme_ns *);
>  #endif
>  
>  	head = kzalloc(size, GFP_KERNEL);
> -- 
> 2.53.0
> 
> 
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Tested-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>

Re: [PATCH] nvme-multipath: fix flex array size in struct nvme_ns_head

[Hannes Reinecke]

On 5/27/26 08:20, Nilay Shroff wrote:
> struct nvme_ns_head contains a flexible array member, current_path[],
> which is indexed using the NUMA node ID:
> head->current_path[numa_node_id()]
> 
> The structure is currently allocated as:
> size = sizeof(struct nvme_ns_head) +
>         (num_possible_nodes() * sizeof(struct nvme_ns *));
> head = kzalloc(size, GFP_KERNEL);
> 
> This allocation assumes that NUMA node IDs are sequential and densely
> packed from 0 .. num_possible_nodes() - 1. While this assumption holds
> on many systems, it is not always true on some architectures such as
> powerpc.
> 
> On some powerpc systems, NUMA node IDs can be sparse. For example:
> NUMA:
>    NUMA node(s):              6
>    NUMA node0 CPU(s):         80-159
>    NUMA node8 CPU(s):         0-79
>    NUMA node252 CPU(s):
>    NUMA node253 CPU(s):
>    NUMA node254 CPU(s):
>    NUMA node255 CPU(s):
> 
> That is, the possible/online NUMA node IDs are: 0, 8, 252, 253, 254, 255
> In this case: num_possible_nodes() = 6
> 
> So memory is allocated for only 6 entries in current_path[]. However,
> the array is later indexed using the actual NUMA node ID. As a result,
> accesses such as:
> head->current_path[8] or
> head->current_path[252]
> goes out of bounds, leading to the following KASAN splat:
> 
> ==================================================================
> BUG: KASAN: slab-out-of-bounds in nvme_mpath_revalidate_paths+0x22c/0x290 [nvme_core]
> Write of size 8 at addr c00020003bda35b8 by task kworker/u641:2/1997
> 
> CPU: 1 UID: 0 PID: 1997 Comm: kworker/u641:2 Not tainted 7.1.0-rc5-dirty #14 PREEMPT(lazy)
> Hardware name: 8335-GTH POWER9 0x4e1202 opal:skiboot-v6.5.3-35-g1851b2a06 PowerNV
> Workqueue: async async_run_entry_fn
> Call Trace:
> [c000200037fa7510] [c0000000021c23d4] dump_stack_lvl+0x88/0xdc (unreliable)
> [c000200037fa7540] [c0000000009fda90] print_report+0x22c/0x67c
> [c000200037fa7630] [c0000000009fd508] kasan_report+0x108/0x220
> [c000200037fa7740] [c0000000009fff48] __asan_store8+0xe8/0x120
> [c000200037fa7760] [c008000018e76474] nvme_mpath_revalidate_paths+0x22c/0x290 [nvme_core]
> [c000200037fa7800] [c008000018e6556c] nvme_update_ns_info+0x4a4/0x5e0 [nvme_core]
> [c000200037fa7a50] [c008000018e66270] nvme_alloc_ns+0x6d8/0x1a70 [nvme_core]
> [c000200037fa7c20] [c008000018e679fc] nvme_scan_ns+0x3f4/0x630 [nvme_core]
> [c000200037fa7d10] [c00000000031f22c] async_run_entry_fn+0x9c/0x3a0
> [c000200037fa7db0] [c0000000002fa544] process_one_work+0x414/0xa10
> [c000200037fa7ec0] [c0000000002fbf00] worker_thread+0x320/0x640
> [c000200037fa7f80] [c00000000030d0f8] kthread+0x278/0x290
> [c000200037fa7fe0] [c00000000000ded8] start_kernel_thread+0x14/0x18
> 
> Allocated by task 1997 on cpu 1 at 35.928317s:
> 
> The buggy address belongs to the object at c00020003bda3000
>   which belongs to the cache kmalloc-rnd-15-2k of size 2048
> The buggy address is located 16 bytes to the right of
>   allocated 1448-byte region [c00020003bda3000, c00020003bda35a8)
> 
> The buggy address belongs to the physical page:
> 
> Memory state around the buggy address:
>   c00020003bda3480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   c00020003bda3500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> c00020003bda3580: 00 00 00 00 00 fc fc fc fc fc fc fc fc fc fc fc
>                                          ^
>   c00020003bda3600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
>   c00020003bda3680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
> ==================================================================
> 
> Fix this by allocating the flexible array using nr_node_ids instead
> of num_possible_nodes(). Since nr_node_ids represents the maximum
> possible NUMA node IDs, indexing current_path[] using numa_node_id()
> becomes safe even on systems with sparse node IDs.
> 
> Fixes: f333444708f8 ("nvme: take node locality into account when selecting a path")
> Signed-off-by: Nilay Shroff <nilay@linux.ibm.com>
> ---
>   drivers/nvme/host/core.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index c3032d6ad6b1..96809227a0e2 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -3926,7 +3926,7 @@ static struct nvme_ns_head *nvme_alloc_ns_head(struct nvme_ctrl *ctrl,
>   	int ret = -ENOMEM;
>   
>   #ifdef CONFIG_NVME_MULTIPATH
> -	size += num_possible_nodes() * sizeof(struct nvme_ns *);
> +	size += nr_node_ids * sizeof(struct nvme_ns *);
>   #endif
>   
>   	head = kzalloc(size, GFP_KERNEL);

Reviewed-by: Hannes Reinecke <hare@kernel.org>

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                  Kernel Storage Architect
hare@suse.de                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich

Re: [PATCH] nvme-multipath: fix flex array size in struct nvme_ns_head

[Keith Busch]

On Wed, May 27, 2026 at 11:50:00AM +0530, Nilay Shroff wrote:
> Fix this by allocating the flexible array using nr_node_ids instead
> of num_possible_nodes(). Since nr_node_ids represents the maximum
> possible NUMA node IDs, indexing current_path[] using numa_node_id()
> becomes safe even on systems with sparse node IDs.

What is the largest nr_node_ids we can reasonably expect to see? I don't
want to end up trying to allocate a billion entry sparse array just to
index a tiny fraction of them. Not that we didn't have a similar problem
before, but this patch makes it clear we haven't been checking.

Re: [PATCH] nvme-multipath: fix flex array size in struct nvme_ns_head

[Nilay Shroff]

On 5/27/26 7:19 PM, Keith Busch wrote:
> On Wed, May 27, 2026 at 11:50:00AM +0530, Nilay Shroff wrote:
>> Fix this by allocating the flexible array using nr_node_ids instead
>> of num_possible_nodes(). Since nr_node_ids represents the maximum
>> possible NUMA node IDs, indexing current_path[] using numa_node_id()
>> becomes safe even on systems with sparse node IDs.
> 
> What is the largest nr_node_ids we can reasonably expect to see? I don't
> want to end up trying to allocate a billion entry sparse array just to
> index a tiny fraction of them. Not that we didn't have a similar problem
> before, but this patch makes it clear we haven't been checking.

The nr_node_ids value is bounded by MAX_NUMNODES, which is a compile-time
constant derived from CONFIG_NODES_SHIFT:

#define MAX_NUMNODES    (1 << CONFIG_NODES_SHIFT)

nr_node_ids itself cannot exceed MAX_NUMNODES.

For example:

x86 (arch/x86/Kconfig):
config NODES_SHIFT
         int "Maximum NUMA Nodes (as a power of 2)" if !MAXSMP
         range 1 10
         default "10" if MAXSMP
         default "6" if X86_64
         default "3"
         depends on NUMA

arm64 (arch/arm64/Kconfig):
config NODES_SHIFT
         int "Maximum NUMA Nodes (as a power of 2)"
         range 1 10
         default "4"
         depends on NUMA

powerpc (arch/powerpc/Kconfig):
config NODES_SHIFT
         int
         default "8" if PPC64
         default "4"
         depends on NUMA

Looking across the architectures supported by Linux, CONFIG_NODES_SHIFT
appears to be capped at 10, which gives:

MAX_NUMNODES = 1 << 10 = 1024

So in practice, nr_node_ids should not exceed 1024.

This means the flexible array allocation remains bounded and should not grow
into an unreasonably large sparse allocation.

Thanks,
--Nilay

Re: [PATCH] nvme-multipath: fix flex array size in struct nvme_ns_head

[Keith Busch]

On Wed, May 27, 2026 at 08:36:43PM +0530, Nilay Shroff wrote:
> Looking across the architectures supported by Linux, CONFIG_NODES_SHIFT
> appears to be capped at 10, which gives:
> 
> MAX_NUMNODES = 1 << 10 = 1024
> 
> So in practice, nr_node_ids should not exceed 1024.
> 
> This means the flexible array allocation remains bounded and should not grow
> into an unreasonably large sparse allocation.

Cool, thanks for checking. Will get this applied to nvme-7.2.