Re: [PATCH 05/17] arch, mm: pull out allocation of NODE_DATA to generic code

David Hildenbrand
2024-07-19T15:47:05Z

On 19.07.24 17:34, Mike Rapoport wrote:
> On Fri, Jul 19, 2024 at 05:07:35PM +0200, David Hildenbrand wrote:
>>>>> - * Allocate node data. Try node-local memory and then any node.
>>>>> - * Never allocate in DMA zone.
>>>>> - */
>>>>> - nd_pa = memblock_phys_alloc_try_nid(nd_size, SMP_CACHE_BYTES, nid);
>>>>> - if (!nd_pa) {
>>>>> - pr_err("Cannot find %zu bytes in any node (initial node: %d)\n",
>>>>> - nd_size, nid);
>>>>> - return;
>>>>> - }
>>>>> - nd = __va(nd_pa);
>>>>> -
>>>>> - /* report and initialize */
>>>>> - printk(KERN_INFO "NODE_DATA(%d) allocated [mem %#010Lx-%#010Lx]\n", nid,
>>>>> - nd_pa, nd_pa + nd_size - 1);
>>>>> - tnid = early_pfn_to_nid(nd_pa >> PAGE_SHIFT);
>>>>> - if (tnid != nid)
>>>>> - printk(KERN_INFO " NODE_DATA(%d) on node %d\n", nid, tnid);
>>>>> -
>>>>> - node_data[nid] = nd;
>>>>> - memset(NODE_DATA(nid), 0, sizeof(pg_data_t));
>>>>> -
>>>>> - node_set_online(nid);
>>>>> -}
>>>>> -
>>>>> /**
>>>>> * numa_cleanup_meminfo - Cleanup a numa_meminfo
>>>>> * @mi: numa_meminfo to clean up
>>>>> @@ -571,6 +538,7 @@ static int __init numa_register_memblks(struct numa_meminfo *mi)
>>>>> continue;
>>>>> alloc_node_data(nid);
>>>>> + node_set_online(nid);
>>>>> }
>>>>
>>>> I can spot that we only remove a single node_set_online() call from x86.
>>>>
>>>> What about all the other architectures? Will there be any change in behavior
>>>> for them? Or do we simply set the nodes online later once more?
>>>
>>> On x86 node_set_online() was a part of alloc_node_data() and I moved it
>>> outside so it's called right after alloc_node_data(). On other
>>> architectures the allocation didn't include that call, so there should be
>>> no difference there.
>>
>> But won't their arch code try setting the nodes online at a later stage?
>>
>> And I think, some architectures only set nodes online conditionally
>> (see most other node_set_online() calls).
>>
>> Sorry if I'm confused here, but with now unconditional node_set_online(), won't
>> we change the behavior of other architectures?
>
> The generic alloc_node_data() does not set the node online:
>
> +/* Allocate NODE_DATA for a node on the local memory */
> +void __init alloc_node_data(int nid)
> +{
> + const size_t nd_size = roundup(sizeof(pg_data_t), PAGE_SIZE);
> + u64 nd_pa;
> + void *nd;
> + int tnid;
> +
> + /* Allocate node data. Try node-local memory and then any node. */
> + nd_pa = memblock_phys_alloc_try_nid(nd_size, SMP_CACHE_BYTES, nid);
> + if (!nd_pa)
> + panic("Cannot allocate %zu bytes for node %d data\n",
> + nd_size, nid);
> + nd = __va(nd_pa);
> +
> + /* report and initialize */
> + pr_info("NODE_DATA(%d) allocated [mem %#010Lx-%#010Lx]\n", nid,
> + nd_pa, nd_pa + nd_size - 1);
> + tnid = early_pfn_to_nid(nd_pa >> PAGE_SHIFT);
> + if (tnid != nid)
> + pr_info(" NODE_DATA(%d) on node %d\n", nid, tnid);
> +
> + node_data[nid] = nd;
> + memset(NODE_DATA(nid), 0, sizeof(pg_data_t));
> +}
>
> I might have missed some architecture except x86 that calls
> node_set_online() in its alloc_node_data(), but the intention was to leave
> that call outside the alloc and explicitly add it after the call to
> alloc_node_data() if needed like in x86.

I'm stupid, I didn't realize it is still only called from x86 :(

Acked-by: David Hildenbrand <david@redhat.com>

--
Cheers,

David / dhildenb