HUGETLB pages on multiple nodes

HUGETLB pages on multiple nodes

[Nick Pollitt]

Seth,  It looks like at boot time all the hugepages will get allocated 
on the boot node.  We'd like to spread the pages across multiple nodes 
on NUMA systems.  Here's a simple patch to round-robin the pages during 
kernel init.  Comments?

Nick

=== arch/ia64/mm/hugetlbpage.c 1.2 vs edited ==--- 1.2/arch/ia64/mm/hugetlbpage.c	Mon May 19 09:18:44 2003
+++ edited/arch/ia64/mm/hugetlbpage.c	Thu Jul 10 11:35:05 2003
@@ -419,7 +419,12 @@
 	struct page *page;
 
 	for (i = 0; i < htlbpage_max; ++i) {
+#ifdef CONFIG_NUMA
+		int node = i % num_memblks;
+		page = alloc_pages_node(node, __GFP_HIGHMEM, HUGETLB_PAGE_ORDER);
+#else
 		page = alloc_pages(__GFP_HIGHMEM, HUGETLB_PAGE_ORDER);
+#endif /* CONFIG_NUMA */
 		if (!page)
 			break;
 		for (j = 0; j < HPAGE_SIZE/PAGE_SIZE; ++j)

Re: HUGETLB pages on multiple nodes

[William Lee Irwin III]

On Mon, Jul 14, 2003 at 12:20:37PM -0700, Nick Pollitt wrote:
> Seth,  It looks like at boot time all the hugepages will get allocated 
> on the boot node.  We'd like to spread the pages across multiple nodes 
> on NUMA systems.  Here's a simple patch to round-robin the pages during 
> kernel init.  Comments?

I shipped some stuff for arch/i386/ that does node-local allocations at
runtime, too.


-- wli

Re: HUGETLB pages on multiple nodes

[Christoph Hellwig]

On Mon, Jul 14, 2003 at 12:20:37PM -0700, Nick Pollitt wrote:
> +#ifdef CONFIG_NUMA
> +		int node = i % num_memblks;
> +		page = alloc_pages_node(node, __GFP_HIGHMEM, HUGETLB_PAGE_ORDER);
> +#else
>  		page = alloc_pages(__GFP_HIGHMEM, HUGETLB_PAGE_ORDER);
> +#endif /* CONFIG_NUMA */

Why the num_memblks?  We care for nodes, not memblks, don't we?

Also you can just use alloc_pages_node directly in the non-NUMA code.

See how alloc_pages is implemented in 2.5:

#define alloc_pages(gfp_mask, order) \
                alloc_pages_node(numa_node_id(), gfp_mask, order)