[PATCH] ring_buffer: NUMA aware page allocations

[PATCH] ring_buffer: NUMA aware page allocations

[Eric Dumazet]

Hi Robert & Steven

While browsing oprofile code in current tree, I discovered 
drivers/oprofile/cpu_buffer.c was using new ring_buffer code.

Apparently we lost in the conversion NUMA allocations, unless
I am mistaken, since rb_allocate_pages() uses plain
__get_free_page(GFP_KERNEL) calls to allocate pages.

All "buffer_page" structs are allocated with kzalloc_node(),
but not the pages themselves.

Thank you

[PATCH] ring_buffer: NUMA aware page allocations

rb_allocate_pages() & ring_buffer_resize() should use alloc_pages_node()
instead of __get_free_page() to allocate pages.

Signed-off-by: Eric Dumazet <dada1@cosmosbay.com>

diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 8b0daf0..feb8482 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -333,21 +333,22 @@ static int rb_allocate_pages(struct ring_buffer_per_cpu *cpu_buffer,
 {
 	struct list_head *head = &cpu_buffer->pages;
 	struct buffer_page *bpage, *tmp;
-	unsigned long addr;
+	struct page *page;
 	LIST_HEAD(pages);
 	unsigned i;
+	int node = cpu_to_node(cpu_buffer->cpu);
 
 	for (i = 0; i < nr_pages; i++) {
 		bpage = kzalloc_node(ALIGN(sizeof(*bpage), cache_line_size()),
-				    GFP_KERNEL, cpu_to_node(cpu_buffer->cpu));
+				    GFP_KERNEL, node);
 		if (!bpage)
 			goto free_pages;
 		list_add(&bpage->list, &pages);
 
-		addr = __get_free_page(GFP_KERNEL);
-		if (!addr)
+		page = alloc_pages_node(node, GFP_KERNEL, 0);
+		if (!page)
 			goto free_pages;
-		bpage->page = (void *)addr;
+		bpage->page = page_address(page);
 		rb_init_page(bpage->page);
 	}
 
@@ -605,7 +606,7 @@ int ring_buffer_resize(struct ring_buffer *buffer, unsigned long size)
 	unsigned nr_pages, rm_pages, new_pages;
 	struct buffer_page *bpage, *tmp;
 	unsigned long buffer_size;
-	unsigned long addr;
+	struct page *page;
 	LIST_HEAD(pages);
 	int i, cpu;
 
@@ -663,17 +664,19 @@ int ring_buffer_resize(struct ring_buffer *buffer, unsigned long size)
 	new_pages = nr_pages - buffer->pages;
 
 	for_each_buffer_cpu(buffer, cpu) {
+		int node = cpu_to_node(cpu);
+
 		for (i = 0; i < new_pages; i++) {
 			bpage = kzalloc_node(ALIGN(sizeof(*bpage),
 						  cache_line_size()),
-					    GFP_KERNEL, cpu_to_node(cpu));
+					    GFP_KERNEL, node);
 			if (!bpage)
 				goto free_pages;
 			list_add(&bpage->list, &pages);
-			addr = __get_free_page(GFP_KERNEL);
-			if (!addr)
+			page = alloc_pages_node(node, GFP_KERNEL, 0);
+			if (!page)
 				goto free_pages;
-			bpage->page = (void *)addr;
+			bpage->page = page_address(page);
 			rb_init_page(bpage->page);
 		}
 	}

Re: [PATCH] ring_buffer: NUMA aware page allocations

[Ingo Molnar]

* Eric Dumazet <dada1@cosmosbay.com> wrote:

> Hi Robert & Steven
> 
> While browsing oprofile code in current tree, I discovered 
> drivers/oprofile/cpu_buffer.c was using new ring_buffer code.
> 
> Apparently we lost in the conversion NUMA allocations, unless
> I am mistaken, since rb_allocate_pages() uses plain
> __get_free_page(GFP_KERNEL) calls to allocate pages.
> 
> All "buffer_page" structs are allocated with kzalloc_node(),
> but not the pages themselves.
> 
> Thank you
> 
> [PATCH] ring_buffer: NUMA aware page allocations
> 
> rb_allocate_pages() & ring_buffer_resize() should use alloc_pages_node()
> instead of __get_free_page() to allocate pages.

nice improvement!

	Ingo