RE: free bootmem feedback patch

Luck, Tony
2004-07-15T23:53:33Z

[-- Attachment #1: Type: text/plain, Size: 1414 bytes --]

>> still be looking at a couple of minutes :-( Since I only got a
>> 55% reduction, rather than a factor of 64 I expect that modifying
>> to look an larger order pages may have diminishing returns.
>
>I'll work relative to this for the rest. I'd recommend using __ffs()
>instead of the loop. Also, combining this with a specialized page
>freeing function that doesn't e.g. fiddle with page references

The returns to freeing larger pages do indeed diminish fast. I
added simple "look at the next word" and "look at the next
three words" hacks to see what the times looked like with
order=7 and order=8 ... and found that order 8 is only 1.8%
faster than order 6.

>The common case is the bitmap and mem_map[] starting at 0.

Sadly not quite 0. PG_reserved is set for each page structure
and must be cleared ... so we have to touch every page structure
at least once :-( On a 4TB machine thats 0.25 billion cache
misses (with a 16K page).

>remaining cases are pretty marginalized. This can actually be checked
>at runtime by checking the alignment of ->node_boot_start, e.g. maybe
>if (!~v && !((__pa(bdata->node_boot_start) >> PAGE_SHIFT) &
>((1 << MAX_ORDER) - 1)))
>instead of just !~v.

That check can be done once (for each node) outside the loop. The
exact expression used to set the "gofast" variable in my patch
make need some tweaking

New patch attached.

-Tony

[-- Attachment #2: bootmem.patch --]
[-- Type: application/octet-stream, Size: 1103 bytes --]

--- linux-2.6.8-rc1-orig/mm/bootmem.c 2004-07-11 10:34:19.000000000 -0700
+++ linux-2.6.8-rc1/mm/bootmem.c 2004-07-15 16:45:04.255195717 -0700
@@ -259,6 +259,7 @@
unsigned long i, count, total = 0;
unsigned long idx;
unsigned long *map;
+ int gofast = 0;

BUG_ON(!bdata->node_bootmem_map);

@@ -267,9 +268,23 @@
page = virt_to_page(phys_to_virt(bdata->node_boot_start));
idx = bdata->node_low_pfn - (bdata->node_boot_start >> PAGE_SHIFT);
map = bdata->node_bootmem_map;
+ if (bdata->node_boot_start == 0 ||
+ ffs(bdata->node_boot_start) - PAGE_SHIFT > ffs(BITS_PER_LONG))
+ gofast = 1;
for (i = 0; i < idx; ) {
unsigned long v = ~map[i / BITS_PER_LONG];
- if (v) {
+ if (gofast && v == ~0UL) {
+ int j;
+
+ count += BITS_PER_LONG;
+ ClearPageReserved(page);
+ set_page_count(page, 1);
+ for (j = 1; j < BITS_PER_LONG; j++)
+ ClearPageReserved(page + j);
+ __free_pages(page, ffs(BITS_PER_LONG)-1);
+ i += BITS_PER_LONG;
+ page += BITS_PER_LONG;
+ } else if (v) {
unsigned long m;
for (m = 1; m && i < idx; m<<=1, page++, i++) {
if (v & m) {