* [PATCH/WIP] lmb cleanups and additions @ 2010-05-10 9:38 Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 01/25] lmb: Rename lmb_region to lmb_type and lmb_property to lmb_region Benjamin Herrenschmidt 0 siblings, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal This is my current WIP series. It's compile tested on ppc and sparc64 and quickly boot tested on ppc64. The "advanced" features such as the new array resize are completely untested in this version. My aim is still to replace the bottom part of Yinghai's patch series rather than build on top of it, and from there, add whatever he needs to successfully port x86 over and turn NO_BOOTMEM into something half decent without adding a ton of unneeded crap to the core lmb. This is not finished tho. Here's a peek at my TODO list: - Move to mm/lmb.c - Various random return types with non-useful return codes (lmb_add, lmb_remove, ...) needs cleaning & documenting - Add docbook for all API functions - lmb_add or lmb_reserve of overlapping regions are going to wreck things in very interesting ways. We could easily error out but that's sub-optimal, instead we should break them up to only add/reserve the bits that aren't yet - Add some pr_debug in various places in there - Improve the NUMA interaction In the meantime, comments are welcome :-) Cheers, Ben. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH 01/25] lmb: Rename lmb_region to lmb_type and lmb_property to lmb_region 2010-05-10 9:38 [PATCH/WIP] lmb cleanups and additions Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 02/25] lmb: No reason to include asm/lmb.h late Benjamin Herrenschmidt 0 siblings, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- arch/microblaze/mm/init.c | 4 +- arch/powerpc/mm/hash_utils_64.c | 2 +- arch/powerpc/mm/mem.c | 26 +++--- arch/powerpc/platforms/embedded6xx/wii.c | 2 +- arch/sparc/mm/init_64.c | 6 +- include/linux/lmb.h | 24 ++-- lib/lmb.c | 168 +++++++++++++++--------------- 7 files changed, 115 insertions(+), 117 deletions(-) diff --git a/arch/microblaze/mm/init.c b/arch/microblaze/mm/init.c index f42c2dd..9d58797 100644 --- a/arch/microblaze/mm/init.c +++ b/arch/microblaze/mm/init.c @@ -76,8 +76,8 @@ void __init setup_memory(void) /* Find main memory where is the kernel */ for (i = 0; i < lmb.memory.cnt; i++) { - memory_start = (u32) lmb.memory.region[i].base; - memory_end = (u32) lmb.memory.region[i].base + memory_start = (u32) lmb.memory.regions[i].base; + memory_end = (u32) lmb.memory.regions[i].base + (u32) lmb.memory.region[i].size; if ((memory_start <= (u32)_text) && ((u32)_text <= memory_end)) { diff --git a/arch/powerpc/mm/hash_utils_64.c b/arch/powerpc/mm/hash_utils_64.c index 3ecdcec..0a232f5 100644 --- a/arch/powerpc/mm/hash_utils_64.c +++ b/arch/powerpc/mm/hash_utils_64.c @@ -660,7 +660,7 @@ static void __init htab_initialize(void) /* create bolted the linear mapping in the hash table */ for (i=0; i < lmb.memory.cnt; i++) { - base = (unsigned long)__va(lmb.memory.region[i].base); + base = (unsigned long)__va(lmb.memory.regions[i].base); size = lmb.memory.region[i].size; DBG("creating mapping for region: %lx..%lx (prot: %lx)\n", diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c index 0f594d7..65acb49 100644 --- a/arch/powerpc/mm/mem.c +++ b/arch/powerpc/mm/mem.c @@ -86,10 +86,10 @@ int page_is_ram(unsigned long pfn) for (i=0; i < lmb.memory.cnt; i++) { unsigned long base; - base = lmb.memory.region[i].base; + base = lmb.memory.regions[i].base; if ((paddr >= base) && - (paddr < (base + lmb.memory.region[i].size))) { + (paddr < (base + lmb.memory.regions[i].size))) { return 1; } } @@ -149,7 +149,7 @@ int walk_system_ram_range(unsigned long start_pfn, unsigned long nr_pages, void *arg, int (*func)(unsigned long, unsigned long, void *)) { - struct lmb_property res; + struct lmb_region res; unsigned long pfn, len; u64 end; int ret = -1; @@ -206,7 +206,7 @@ void __init do_init_bootmem(void) /* Add active regions with valid PFNs */ for (i = 0; i < lmb.memory.cnt; i++) { unsigned long start_pfn, end_pfn; - start_pfn = lmb.memory.region[i].base >> PAGE_SHIFT; + start_pfn = lmb.memory.regions[i].base >> PAGE_SHIFT; end_pfn = start_pfn + lmb_size_pages(&lmb.memory, i); add_active_range(0, start_pfn, end_pfn); } @@ -219,16 +219,16 @@ void __init do_init_bootmem(void) /* reserve the sections we're already using */ for (i = 0; i < lmb.reserved.cnt; i++) { - unsigned long addr = lmb.reserved.region[i].base + + unsigned long addr = lmb.reserved.regions[i].base + lmb_size_bytes(&lmb.reserved, i) - 1; if (addr < lowmem_end_addr) - reserve_bootmem(lmb.reserved.region[i].base, + reserve_bootmem(lmb.reserved.regions[i].base, lmb_size_bytes(&lmb.reserved, i), BOOTMEM_DEFAULT); - else if (lmb.reserved.region[i].base < lowmem_end_addr) { + else if (lmb.reserved.regions[i].base < lowmem_end_addr) { unsigned long adjusted_size = lowmem_end_addr - - lmb.reserved.region[i].base; - reserve_bootmem(lmb.reserved.region[i].base, + lmb.reserved.regions[i].base; + reserve_bootmem(lmb.reserved.regions[i].base, adjusted_size, BOOTMEM_DEFAULT); } } @@ -237,7 +237,7 @@ void __init do_init_bootmem(void) /* reserve the sections we're already using */ for (i = 0; i < lmb.reserved.cnt; i++) - reserve_bootmem(lmb.reserved.region[i].base, + reserve_bootmem(lmb.reserved.regions[i].base, lmb_size_bytes(&lmb.reserved, i), BOOTMEM_DEFAULT); @@ -257,10 +257,10 @@ static int __init mark_nonram_nosave(void) for (i = 0; i < lmb.memory.cnt - 1; i++) { lmb_region_max_pfn = - (lmb.memory.region[i].base >> PAGE_SHIFT) + - (lmb.memory.region[i].size >> PAGE_SHIFT); + (lmb.memory.regions[i].base >> PAGE_SHIFT) + + (lmb.memory.regions[i].size >> PAGE_SHIFT); lmb_next_region_start_pfn = - lmb.memory.region[i+1].base >> PAGE_SHIFT; + lmb.memory.regions[i+1].base >> PAGE_SHIFT; if (lmb_region_max_pfn < lmb_next_region_start_pfn) register_nosave_region(lmb_region_max_pfn, diff --git a/arch/powerpc/platforms/embedded6xx/wii.c b/arch/powerpc/platforms/embedded6xx/wii.c index 57e5b60..42f346c 100644 --- a/arch/powerpc/platforms/embedded6xx/wii.c +++ b/arch/powerpc/platforms/embedded6xx/wii.c @@ -65,7 +65,7 @@ static int __init page_aligned(unsigned long x) void __init wii_memory_fixups(void) { - struct lmb_property *p = lmb.memory.region; + struct lmb_region *p = lmb.memory.region; /* * This is part of a workaround to allow the use of two diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c index b2831dc..33628b4 100644 --- a/arch/sparc/mm/init_64.c +++ b/arch/sparc/mm/init_64.c @@ -978,7 +978,7 @@ static void __init add_node_ranges(void) unsigned long size = lmb_size_bytes(&lmb.memory, i); unsigned long start, end; - start = lmb.memory.region[i].base; + start = lmb.memory.regions[i].base; end = start + size; while (start < end) { unsigned long this_end; @@ -1299,7 +1299,7 @@ static void __init bootmem_init_nonnuma(void) if (!size) continue; - start_pfn = lmb.memory.region[i].base >> PAGE_SHIFT; + start_pfn = lmb.memory.regions[i].base >> PAGE_SHIFT; end_pfn = start_pfn + lmb_size_pages(&lmb.memory, i); add_active_range(0, start_pfn, end_pfn); } @@ -1339,7 +1339,7 @@ static void __init trim_reserved_in_node(int nid) numadbg(" trim_reserved_in_node(%d)\n", nid); for (i = 0; i < lmb.reserved.cnt; i++) { - unsigned long start = lmb.reserved.region[i].base; + unsigned long start = lmb.reserved.regions[i].base; unsigned long size = lmb_size_bytes(&lmb.reserved, i); unsigned long end = start + size; diff --git a/include/linux/lmb.h b/include/linux/lmb.h index f3d1433..d225d78 100644 --- a/include/linux/lmb.h +++ b/include/linux/lmb.h @@ -18,22 +18,22 @@ #define MAX_LMB_REGIONS 128 -struct lmb_property { +struct lmb_region { u64 base; u64 size; }; -struct lmb_region { +struct lmb_type { unsigned long cnt; u64 size; - struct lmb_property region[MAX_LMB_REGIONS+1]; + struct lmb_region regions[MAX_LMB_REGIONS+1]; }; struct lmb { unsigned long debug; u64 rmo_size; - struct lmb_region memory; - struct lmb_region reserved; + struct lmb_type memory; + struct lmb_type reserved; }; extern struct lmb lmb; @@ -56,27 +56,27 @@ extern u64 lmb_end_of_DRAM(void); extern void __init lmb_enforce_memory_limit(u64 memory_limit); extern int __init lmb_is_reserved(u64 addr); extern int lmb_is_region_reserved(u64 base, u64 size); -extern int lmb_find(struct lmb_property *res); +extern int lmb_find(struct lmb_region *res); extern void lmb_dump_all(void); static inline u64 -lmb_size_bytes(struct lmb_region *type, unsigned long region_nr) +lmb_size_bytes(struct lmb_type *type, unsigned long region_nr) { - return type->region[region_nr].size; + return type->regions[region_nr].size; } static inline u64 -lmb_size_pages(struct lmb_region *type, unsigned long region_nr) +lmb_size_pages(struct lmb_type *type, unsigned long region_nr) { return lmb_size_bytes(type, region_nr) >> PAGE_SHIFT; } static inline u64 -lmb_start_pfn(struct lmb_region *type, unsigned long region_nr) +lmb_start_pfn(struct lmb_type *type, unsigned long region_nr) { - return type->region[region_nr].base >> PAGE_SHIFT; + return type->regions[region_nr].base >> PAGE_SHIFT; } static inline u64 -lmb_end_pfn(struct lmb_region *type, unsigned long region_nr) +lmb_end_pfn(struct lmb_type *type, unsigned long region_nr) { return lmb_start_pfn(type, region_nr) + lmb_size_pages(type, region_nr); diff --git a/lib/lmb.c b/lib/lmb.c index b1fc526..f07337e 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -29,7 +29,7 @@ static int __init early_lmb(char *p) } early_param("lmb", early_lmb); -static void lmb_dump(struct lmb_region *region, char *name) +static void lmb_dump(struct lmb_type *region, char *name) { unsigned long long base, size; int i; @@ -37,8 +37,8 @@ static void lmb_dump(struct lmb_region *region, char *name) pr_info(" %s.cnt = 0x%lx\n", name, region->cnt); for (i = 0; i < region->cnt; i++) { - base = region->region[i].base; - size = region->region[i].size; + base = region->regions[i].base; + size = region->regions[i].size; pr_info(" %s[0x%x]\t0x%016llx - 0x%016llx, 0x%llx bytes\n", name, i, base, base + size - 1, size); @@ -74,34 +74,34 @@ static long lmb_addrs_adjacent(u64 base1, u64 size1, u64 base2, u64 size2) return 0; } -static long lmb_regions_adjacent(struct lmb_region *rgn, +static long lmb_regions_adjacent(struct lmb_type *type, unsigned long r1, unsigned long r2) { - u64 base1 = rgn->region[r1].base; - u64 size1 = rgn->region[r1].size; - u64 base2 = rgn->region[r2].base; - u64 size2 = rgn->region[r2].size; + u64 base1 = type->regions[r1].base; + u64 size1 = type->regions[r1].size; + u64 base2 = type->regions[r2].base; + u64 size2 = type->regions[r2].size; return lmb_addrs_adjacent(base1, size1, base2, size2); } -static void lmb_remove_region(struct lmb_region *rgn, unsigned long r) +static void lmb_remove_region(struct lmb_type *type, unsigned long r) { unsigned long i; - for (i = r; i < rgn->cnt - 1; i++) { - rgn->region[i].base = rgn->region[i + 1].base; - rgn->region[i].size = rgn->region[i + 1].size; + for (i = r; i < type->cnt - 1; i++) { + type->regions[i].base = type->regions[i + 1].base; + type->regions[i].size = type->regions[i + 1].size; } - rgn->cnt--; + type->cnt--; } /* Assumption: base addr of region 1 < base addr of region 2 */ -static void lmb_coalesce_regions(struct lmb_region *rgn, +static void lmb_coalesce_regions(struct lmb_type *type, unsigned long r1, unsigned long r2) { - rgn->region[r1].size += rgn->region[r2].size; - lmb_remove_region(rgn, r2); + type->regions[r1].size += type->regions[r2].size; + lmb_remove_region(type, r2); } void __init lmb_init(void) @@ -109,13 +109,13 @@ void __init lmb_init(void) /* Create a dummy zero size LMB which will get coalesced away later. * This simplifies the lmb_add() code below... */ - lmb.memory.region[0].base = 0; - lmb.memory.region[0].size = 0; + lmb.memory.regions[0].base = 0; + lmb.memory.regions[0].size = 0; lmb.memory.cnt = 1; /* Ditto. */ - lmb.reserved.region[0].base = 0; - lmb.reserved.region[0].size = 0; + lmb.reserved.regions[0].base = 0; + lmb.reserved.regions[0].size = 0; lmb.reserved.cnt = 1; } @@ -126,24 +126,24 @@ void __init lmb_analyze(void) lmb.memory.size = 0; for (i = 0; i < lmb.memory.cnt; i++) - lmb.memory.size += lmb.memory.region[i].size; + lmb.memory.size += lmb.memory.regions[i].size; } -static long lmb_add_region(struct lmb_region *rgn, u64 base, u64 size) +static long lmb_add_region(struct lmb_type *type, u64 base, u64 size) { unsigned long coalesced = 0; long adjacent, i; - if ((rgn->cnt == 1) && (rgn->region[0].size == 0)) { - rgn->region[0].base = base; - rgn->region[0].size = size; + if ((type->cnt == 1) && (type->regions[0].size == 0)) { + type->regions[0].base = base; + type->regions[0].size = size; return 0; } /* First try and coalesce this LMB with another. */ - for (i = 0; i < rgn->cnt; i++) { - u64 rgnbase = rgn->region[i].base; - u64 rgnsize = rgn->region[i].size; + for (i = 0; i < type->cnt; i++) { + u64 rgnbase = type->regions[i].base; + u64 rgnsize = type->regions[i].size; if ((rgnbase == base) && (rgnsize == size)) /* Already have this region, so we're done */ @@ -151,61 +151,59 @@ static long lmb_add_region(struct lmb_region *rgn, u64 base, u64 size) adjacent = lmb_addrs_adjacent(base, size, rgnbase, rgnsize); if (adjacent > 0) { - rgn->region[i].base -= size; - rgn->region[i].size += size; + type->regions[i].base -= size; + type->regions[i].size += size; coalesced++; break; } else if (adjacent < 0) { - rgn->region[i].size += size; + type->regions[i].size += size; coalesced++; break; } } - if ((i < rgn->cnt - 1) && lmb_regions_adjacent(rgn, i, i+1)) { - lmb_coalesce_regions(rgn, i, i+1); + if ((i < type->cnt - 1) && lmb_regions_adjacent(type, i, i+1)) { + lmb_coalesce_regions(type, i, i+1); coalesced++; } if (coalesced) return coalesced; - if (rgn->cnt >= MAX_LMB_REGIONS) + if (type->cnt >= MAX_LMB_REGIONS) return -1; /* Couldn't coalesce the LMB, so add it to the sorted table. */ - for (i = rgn->cnt - 1; i >= 0; i--) { - if (base < rgn->region[i].base) { - rgn->region[i+1].base = rgn->region[i].base; - rgn->region[i+1].size = rgn->region[i].size; + for (i = type->cnt - 1; i >= 0; i--) { + if (base < type->regions[i].base) { + type->regions[i+1].base = type->regions[i].base; + type->regions[i+1].size = type->regions[i].size; } else { - rgn->region[i+1].base = base; - rgn->region[i+1].size = size; + type->regions[i+1].base = base; + type->regions[i+1].size = size; break; } } - if (base < rgn->region[0].base) { - rgn->region[0].base = base; - rgn->region[0].size = size; + if (base < type->regions[0].base) { + type->regions[0].base = base; + type->regions[0].size = size; } - rgn->cnt++; + type->cnt++; return 0; } long lmb_add(u64 base, u64 size) { - struct lmb_region *_rgn = &lmb.memory; - /* On pSeries LPAR systems, the first LMB is our RMO region. */ if (base == 0) lmb.rmo_size = size; - return lmb_add_region(_rgn, base, size); + return lmb_add_region(&lmb.memory, base, size); } -static long __lmb_remove(struct lmb_region *rgn, u64 base, u64 size) +static long __lmb_remove(struct lmb_type *type, u64 base, u64 size) { u64 rgnbegin, rgnend; u64 end = base + size; @@ -214,34 +212,34 @@ static long __lmb_remove(struct lmb_region *rgn, u64 base, u64 size) rgnbegin = rgnend = 0; /* supress gcc warnings */ /* Find the region where (base, size) belongs to */ - for (i=0; i < rgn->cnt; i++) { - rgnbegin = rgn->region[i].base; - rgnend = rgnbegin + rgn->region[i].size; + for (i=0; i < type->cnt; i++) { + rgnbegin = type->regions[i].base; + rgnend = rgnbegin + type->regions[i].size; if ((rgnbegin <= base) && (end <= rgnend)) break; } /* Didn't find the region */ - if (i == rgn->cnt) + if (i == type->cnt) return -1; /* Check to see if we are removing entire region */ if ((rgnbegin == base) && (rgnend == end)) { - lmb_remove_region(rgn, i); + lmb_remove_region(type, i); return 0; } /* Check to see if region is matching at the front */ if (rgnbegin == base) { - rgn->region[i].base = end; - rgn->region[i].size -= size; + type->regions[i].base = end; + type->regions[i].size -= size; return 0; } /* Check to see if the region is matching at the end */ if (rgnend == end) { - rgn->region[i].size -= size; + type->regions[i].size -= size; return 0; } @@ -249,8 +247,8 @@ static long __lmb_remove(struct lmb_region *rgn, u64 base, u64 size) * We need to split the entry - adjust the current one to the * beginging of the hole and add the region after hole. */ - rgn->region[i].size = base - rgn->region[i].base; - return lmb_add_region(rgn, end, rgnend - end); + type->regions[i].size = base - type->regions[i].base; + return lmb_add_region(type, end, rgnend - end); } long lmb_remove(u64 base, u64 size) @@ -265,25 +263,25 @@ long __init lmb_free(u64 base, u64 size) long __init lmb_reserve(u64 base, u64 size) { - struct lmb_region *_rgn = &lmb.reserved; + struct lmb_type *_rgn = &lmb.reserved; BUG_ON(0 == size); return lmb_add_region(_rgn, base, size); } -long lmb_overlaps_region(struct lmb_region *rgn, u64 base, u64 size) +long lmb_overlaps_region(struct lmb_type *type, u64 base, u64 size) { unsigned long i; - for (i = 0; i < rgn->cnt; i++) { - u64 rgnbase = rgn->region[i].base; - u64 rgnsize = rgn->region[i].size; + for (i = 0; i < type->cnt; i++) { + u64 rgnbase = type->regions[i].base; + u64 rgnsize = type->regions[i].size; if (lmb_addrs_overlap(base, size, rgnbase, rgnsize)) break; } - return (i < rgn->cnt) ? i : -1; + return (i < type->cnt) ? i : -1; } static u64 lmb_align_down(u64 addr, u64 size) @@ -311,7 +309,7 @@ static u64 __init lmb_alloc_nid_unreserved(u64 start, u64 end, base = ~(u64)0; return base; } - res_base = lmb.reserved.region[j].base; + res_base = lmb.reserved.regions[j].base; if (res_base < size) break; base = lmb_align_down(res_base - size, align); @@ -320,7 +318,7 @@ static u64 __init lmb_alloc_nid_unreserved(u64 start, u64 end, return ~(u64)0; } -static u64 __init lmb_alloc_nid_region(struct lmb_property *mp, +static u64 __init lmb_alloc_nid_region(struct lmb_region *mp, u64 (*nid_range)(u64, u64, int *), u64 size, u64 align, int nid) { @@ -350,7 +348,7 @@ static u64 __init lmb_alloc_nid_region(struct lmb_property *mp, u64 __init lmb_alloc_nid(u64 size, u64 align, int nid, u64 (*nid_range)(u64 start, u64 end, int *nid)) { - struct lmb_region *mem = &lmb.memory; + struct lmb_type *mem = &lmb.memory; int i; BUG_ON(0 == size); @@ -358,7 +356,7 @@ u64 __init lmb_alloc_nid(u64 size, u64 align, int nid, size = lmb_align_up(size, align); for (i = 0; i < mem->cnt; i++) { - u64 ret = lmb_alloc_nid_region(&mem->region[i], + u64 ret = lmb_alloc_nid_region(&mem->regions[i], nid_range, size, align, nid); if (ret != ~(u64)0) @@ -402,8 +400,8 @@ u64 __init __lmb_alloc_base(u64 size, u64 align, u64 max_addr) max_addr = LMB_REAL_LIMIT; for (i = lmb.memory.cnt - 1; i >= 0; i--) { - u64 lmbbase = lmb.memory.region[i].base; - u64 lmbsize = lmb.memory.region[i].size; + u64 lmbbase = lmb.memory.regions[i].base; + u64 lmbsize = lmb.memory.regions[i].size; if (lmbsize < size) continue; @@ -423,7 +421,7 @@ u64 __init __lmb_alloc_base(u64 size, u64 align, u64 max_addr) return 0; return base; } - res_base = lmb.reserved.region[j].base; + res_base = lmb.reserved.regions[j].base; if (res_base < size) break; base = lmb_align_down(res_base - size, align); @@ -442,7 +440,7 @@ u64 lmb_end_of_DRAM(void) { int idx = lmb.memory.cnt - 1; - return (lmb.memory.region[idx].base + lmb.memory.region[idx].size); + return (lmb.memory.regions[idx].base + lmb.memory.regions[idx].size); } /* You must call lmb_analyze() after this. */ @@ -450,7 +448,7 @@ void __init lmb_enforce_memory_limit(u64 memory_limit) { unsigned long i; u64 limit; - struct lmb_property *p; + struct lmb_region *p; if (!memory_limit) return; @@ -458,24 +456,24 @@ void __init lmb_enforce_memory_limit(u64 memory_limit) /* Truncate the lmb regions to satisfy the memory limit. */ limit = memory_limit; for (i = 0; i < lmb.memory.cnt; i++) { - if (limit > lmb.memory.region[i].size) { - limit -= lmb.memory.region[i].size; + if (limit > lmb.memory.regions[i].size) { + limit -= lmb.memory.regions[i].size; continue; } - lmb.memory.region[i].size = limit; + lmb.memory.regions[i].size = limit; lmb.memory.cnt = i + 1; break; } - if (lmb.memory.region[0].size < lmb.rmo_size) - lmb.rmo_size = lmb.memory.region[0].size; + if (lmb.memory.regions[0].size < lmb.rmo_size) + lmb.rmo_size = lmb.memory.regions[0].size; memory_limit = lmb_end_of_DRAM(); /* And truncate any reserves above the limit also. */ for (i = 0; i < lmb.reserved.cnt; i++) { - p = &lmb.reserved.region[i]; + p = &lmb.reserved.regions[i]; if (p->base > memory_limit) p->size = 0; @@ -494,9 +492,9 @@ int __init lmb_is_reserved(u64 addr) int i; for (i = 0; i < lmb.reserved.cnt; i++) { - u64 upper = lmb.reserved.region[i].base + - lmb.reserved.region[i].size - 1; - if ((addr >= lmb.reserved.region[i].base) && (addr <= upper)) + u64 upper = lmb.reserved.regions[i].base + + lmb.reserved.regions[i].size - 1; + if ((addr >= lmb.reserved.regions[i].base) && (addr <= upper)) return 1; } return 0; @@ -511,7 +509,7 @@ int lmb_is_region_reserved(u64 base, u64 size) * Given a <base, len>, find which memory regions belong to this range. * Adjust the request and return a contiguous chunk. */ -int lmb_find(struct lmb_property *res) +int lmb_find(struct lmb_region *res) { int i; u64 rstart, rend; @@ -520,8 +518,8 @@ int lmb_find(struct lmb_property *res) rend = rstart + res->size - 1; for (i = 0; i < lmb.memory.cnt; i++) { - u64 start = lmb.memory.region[i].base; - u64 end = start + lmb.memory.region[i].size - 1; + u64 start = lmb.memory.regions[i].base; + u64 end = start + lmb.memory.regions[i].size - 1; if (start > rend) return -1; -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 02/25] lmb: No reason to include asm/lmb.h late 2010-05-10 9:38 ` [PATCH 01/25] lmb: Rename lmb_region to lmb_type and lmb_property to lmb_region Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 03/25] lmb: Introduce for_each_lmb() and new accessors, and use it Benjamin Herrenschmidt 0 siblings, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- include/linux/lmb.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/lmb.h b/include/linux/lmb.h index d225d78..de8031f 100644 --- a/include/linux/lmb.h +++ b/include/linux/lmb.h @@ -16,6 +16,8 @@ #include <linux/init.h> #include <linux/mm.h> +#include <asm/lmb.h> + #define MAX_LMB_REGIONS 128 struct lmb_region { @@ -82,8 +84,6 @@ lmb_end_pfn(struct lmb_type *type, unsigned long region_nr) lmb_size_pages(type, region_nr); } -#include <asm/lmb.h> - #endif /* __KERNEL__ */ #endif /* _LINUX_LMB_H */ -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 03/25] lmb: Introduce for_each_lmb() and new accessors, and use it 2010-05-10 9:38 ` [PATCH 02/25] lmb: No reason to include asm/lmb.h late Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 04/25] lmb: Remove nid_range argument, arch provides lmb_nid_range() instead Benjamin Herrenschmidt 2010-05-10 15:39 ` [PATCH 03/25] lmb: Introduce for_each_lmb() and new accessors, and use it Thomas Gleixner 0 siblings, 2 replies; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt Walk lmb's using for_each_lmb() and use lmb_region_base/end_pfn() for getting to PFNs. Update sparc, powerpc, microblaze and sh. Note: This is -almost- a direct conversion. It doesn't fix some existing crap when/if lmb's aren't page aligned in the first place. This will be sorted out separately. This removes lmb_find() as well, which isn't used anymore Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- arch/microblaze/mm/init.c | 18 +++---- arch/powerpc/mm/hash_utils_64.c | 8 ++-- arch/powerpc/mm/mem.c | 92 ++++++++++++++------------------------- arch/powerpc/mm/numa.c | 17 ++++--- arch/sh/kernel/setup.c | 14 +++--- arch/sparc/mm/init_64.c | 30 +++++-------- include/linux/lmb.h | 56 ++++++++++++++++++------ lib/lmb.c | 32 ------------- 8 files changed, 114 insertions(+), 153 deletions(-) diff --git a/arch/microblaze/mm/init.c b/arch/microblaze/mm/init.c index 9d58797..11048b8 100644 --- a/arch/microblaze/mm/init.c +++ b/arch/microblaze/mm/init.c @@ -69,16 +69,16 @@ static void __init paging_init(void) void __init setup_memory(void) { - int i; unsigned long map_size; + struct lmb_region *reg; + #ifndef CONFIG_MMU u32 kernel_align_start, kernel_align_size; /* Find main memory where is the kernel */ - for (i = 0; i < lmb.memory.cnt; i++) { - memory_start = (u32) lmb.memory.regions[i].base; - memory_end = (u32) lmb.memory.regions[i].base - + (u32) lmb.memory.region[i].size; + for_each_lmb(memory, reg) { + memory_start = (u32)reg->base; + memory_end = (u32) reg->base + reg->size; if ((memory_start <= (u32)_text) && ((u32)_text <= memory_end)) { memory_size = memory_end - memory_start; @@ -146,12 +146,10 @@ void __init setup_memory(void) free_bootmem(memory_start, memory_size); /* reserve allocate blocks */ - for (i = 0; i < lmb.reserved.cnt; i++) { + for_each_lmb(reserved, reg) { pr_debug("reserved %d - 0x%08x-0x%08x\n", i, - (u32) lmb.reserved.region[i].base, - (u32) lmb_size_bytes(&lmb.reserved, i)); - reserve_bootmem(lmb.reserved.region[i].base, - lmb_size_bytes(&lmb.reserved, i) - 1, BOOTMEM_DEFAULT); + (u32) reg->base, (u32) reg->size); + reserve_bootmem(reg->base, reg->size, BOOTMEM_DEFAULT); } #ifdef CONFIG_MMU init_bootmem_done = 1; diff --git a/arch/powerpc/mm/hash_utils_64.c b/arch/powerpc/mm/hash_utils_64.c index 0a232f5..2fdeedf 100644 --- a/arch/powerpc/mm/hash_utils_64.c +++ b/arch/powerpc/mm/hash_utils_64.c @@ -588,7 +588,7 @@ static void __init htab_initialize(void) unsigned long pteg_count; unsigned long prot; unsigned long base = 0, size = 0, limit; - int i; + struct lmb_region *reg; DBG(" -> htab_initialize()\n"); @@ -659,9 +659,9 @@ static void __init htab_initialize(void) */ /* create bolted the linear mapping in the hash table */ - for (i=0; i < lmb.memory.cnt; i++) { - base = (unsigned long)__va(lmb.memory.regions[i].base); - size = lmb.memory.region[i].size; + for_each_lmb(memory, reg) { + base = (unsigned long)__va(reg->base); + size = reg->size; DBG("creating mapping for region: %lx..%lx (prot: %lx)\n", base, size, prot); diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c index 65acb49..17a8027 100644 --- a/arch/powerpc/mm/mem.c +++ b/arch/powerpc/mm/mem.c @@ -82,18 +82,11 @@ int page_is_ram(unsigned long pfn) return pfn < max_pfn; #else unsigned long paddr = (pfn << PAGE_SHIFT); - int i; - for (i=0; i < lmb.memory.cnt; i++) { - unsigned long base; + struct lmb_region *reg; - base = lmb.memory.regions[i].base; - - if ((paddr >= base) && - (paddr < (base + lmb.memory.regions[i].size))) { + for_each_lmb(memory, reg) + if (paddr >= reg->base && paddr < (reg->base + reg->size)) return 1; - } - } - return 0; #endif } @@ -149,23 +142,19 @@ int walk_system_ram_range(unsigned long start_pfn, unsigned long nr_pages, void *arg, int (*func)(unsigned long, unsigned long, void *)) { - struct lmb_region res; - unsigned long pfn, len; - u64 end; + struct lmb_region *reg; + unsigned long end_pfn = start_pfn + nr_pages; + unsigned long tstart, tend; int ret = -1; - res.base = (u64) start_pfn << PAGE_SHIFT; - res.size = (u64) nr_pages << PAGE_SHIFT; - - end = res.base + res.size - 1; - while ((res.base < end) && (lmb_find(&res) >= 0)) { - pfn = (unsigned long)(res.base >> PAGE_SHIFT); - len = (unsigned long)(res.size >> PAGE_SHIFT); - ret = (*func)(pfn, len, arg); + for_each_lmb(memory, reg) { + tstart = max(start_pfn, lmb_region_base_pfn(reg)); + tend = min(end_pfn, lmb_region_end_pfn(reg)); + if (tstart >= tend) + continue; + ret = (*func)(tstart, tend - tstart, arg); if (ret) break; - res.base += (res.size + 1); - res.size = (end - res.base + 1); } return ret; } @@ -179,9 +168,9 @@ EXPORT_SYMBOL_GPL(walk_system_ram_range); #ifndef CONFIG_NEED_MULTIPLE_NODES void __init do_init_bootmem(void) { - unsigned long i; unsigned long start, bootmap_pages; unsigned long total_pages; + struct lmb_region *reg; int boot_mapsize; max_low_pfn = max_pfn = lmb_end_of_DRAM() >> PAGE_SHIFT; @@ -204,10 +193,10 @@ void __init do_init_bootmem(void) boot_mapsize = init_bootmem_node(NODE_DATA(0), start >> PAGE_SHIFT, min_low_pfn, max_low_pfn); /* Add active regions with valid PFNs */ - for (i = 0; i < lmb.memory.cnt; i++) { + for_each_lmb(memory, reg) { unsigned long start_pfn, end_pfn; - start_pfn = lmb.memory.regions[i].base >> PAGE_SHIFT; - end_pfn = start_pfn + lmb_size_pages(&lmb.memory, i); + start_pfn = lmb_region_base_pfn(reg); + end_pfn = lmb_region_end_pfn(reg); add_active_range(0, start_pfn, end_pfn); } @@ -218,29 +207,21 @@ void __init do_init_bootmem(void) free_bootmem_with_active_regions(0, lowmem_end_addr >> PAGE_SHIFT); /* reserve the sections we're already using */ - for (i = 0; i < lmb.reserved.cnt; i++) { - unsigned long addr = lmb.reserved.regions[i].base + - lmb_size_bytes(&lmb.reserved, i) - 1; - if (addr < lowmem_end_addr) - reserve_bootmem(lmb.reserved.regions[i].base, - lmb_size_bytes(&lmb.reserved, i), - BOOTMEM_DEFAULT); - else if (lmb.reserved.regions[i].base < lowmem_end_addr) { - unsigned long adjusted_size = lowmem_end_addr - - lmb.reserved.regions[i].base; - reserve_bootmem(lmb.reserved.regions[i].base, - adjusted_size, BOOTMEM_DEFAULT); + for_each_lmb(reserved, reg) { + unsigned long top = reg->base + reg->size - 1; + if (top < lowmem_end_addr) + reserve_bootmem(reg->base, reg->size, BOOTMEM_DEFAULT); + else if (reg->base < lowmem_end_addr) { + unsigned long trunc_size = lowmem_end_addr - reg->base; + reserve_bootmem(reg->base, trunc_size, BOOTMEM_DEFAULT); } } #else free_bootmem_with_active_regions(0, max_pfn); /* reserve the sections we're already using */ - for (i = 0; i < lmb.reserved.cnt; i++) - reserve_bootmem(lmb.reserved.regions[i].base, - lmb_size_bytes(&lmb.reserved, i), - BOOTMEM_DEFAULT); - + for_each_lmb(reserved, reg) + reserve_bootmem(reg->base, reg->size, BOOTMEM_DEFAULT); #endif /* XXX need to clip this if using highmem? */ sparse_memory_present_with_active_regions(0); @@ -251,22 +232,15 @@ void __init do_init_bootmem(void) /* mark pages that don't exist as nosave */ static int __init mark_nonram_nosave(void) { - unsigned long lmb_next_region_start_pfn, - lmb_region_max_pfn; - int i; - - for (i = 0; i < lmb.memory.cnt - 1; i++) { - lmb_region_max_pfn = - (lmb.memory.regions[i].base >> PAGE_SHIFT) + - (lmb.memory.regions[i].size >> PAGE_SHIFT); - lmb_next_region_start_pfn = - lmb.memory.regions[i+1].base >> PAGE_SHIFT; - - if (lmb_region_max_pfn < lmb_next_region_start_pfn) - register_nosave_region(lmb_region_max_pfn, - lmb_next_region_start_pfn); + struct lmb_region *reg, *prev = NULL; + + for_each_lmb(memory, reg) { + if (prev && + lmb_region_end_pfn(prev) < lmb_region_base_pfn(reg)) + register_nosave_region(lmb_region_end_pfn(prev), + lmb_region_base_pfn(reg)); + prev = reg; } - return 0; } diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c index aace5e5..821d4ef 100644 --- a/arch/powerpc/mm/numa.c +++ b/arch/powerpc/mm/numa.c @@ -742,16 +742,17 @@ static void __init setup_nonnuma(void) unsigned long top_of_ram = lmb_end_of_DRAM(); unsigned long total_ram = lmb_phys_mem_size(); unsigned long start_pfn, end_pfn; - unsigned int i, nid = 0; + unsigned int nid = 0; + struct lmb_region *reg; printk(KERN_DEBUG "Top of RAM: 0x%lx, Total RAM: 0x%lx\n", top_of_ram, total_ram); printk(KERN_DEBUG "Memory hole size: %ldMB\n", (top_of_ram - total_ram) >> 20); - for (i = 0; i < lmb.memory.cnt; ++i) { - start_pfn = lmb.memory.region[i].base >> PAGE_SHIFT; - end_pfn = start_pfn + lmb_size_pages(&lmb.memory, i); + for_each_lmb(memory, reg) { + start_pfn = lmb_region_base_pfn(reg); + end_pfn = lmb_region_end_pfn(reg); fake_numa_create_new_node(end_pfn, &nid); add_active_range(nid, start_pfn, end_pfn); @@ -887,11 +888,11 @@ static struct notifier_block __cpuinitdata ppc64_numa_nb = { static void mark_reserved_regions_for_nid(int nid) { struct pglist_data *node = NODE_DATA(nid); - int i; + struct lmb_region *reg; - for (i = 0; i < lmb.reserved.cnt; i++) { - unsigned long physbase = lmb.reserved.region[i].base; - unsigned long size = lmb.reserved.region[i].size; + for_each_lmb(reserved, reg) { + unsigned long physbase = reg->base; + unsigned long size = reg->size; unsigned long start_pfn = physbase >> PAGE_SHIFT; unsigned long end_pfn = PFN_UP(physbase + size); struct node_active_region node_ar; diff --git a/arch/sh/kernel/setup.c b/arch/sh/kernel/setup.c index 8870d6b..52f62f7 100644 --- a/arch/sh/kernel/setup.c +++ b/arch/sh/kernel/setup.c @@ -237,7 +237,7 @@ void __init setup_bootmem_allocator(unsigned long free_pfn) unsigned long bootmap_size; unsigned long bootmap_pages, bootmem_paddr; u64 total_pages = (lmb_end_of_DRAM() - __MEMORY_START) >> PAGE_SHIFT; - int i; + struct lmb_region *reg; bootmap_pages = bootmem_bootmap_pages(total_pages); @@ -253,10 +253,10 @@ void __init setup_bootmem_allocator(unsigned long free_pfn) min_low_pfn, max_low_pfn); /* Add active regions with valid PFNs. */ - for (i = 0; i < lmb.memory.cnt; i++) { + for_each_lmb(memory, reg) { unsigned long start_pfn, end_pfn; - start_pfn = lmb.memory.region[i].base >> PAGE_SHIFT; - end_pfn = start_pfn + lmb_size_pages(&lmb.memory, i); + start_pfn = lmb_region_base_pfn(reg); + end_pfn = lmb_region_end_pfn(reg); __add_active_range(0, start_pfn, end_pfn); } @@ -267,10 +267,8 @@ void __init setup_bootmem_allocator(unsigned long free_pfn) register_bootmem_low_pages(); /* Reserve the sections we're already using. */ - for (i = 0; i < lmb.reserved.cnt; i++) - reserve_bootmem(lmb.reserved.region[i].base, - lmb_size_bytes(&lmb.reserved, i), - BOOTMEM_DEFAULT); + for_each_lmb(reserved, reg) + reserve_bootmem(reg->base, reg->size, BOOTMEM_DEFAULT); node_set_online(0); diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c index 33628b4..04590c9 100644 --- a/arch/sparc/mm/init_64.c +++ b/arch/sparc/mm/init_64.c @@ -972,13 +972,13 @@ int of_node_to_nid(struct device_node *dp) static void __init add_node_ranges(void) { - int i; + struct lmb_region *reg; - for (i = 0; i < lmb.memory.cnt; i++) { - unsigned long size = lmb_size_bytes(&lmb.memory, i); + for_each_lmb(memory, reg) { + unsigned long size = reg->size; unsigned long start, end; - start = lmb.memory.regions[i].base; + start = reg->base; end = start + size; while (start < end) { unsigned long this_end; @@ -1281,7 +1281,7 @@ static void __init bootmem_init_nonnuma(void) { unsigned long top_of_ram = lmb_end_of_DRAM(); unsigned long total_ram = lmb_phys_mem_size(); - unsigned int i; + struct lmb_region *reg; numadbg("bootmem_init_nonnuma()\n"); @@ -1292,15 +1292,14 @@ static void __init bootmem_init_nonnuma(void) init_node_masks_nonnuma(); - for (i = 0; i < lmb.memory.cnt; i++) { - unsigned long size = lmb_size_bytes(&lmb.memory, i); + for_each_lmb(memory, reg) { unsigned long start_pfn, end_pfn; - if (!size) + if (!reg->size) continue; - start_pfn = lmb.memory.regions[i].base >> PAGE_SHIFT; - end_pfn = start_pfn + lmb_size_pages(&lmb.memory, i); + start_pfn = lmb_region_base_pfn(reg); + end_pfn = lmb_region_end_pfn(reg); add_active_range(0, start_pfn, end_pfn); } @@ -1334,17 +1333,12 @@ static void __init reserve_range_in_node(int nid, unsigned long start, static void __init trim_reserved_in_node(int nid) { - int i; + struct lmb_region *reg; numadbg(" trim_reserved_in_node(%d)\n", nid); - for (i = 0; i < lmb.reserved.cnt; i++) { - unsigned long start = lmb.reserved.regions[i].base; - unsigned long size = lmb_size_bytes(&lmb.reserved, i); - unsigned long end = start + size; - - reserve_range_in_node(nid, start, end); - } + for_each_lmb(reserved, reg) + reserve_range_in_node(nid, reg->base, reg->base + reg->size); } static void __init bootmem_init_one_node(int nid) diff --git a/include/linux/lmb.h b/include/linux/lmb.h index de8031f..c2410fe 100644 --- a/include/linux/lmb.h +++ b/include/linux/lmb.h @@ -58,32 +58,60 @@ extern u64 lmb_end_of_DRAM(void); extern void __init lmb_enforce_memory_limit(u64 memory_limit); extern int __init lmb_is_reserved(u64 addr); extern int lmb_is_region_reserved(u64 base, u64 size); -extern int lmb_find(struct lmb_region *res); extern void lmb_dump_all(void); -static inline u64 -lmb_size_bytes(struct lmb_type *type, unsigned long region_nr) +/* + * pfn conversion functions + * + * While the memory LMBs should always be page aligned, the reserved + * LMBs may not be. This accessor attempt to provide a very clear + * idea of what they return for such non aligned LMBs. + */ + +/** + * lmb_region_base_pfn - Return the lowest pfn intersecting with the region + * @reg: lmb_region structure + */ +static inline unsigned long lmb_region_base_pfn(const struct lmb_region *reg) { - return type->regions[region_nr].size; + return reg->base >> PAGE_SHIFT; } -static inline u64 -lmb_size_pages(struct lmb_type *type, unsigned long region_nr) + +/** + * lmb_region_last_pfn - Return the highest pfn intersecting with the region + * @reg: lmb_region structure + */ +static inline unsigned long lmb_region_last_pfn(const struct lmb_region *reg) { - return lmb_size_bytes(type, region_nr) >> PAGE_SHIFT; + return (reg->base + reg->size - 1) >> PAGE_SHIFT; } -static inline u64 -lmb_start_pfn(struct lmb_type *type, unsigned long region_nr) + +/** + * lmb_region_end_pfn - Return the pfn of the first page following the region + * but not intersecting it + * @reg: lmb_region structure + */ +static inline unsigned long lmb_region_end_pfn(const struct lmb_region *reg) { - return type->regions[region_nr].base >> PAGE_SHIFT; + return lmb_region_last_pfn(reg) + 1; } -static inline u64 -lmb_end_pfn(struct lmb_type *type, unsigned long region_nr) + +/** + * lmb_region_pages - Return the number of pages covering a region + * @reg: lmb_region structure + */ +static inline unsigned long lmb_region_pages(const struct lmb_region *reg) { - return lmb_start_pfn(type, region_nr) + - lmb_size_pages(type, region_nr); + return lmb_region_end_pfn(reg) - lmb_region_end_pfn(reg); } +#define for_each_lmb(lmb_type, region) \ + for (reg = lmb.lmb_type.regions; \ + region < (lmb.lmb_type.regions + lmb.lmb_type.cnt); \ + region++) + + #endif /* __KERNEL__ */ #endif /* _LINUX_LMB_H */ diff --git a/lib/lmb.c b/lib/lmb.c index f07337e..5f21033 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -505,35 +505,3 @@ int lmb_is_region_reserved(u64 base, u64 size) return lmb_overlaps_region(&lmb.reserved, base, size); } -/* - * Given a <base, len>, find which memory regions belong to this range. - * Adjust the request and return a contiguous chunk. - */ -int lmb_find(struct lmb_region *res) -{ - int i; - u64 rstart, rend; - - rstart = res->base; - rend = rstart + res->size - 1; - - for (i = 0; i < lmb.memory.cnt; i++) { - u64 start = lmb.memory.regions[i].base; - u64 end = start + lmb.memory.regions[i].size - 1; - - if (start > rend) - return -1; - - if ((end >= rstart) && (start < rend)) { - /* adjust the request */ - if (rstart < start) - rstart = start; - if (rend > end) - rend = end; - res->base = rstart; - res->size = rend - rstart + 1; - return 0; - } - } - return -1; -} -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 04/25] lmb: Remove nid_range argument, arch provides lmb_nid_range() instead 2010-05-10 9:38 ` [PATCH 03/25] lmb: Introduce for_each_lmb() and new accessors, and use it Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 05/25] lmb: Factor the lowest level alloc function Benjamin Herrenschmidt 2010-05-10 15:39 ` [PATCH 03/25] lmb: Introduce for_each_lmb() and new accessors, and use it Thomas Gleixner 1 sibling, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- arch/sparc/mm/init_64.c | 16 ++++++---------- include/linux/lmb.h | 7 +++++-- lib/lmb.c | 13 ++++++++----- 3 files changed, 19 insertions(+), 17 deletions(-) diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c index 04590c9..88443c8 100644 --- a/arch/sparc/mm/init_64.c +++ b/arch/sparc/mm/init_64.c @@ -785,8 +785,7 @@ static int find_node(unsigned long addr) return -1; } -static unsigned long long nid_range(unsigned long long start, - unsigned long long end, int *nid) +u64 lmb_nid_range(u64 start, u64 end, int *nid) { *nid = find_node(start); start += PAGE_SIZE; @@ -804,8 +803,7 @@ static unsigned long long nid_range(unsigned long long start, return start; } #else -static unsigned long long nid_range(unsigned long long start, - unsigned long long end, int *nid) +u64 lmb_nid_range(u64 start, u64 end, int *nid) { *nid = 0; return end; @@ -822,8 +820,7 @@ static void __init allocate_node_data(int nid) struct pglist_data *p; #ifdef CONFIG_NEED_MULTIPLE_NODES - paddr = lmb_alloc_nid(sizeof(struct pglist_data), - SMP_CACHE_BYTES, nid, nid_range); + paddr = lmb_alloc_nid(sizeof(struct pglist_data), SMP_CACHE_BYTES, nid); if (!paddr) { prom_printf("Cannot allocate pglist_data for nid[%d]\n", nid); prom_halt(); @@ -843,8 +840,7 @@ static void __init allocate_node_data(int nid) if (p->node_spanned_pages) { num_pages = bootmem_bootmap_pages(p->node_spanned_pages); - paddr = lmb_alloc_nid(num_pages << PAGE_SHIFT, PAGE_SIZE, nid, - nid_range); + paddr = lmb_alloc_nid(num_pages << PAGE_SHIFT, PAGE_SIZE, nid); if (!paddr) { prom_printf("Cannot allocate bootmap for nid[%d]\n", nid); @@ -984,7 +980,7 @@ static void __init add_node_ranges(void) unsigned long this_end; int nid; - this_end = nid_range(start, end, &nid); + this_end = lmb_nid_range(start, end, &nid); numadbg("Adding active range nid[%d] " "start[%lx] end[%lx]\n", @@ -1317,7 +1313,7 @@ static void __init reserve_range_in_node(int nid, unsigned long start, unsigned long this_end; int n; - this_end = nid_range(start, end, &n); + this_end = lmb_nid_range(start, end, &n); if (n == nid) { numadbg(" MATCH reserving range [%lx:%lx]\n", start, this_end); diff --git a/include/linux/lmb.h b/include/linux/lmb.h index c2410fe..9caa67a 100644 --- a/include/linux/lmb.h +++ b/include/linux/lmb.h @@ -46,8 +46,7 @@ extern long lmb_add(u64 base, u64 size); extern long lmb_remove(u64 base, u64 size); extern long __init lmb_free(u64 base, u64 size); extern long __init lmb_reserve(u64 base, u64 size); -extern u64 __init lmb_alloc_nid(u64 size, u64 align, int nid, - u64 (*nid_range)(u64, u64, int *)); +extern u64 __init lmb_alloc_nid(u64 size, u64 align, int nid); extern u64 __init lmb_alloc(u64 size, u64 align); extern u64 __init lmb_alloc_base(u64 size, u64, u64 max_addr); @@ -61,6 +60,10 @@ extern int lmb_is_region_reserved(u64 base, u64 size); extern void lmb_dump_all(void); +/* Provided by the architecture */ +extern u64 lmb_nid_range(u64 start, u64 end, int *nid); + + /* * pfn conversion functions * diff --git a/lib/lmb.c b/lib/lmb.c index 5f21033..be3d7d9 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -319,7 +319,6 @@ static u64 __init lmb_alloc_nid_unreserved(u64 start, u64 end, } static u64 __init lmb_alloc_nid_region(struct lmb_region *mp, - u64 (*nid_range)(u64, u64, int *), u64 size, u64 align, int nid) { u64 start, end; @@ -332,7 +331,7 @@ static u64 __init lmb_alloc_nid_region(struct lmb_region *mp, u64 this_end; int this_nid; - this_end = nid_range(start, end, &this_nid); + this_end = lmb_nid_range(start, end, &this_nid); if (this_nid == nid) { u64 ret = lmb_alloc_nid_unreserved(start, this_end, size, align); @@ -345,8 +344,7 @@ static u64 __init lmb_alloc_nid_region(struct lmb_region *mp, return ~(u64)0; } -u64 __init lmb_alloc_nid(u64 size, u64 align, int nid, - u64 (*nid_range)(u64 start, u64 end, int *nid)) +u64 __init lmb_alloc_nid(u64 size, u64 align, int nid) { struct lmb_type *mem = &lmb.memory; int i; @@ -357,7 +355,6 @@ u64 __init lmb_alloc_nid(u64 size, u64 align, int nid, for (i = 0; i < mem->cnt; i++) { u64 ret = lmb_alloc_nid_region(&mem->regions[i], - nid_range, size, align, nid); if (ret != ~(u64)0) return ret; @@ -505,3 +502,9 @@ int lmb_is_region_reserved(u64 base, u64 size) return lmb_overlaps_region(&lmb.reserved, base, size); } +u64 __weak lmb_nid_range(u64 start, u64 end, int *nid) +{ + *nid = 0; + + return end; +} -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 05/25] lmb: Factor the lowest level alloc function 2010-05-10 9:38 ` [PATCH 04/25] lmb: Remove nid_range argument, arch provides lmb_nid_range() instead Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 06/25] lmb: Expose LMB_ALLOC_ANYWHERE Benjamin Herrenschmidt 2010-05-11 12:30 ` [PATCH 05/25] lmb: Factor the lowest level alloc function Thomas Gleixner 0 siblings, 2 replies; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- lib/lmb.c | 59 +++++++++++++++++++++++++++-------------------------------- 1 files changed, 27 insertions(+), 32 deletions(-) diff --git a/lib/lmb.c b/lib/lmb.c index be3d7d9..00d5808 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -294,8 +294,8 @@ static u64 lmb_align_up(u64 addr, u64 size) return (addr + (size - 1)) & ~(size - 1); } -static u64 __init lmb_alloc_nid_unreserved(u64 start, u64 end, - u64 size, u64 align) +static u64 __init lmb_alloc_region(u64 start, u64 end, + u64 size, u64 align) { u64 base, res_base; long j; @@ -318,6 +318,13 @@ static u64 __init lmb_alloc_nid_unreserved(u64 start, u64 end, return ~(u64)0; } +u64 __weak __init lmb_nid_range(u64 start, u64 end, int *nid) +{ + *nid = 0; + + return end; +} + static u64 __init lmb_alloc_nid_region(struct lmb_region *mp, u64 size, u64 align, int nid) { @@ -333,8 +340,7 @@ static u64 __init lmb_alloc_nid_region(struct lmb_region *mp, this_end = lmb_nid_range(start, end, &this_nid); if (this_nid == nid) { - u64 ret = lmb_alloc_nid_unreserved(start, this_end, - size, align); + u64 ret = lmb_alloc_region(start, this_end, size, align); if (ret != ~(u64)0) return ret; } @@ -351,6 +357,10 @@ u64 __init lmb_alloc_nid(u64 size, u64 align, int nid) BUG_ON(0 == size); + /* We do a bottom-up search for a region with the right + * nid since that's easier considering how lmb_nid_range() + * works + */ size = lmb_align_up(size, align); for (i = 0; i < mem->cnt; i++) { @@ -383,7 +393,7 @@ u64 __init lmb_alloc_base(u64 size, u64 align, u64 max_addr) u64 __init __lmb_alloc_base(u64 size, u64 align, u64 max_addr) { - long i, j; + long i; u64 base = 0; u64 res_base; @@ -396,33 +406,24 @@ u64 __init __lmb_alloc_base(u64 size, u64 align, u64 max_addr) if (max_addr == LMB_ALLOC_ANYWHERE) max_addr = LMB_REAL_LIMIT; + /* Pump up max_addr */ + if (max_addr == LMB_ALLOC_ANYWHERE) + max_addr = ~(u64)0; + + /* We do a top-down search, this tends to limit memory + * fragmentation by keeping early boot allocs near the + * top of memory + */ for (i = lmb.memory.cnt - 1; i >= 0; i--) { u64 lmbbase = lmb.memory.regions[i].base; u64 lmbsize = lmb.memory.regions[i].size; if (lmbsize < size) continue; - if (max_addr == LMB_ALLOC_ANYWHERE) - base = lmb_align_down(lmbbase + lmbsize - size, align); - else if (lmbbase < max_addr) { - base = min(lmbbase + lmbsize, max_addr); - base = lmb_align_down(base - size, align); - } else - continue; - - while (base && lmbbase <= base) { - j = lmb_overlaps_region(&lmb.reserved, base, size); - if (j < 0) { - /* this area isn't reserved, take it */ - if (lmb_add_region(&lmb.reserved, base, size) < 0) - return 0; - return base; - } - res_base = lmb.reserved.regions[j].base; - if (res_base < size) - break; - base = lmb_align_down(res_base - size, align); - } + base = min(lmbbase + lmbsize, max_addr); + res_base = lmb_alloc_region(lmbbase, base, size, align); + if (res_base != ~(u64)0) + return res_base; } return 0; } @@ -502,9 +503,3 @@ int lmb_is_region_reserved(u64 base, u64 size) return lmb_overlaps_region(&lmb.reserved, base, size); } -u64 __weak lmb_nid_range(u64 start, u64 end, int *nid) -{ - *nid = 0; - - return end; -} -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 06/25] lmb: Expose LMB_ALLOC_ANYWHERE 2010-05-10 9:38 ` [PATCH 05/25] lmb: Factor the lowest level alloc function Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 07/25] lmb: Introduce default allocation limit and use it to replace explicit ones Benjamin Herrenschmidt 2010-05-11 12:30 ` [PATCH 05/25] lmb: Factor the lowest level alloc function Thomas Gleixner 1 sibling, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- arch/powerpc/mm/hash_utils_64.c | 2 +- include/linux/lmb.h | 1 + lib/lmb.c | 2 -- 3 files changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/powerpc/mm/hash_utils_64.c b/arch/powerpc/mm/hash_utils_64.c index 2fdeedf..28838e3 100644 --- a/arch/powerpc/mm/hash_utils_64.c +++ b/arch/powerpc/mm/hash_utils_64.c @@ -625,7 +625,7 @@ static void __init htab_initialize(void) if (machine_is(cell)) limit = 0x80000000; else - limit = 0; + limit = LMB_ALLOC_ANYWHERE; table = lmb_alloc_base(htab_size_bytes, htab_size_bytes, limit); diff --git a/include/linux/lmb.h b/include/linux/lmb.h index 9caa67a..f0d2cab 100644 --- a/include/linux/lmb.h +++ b/include/linux/lmb.h @@ -50,6 +50,7 @@ extern u64 __init lmb_alloc_nid(u64 size, u64 align, int nid); extern u64 __init lmb_alloc(u64 size, u64 align); extern u64 __init lmb_alloc_base(u64 size, u64, u64 max_addr); +#define LMB_ALLOC_ANYWHERE 0 extern u64 __init __lmb_alloc_base(u64 size, u64 align, u64 max_addr); extern u64 __init lmb_phys_mem_size(void); diff --git a/lib/lmb.c b/lib/lmb.c index 00d5808..bd81266 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -15,8 +15,6 @@ #include <linux/bitops.h> #include <linux/lmb.h> -#define LMB_ALLOC_ANYWHERE 0 - struct lmb lmb; static int lmb_debug; -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 07/25] lmb: Introduce default allocation limit and use it to replace explicit ones 2010-05-10 9:38 ` [PATCH 06/25] lmb: Expose LMB_ALLOC_ANYWHERE Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 08/25] lmb: Remove rmo_size, burry it in arch/powerpc where it belongs Benjamin Herrenschmidt 0 siblings, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt This introduce lmb.current_limit which is used to limit allocations from lmb_alloc() or lmb_alloc_base(..., LMB_ALLOC_ACCESSIBLE). The old LMB_ALLOC_ANYWHERE changes value from 0 to ~(u64)0 and can still be used to alloc really anywhere. It is -no-longer- cropped to LMB_REAL_LIMIT which disappears. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- arch/microblaze/include/asm/lmb.h | 3 --- arch/powerpc/include/asm/lmb.h | 7 ------- arch/powerpc/kernel/prom.c | 20 +++++++++++++++++++- arch/powerpc/kernel/setup_32.c | 2 +- arch/powerpc/mm/40x_mmu.c | 5 +++-- arch/powerpc/mm/fsl_booke_mmu.c | 3 ++- arch/powerpc/mm/hash_utils_64.c | 3 ++- arch/powerpc/mm/init_32.c | 29 +++++++---------------------- arch/powerpc/mm/ppc_mmu_32.c | 3 +-- arch/powerpc/mm/tlb_nohash.c | 2 ++ arch/sh/include/asm/lmb.h | 2 -- arch/sparc/include/asm/lmb.h | 2 -- include/linux/lmb.h | 16 +++++++++++++++- lib/lmb.c | 19 +++++++++++-------- 14 files changed, 63 insertions(+), 53 deletions(-) diff --git a/arch/microblaze/include/asm/lmb.h b/arch/microblaze/include/asm/lmb.h index a0a0a92..fb4803f 100644 --- a/arch/microblaze/include/asm/lmb.h +++ b/arch/microblaze/include/asm/lmb.h @@ -9,9 +9,6 @@ #ifndef _ASM_MICROBLAZE_LMB_H #define _ASM_MICROBLAZE_LMB_H -/* LMB limit is OFF */ -#define LMB_REAL_LIMIT 0xFFFFFFFF - #endif /* _ASM_MICROBLAZE_LMB_H */ diff --git a/arch/powerpc/include/asm/lmb.h b/arch/powerpc/include/asm/lmb.h index 6f5fdf0..c2d51c9 100644 --- a/arch/powerpc/include/asm/lmb.h +++ b/arch/powerpc/include/asm/lmb.h @@ -5,11 +5,4 @@ #define LMB_DBG(fmt...) udbg_printf(fmt) -#ifdef CONFIG_PPC32 -extern phys_addr_t lowmem_end_addr; -#define LMB_REAL_LIMIT lowmem_end_addr -#else -#define LMB_REAL_LIMIT 0 -#endif - #endif /* _ASM_POWERPC_LMB_H */ diff --git a/arch/powerpc/kernel/prom.c b/arch/powerpc/kernel/prom.c index 05131d6..b8428d3 100644 --- a/arch/powerpc/kernel/prom.c +++ b/arch/powerpc/kernel/prom.c @@ -98,7 +98,7 @@ static void __init move_device_tree(void) if ((memory_limit && (start + size) > memory_limit) || overlaps_crashkernel(start, size)) { - p = __va(lmb_alloc_base(size, PAGE_SIZE, lmb.rmo_size)); + p = __va(lmb_alloc(size, PAGE_SIZE)); memcpy(p, initial_boot_params, size); initial_boot_params = (struct boot_param_header *)p; DBG("Moved device tree to 0x%p\n", p); @@ -655,6 +655,21 @@ static void __init phyp_dump_reserve_mem(void) static inline void __init phyp_dump_reserve_mem(void) {} #endif /* CONFIG_PHYP_DUMP && CONFIG_PPC_RTAS */ +static void set_boot_memory_limit(void) +{ +#ifdef CONFIG_PPC32 + /* 601 can only access 16MB at the moment */ + if (PVR_VER(mfspr(SPRN_PVR)) == 1) + lmb_set_current_limit(0x01000000); + /* 8xx can only access 8MB at the moment */ + else if (PVR_VER(mfspr(SPRN_PVR)) == 0x50) + lmb_set_current_limit(0x00800000); + else + lmb_set_current_limit(0x10000000); +#else + lmb_set_current_limit(lmb.rmo_size); +#endif +} void __init early_init_devtree(void *params) { @@ -683,6 +698,7 @@ void __init early_init_devtree(void *params) /* Scan memory nodes and rebuild LMBs */ lmb_init(); + of_scan_flat_dt(early_init_dt_scan_root, NULL); of_scan_flat_dt(early_init_dt_scan_memory_ppc, NULL); @@ -718,6 +734,8 @@ void __init early_init_devtree(void *params) DBG("Phys. mem: %llx\n", lmb_phys_mem_size()); + set_boot_memory_limit(); + /* We may need to relocate the flat tree, do it now. * FIXME .. and the initrd too? */ move_device_tree(); diff --git a/arch/powerpc/kernel/setup_32.c b/arch/powerpc/kernel/setup_32.c index 8f58986..fd3339c 100644 --- a/arch/powerpc/kernel/setup_32.c +++ b/arch/powerpc/kernel/setup_32.c @@ -247,7 +247,7 @@ static void __init irqstack_early_init(void) unsigned int i; /* interrupt stacks must be in lowmem, we get that for free on ppc32 - * as the lmb is limited to lowmem by LMB_REAL_LIMIT */ + * as the lmb is limited to lowmem by default */ for_each_possible_cpu(i) { softirq_ctx[i] = (struct thread_info *) __va(lmb_alloc(THREAD_SIZE, THREAD_SIZE)); diff --git a/arch/powerpc/mm/40x_mmu.c b/arch/powerpc/mm/40x_mmu.c index 65abfcf..809f655 100644 --- a/arch/powerpc/mm/40x_mmu.c +++ b/arch/powerpc/mm/40x_mmu.c @@ -35,6 +35,7 @@ #include <linux/init.h> #include <linux/delay.h> #include <linux/highmem.h> +#include <linux/lmb.h> #include <asm/pgalloc.h> #include <asm/prom.h> @@ -47,6 +48,7 @@ #include <asm/bootx.h> #include <asm/machdep.h> #include <asm/setup.h> + #include "mmu_decl.h" extern int __map_without_ltlbs; @@ -139,8 +141,7 @@ unsigned long __init mmu_mapin_ram(unsigned long top) * coverage with normal-sized pages (or other reasons) do not * attempt to allocate outside the allowed range. */ - - __initial_memory_limit_addr = memstart_addr + mapped; + lmb_set_current_limit(memstart_addr + mapped); return mapped; } diff --git a/arch/powerpc/mm/fsl_booke_mmu.c b/arch/powerpc/mm/fsl_booke_mmu.c index 1ed6b52..038cb29 100644 --- a/arch/powerpc/mm/fsl_booke_mmu.c +++ b/arch/powerpc/mm/fsl_booke_mmu.c @@ -40,6 +40,7 @@ #include <linux/init.h> #include <linux/delay.h> #include <linux/highmem.h> +#include <linux/lmb.h> #include <asm/pgalloc.h> #include <asm/prom.h> @@ -231,5 +232,5 @@ void __init adjust_total_lowmem(void) pr_cont("%lu Mb, residual: %dMb\n", tlbcam_sz(tlbcam_index - 1) >> 20, (unsigned int)((total_lowmem - __max_low_memory) >> 20)); - __initial_memory_limit_addr = memstart_addr + __max_low_memory; + lmb_set_current_limit(memstart_addr + __max_low_memory); } diff --git a/arch/powerpc/mm/hash_utils_64.c b/arch/powerpc/mm/hash_utils_64.c index 28838e3..ae7a8f1 100644 --- a/arch/powerpc/mm/hash_utils_64.c +++ b/arch/powerpc/mm/hash_utils_64.c @@ -696,7 +696,8 @@ static void __init htab_initialize(void) #endif /* CONFIG_U3_DART */ BUG_ON(htab_bolt_mapping(base, base + size, __pa(base), prot, mmu_linear_psize, mmu_kernel_ssize)); - } + } + lmb_set_current_limit(LMB_ALLOC_ANYWHERE); /* * If we have a memory_limit and we've allocated TCEs then we need to diff --git a/arch/powerpc/mm/init_32.c b/arch/powerpc/mm/init_32.c index 7673330..62d2242 100644 --- a/arch/powerpc/mm/init_32.c +++ b/arch/powerpc/mm/init_32.c @@ -92,12 +92,6 @@ int __allow_ioremap_reserved; unsigned long __max_low_memory = MAX_LOW_MEM; /* - * address of the limit of what is accessible with initial MMU setup - - * 256MB usually, but only 16MB on 601. - */ -phys_addr_t __initial_memory_limit_addr = (phys_addr_t)0x10000000; - -/* * Check for command-line options that affect what MMU_init will do. */ void MMU_setup(void) @@ -126,13 +120,6 @@ void __init MMU_init(void) if (ppc_md.progress) ppc_md.progress("MMU:enter", 0x111); - /* 601 can only access 16MB at the moment */ - if (PVR_VER(mfspr(SPRN_PVR)) == 1) - __initial_memory_limit_addr = 0x01000000; - /* 8xx can only access 8MB at the moment */ - if (PVR_VER(mfspr(SPRN_PVR)) == 0x50) - __initial_memory_limit_addr = 0x00800000; - /* parse args from command line */ MMU_setup(); @@ -190,20 +177,18 @@ void __init MMU_init(void) #ifdef CONFIG_BOOTX_TEXT btext_unmap(); #endif + + /* Shortly after that, the entire linear mapping will be available */ + lmb_set_current_limit(lowmem_end_addr); } /* This is only called until mem_init is done. */ void __init *early_get_page(void) { - void *p; - - if (init_bootmem_done) { - p = alloc_bootmem_pages(PAGE_SIZE); - } else { - p = __va(lmb_alloc_base(PAGE_SIZE, PAGE_SIZE, - __initial_memory_limit_addr)); - } - return p; + if (init_bootmem_done) + return alloc_bootmem_pages(PAGE_SIZE); + else + return __va(lmb_alloc(PAGE_SIZE, PAGE_SIZE)); } /* Free up now-unused memory */ diff --git a/arch/powerpc/mm/ppc_mmu_32.c b/arch/powerpc/mm/ppc_mmu_32.c index f11c2cd..fe6af92 100644 --- a/arch/powerpc/mm/ppc_mmu_32.c +++ b/arch/powerpc/mm/ppc_mmu_32.c @@ -223,8 +223,7 @@ void __init MMU_init_hw(void) * Find some memory for the hash table. */ if ( ppc_md.progress ) ppc_md.progress("hash:find piece", 0x322); - Hash = __va(lmb_alloc_base(Hash_size, Hash_size, - __initial_memory_limit_addr)); + Hash = __va(lmb_alloc(Hash_size, Hash_size)); cacheable_memzero(Hash, Hash_size); _SDR1 = __pa(Hash) | SDR1_LOW_BITS; diff --git a/arch/powerpc/mm/tlb_nohash.c b/arch/powerpc/mm/tlb_nohash.c index e81d5d6..4a09475 100644 --- a/arch/powerpc/mm/tlb_nohash.c +++ b/arch/powerpc/mm/tlb_nohash.c @@ -432,6 +432,8 @@ static void __early_init_mmu(int boot_cpu) * the MMU configuration */ mb(); + + lmb_set_current_limit(linear_map_top); } void __init early_init_mmu(void) diff --git a/arch/sh/include/asm/lmb.h b/arch/sh/include/asm/lmb.h index 9b437f6..8477be2 100644 --- a/arch/sh/include/asm/lmb.h +++ b/arch/sh/include/asm/lmb.h @@ -1,6 +1,4 @@ #ifndef __ASM_SH_LMB_H #define __ASM_SH_LMB_H -#define LMB_REAL_LIMIT 0 - #endif /* __ASM_SH_LMB_H */ diff --git a/arch/sparc/include/asm/lmb.h b/arch/sparc/include/asm/lmb.h index 6a352cb..2275165 100644 --- a/arch/sparc/include/asm/lmb.h +++ b/arch/sparc/include/asm/lmb.h @@ -5,6 +5,4 @@ #define LMB_DBG(fmt...) prom_printf(fmt) -#define LMB_REAL_LIMIT 0 - #endif /* !(_SPARC64_LMB_H) */ diff --git a/include/linux/lmb.h b/include/linux/lmb.h index f0d2cab..3b950c3 100644 --- a/include/linux/lmb.h +++ b/include/linux/lmb.h @@ -34,6 +34,7 @@ struct lmb_type { struct lmb { unsigned long debug; u64 rmo_size; + u64 current_limit; struct lmb_type memory; struct lmb_type reserved; }; @@ -46,11 +47,16 @@ extern long lmb_add(u64 base, u64 size); extern long lmb_remove(u64 base, u64 size); extern long __init lmb_free(u64 base, u64 size); extern long __init lmb_reserve(u64 base, u64 size); + extern u64 __init lmb_alloc_nid(u64 size, u64 align, int nid); extern u64 __init lmb_alloc(u64 size, u64 align); + +/* Flags for lmb_alloc_base() amd __lmb_alloc_base() */ +#define LMB_ALLOC_ANYWHERE (~(u64)0) +#define LMB_ALLOC_ACCESSIBLE 0 + extern u64 __init lmb_alloc_base(u64 size, u64, u64 max_addr); -#define LMB_ALLOC_ANYWHERE 0 extern u64 __init __lmb_alloc_base(u64 size, u64 align, u64 max_addr); extern u64 __init lmb_phys_mem_size(void); @@ -64,6 +70,14 @@ extern void lmb_dump_all(void); /* Provided by the architecture */ extern u64 lmb_nid_range(u64 start, u64 end, int *nid); +/** + * lmb_set_current_limit - Set the current allocation limit to allow + * limiting allocations to what is currently + * accessible during boot + * @limit: New limit value (physical address) + */ +extern void lmb_set_current_limit(u64 limit); + /* * pfn conversion functions diff --git a/lib/lmb.c b/lib/lmb.c index bd81266..34a558c 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -115,6 +115,8 @@ void __init lmb_init(void) lmb.reserved.regions[0].base = 0; lmb.reserved.regions[0].size = 0; lmb.reserved.cnt = 1; + + lmb.current_limit = LMB_ALLOC_ANYWHERE; } void __init lmb_analyze(void) @@ -373,7 +375,7 @@ u64 __init lmb_alloc_nid(u64 size, u64 align, int nid) u64 __init lmb_alloc(u64 size, u64 align) { - return lmb_alloc_base(size, align, LMB_ALLOC_ANYWHERE); + return lmb_alloc_base(size, align, LMB_ALLOC_ACCESSIBLE); } u64 __init lmb_alloc_base(u64 size, u64 align, u64 max_addr) @@ -399,14 +401,9 @@ u64 __init __lmb_alloc_base(u64 size, u64 align, u64 max_addr) size = lmb_align_up(size, align); - /* On some platforms, make sure we allocate lowmem */ - /* Note that LMB_REAL_LIMIT may be LMB_ALLOC_ANYWHERE */ - if (max_addr == LMB_ALLOC_ANYWHERE) - max_addr = LMB_REAL_LIMIT; - /* Pump up max_addr */ - if (max_addr == LMB_ALLOC_ANYWHERE) - max_addr = ~(u64)0; + if (max_addr == LMB_ALLOC_ACCESSIBLE) + max_addr = lmb.current_limit; /* We do a top-down search, this tends to limit memory * fragmentation by keeping early boot allocs near the @@ -501,3 +498,9 @@ int lmb_is_region_reserved(u64 base, u64 size) return lmb_overlaps_region(&lmb.reserved, base, size); } + +void __init lmb_set_current_limit(u64 limit) +{ + lmb.current_limit = limit; +} + -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 08/25] lmb: Remove rmo_size, burry it in arch/powerpc where it belongs 2010-05-10 9:38 ` [PATCH 07/25] lmb: Introduce default allocation limit and use it to replace explicit ones Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 09/25] lmb: Change u64 to phys_addr_t Benjamin Herrenschmidt 0 siblings, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt The RMA (RMO is a misnomer) is a concept specific to ppc64 (in fact server ppc64 though I hijack it on embedded ppc64 for similar purposes) and represents the area of memory that can be accessed in real mode (aka with MMU off), or on embedded, from the exception vectors (which is bolted in the TLB) which pretty much boils down to the same thing. We take that out of the generic LMB data structure and move it into arch/powerpc where it belongs, renaming it to "RMA" while at it. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- arch/powerpc/include/asm/mmu.h | 12 ++++++++++++ arch/powerpc/kernel/head_40x.S | 6 +----- arch/powerpc/kernel/paca.c | 2 +- arch/powerpc/kernel/prom.c | 29 ++++++++--------------------- arch/powerpc/kernel/rtas.c | 2 +- arch/powerpc/kernel/setup_64.c | 2 +- arch/powerpc/mm/40x_mmu.c | 14 +++++++++++++- arch/powerpc/mm/44x_mmu.c | 14 ++++++++++++++ arch/powerpc/mm/fsl_booke_mmu.c | 9 +++++++++ arch/powerpc/mm/hash_utils_64.c | 22 +++++++++++++++++++++- arch/powerpc/mm/init_32.c | 14 ++++++++++++++ arch/powerpc/mm/init_64.c | 1 + arch/powerpc/mm/ppc_mmu_32.c | 15 +++++++++++++++ arch/powerpc/mm/tlb_nohash.c | 14 ++++++++++++++ include/linux/lmb.h | 1 - lib/lmb.c | 8 -------- 16 files changed, 125 insertions(+), 40 deletions(-) diff --git a/arch/powerpc/include/asm/mmu.h b/arch/powerpc/include/asm/mmu.h index 7ebf42e..bc68624 100644 --- a/arch/powerpc/include/asm/mmu.h +++ b/arch/powerpc/include/asm/mmu.h @@ -2,6 +2,8 @@ #define _ASM_POWERPC_MMU_H_ #ifdef __KERNEL__ +#include <linux/types.h> + #include <asm/asm-compat.h> #include <asm/feature-fixups.h> @@ -82,6 +84,16 @@ extern unsigned int __start___mmu_ftr_fixup, __stop___mmu_ftr_fixup; extern void early_init_mmu(void); extern void early_init_mmu_secondary(void); +extern void setup_initial_memory_limit(phys_addr_t first_lmb_base, + phys_addr_t first_lmb_size); + +#ifdef CONFIG_PPC64 +/* This is our real memory area size on ppc64 server, on embedded, we + * make it match the size our of bolted TLB area + */ +extern u64 ppc64_rma_size; +#endif /* CONFIG_PPC64 */ + #endif /* !__ASSEMBLY__ */ /* The kernel use the constants below to index in the page sizes array. diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index a90625f..8278e8b 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -923,11 +923,7 @@ initial_mmu: mtspr SPRN_PID,r0 sync - /* Configure and load two entries into TLB slots 62 and 63. - * In case we are pinning TLBs, these are reserved in by the - * other TLB functions. If not reserving, then it doesn't - * matter where they are loaded. - */ + /* Configure and load one entry into TLB slots 63 */ clrrwi r4,r4,10 /* Mask off the real page number */ ori r4,r4,(TLB_WR | TLB_EX) /* Set the write and execute bits */ diff --git a/arch/powerpc/kernel/paca.c b/arch/powerpc/kernel/paca.c index 0c40c6f..717185b 100644 --- a/arch/powerpc/kernel/paca.c +++ b/arch/powerpc/kernel/paca.c @@ -115,7 +115,7 @@ void __init allocate_pacas(void) * the first segment. On iSeries they must be within the area mapped * by the HV, which is HvPagesToMap * HVPAGESIZE bytes. */ - limit = min(0x10000000ULL, lmb.rmo_size); + limit = min(0x10000000ULL, ppc64_rma_size); if (firmware_has_feature(FW_FEATURE_ISERIES)) limit = min(limit, HvPagesToMap * HVPAGESIZE); diff --git a/arch/powerpc/kernel/prom.c b/arch/powerpc/kernel/prom.c index b8428d3..7bec9ac 100644 --- a/arch/powerpc/kernel/prom.c +++ b/arch/powerpc/kernel/prom.c @@ -66,6 +66,7 @@ int __initdata iommu_is_off; int __initdata iommu_force_on; unsigned long tce_alloc_start, tce_alloc_end; +u64 ppc64_rma_size; #endif static int __init early_parse_mem(char *p) @@ -492,7 +493,7 @@ static int __init early_init_dt_scan_memory_ppc(unsigned long node, void __init early_init_dt_add_memory_arch(u64 base, u64 size) { -#if defined(CONFIG_PPC64) +#ifdef CONFIG_PPC64 if (iommu_is_off) { if (base >= 0x80000000ul) return; @@ -501,9 +502,13 @@ void __init early_init_dt_add_memory_arch(u64 base, u64 size) } #endif - lmb_add(base, size); - + /* First LMB added, do some special initializations */ + if (memstart_addr == ~(phys_addr_t)0) + setup_initial_memory_limit(base, size); memstart_addr = min((u64)memstart_addr, base); + + /* Add the chunk to the LMB list */ + lmb_add(base, size); } u64 __init early_init_dt_alloc_memory_arch(u64 size, u64 align) @@ -655,22 +660,6 @@ static void __init phyp_dump_reserve_mem(void) static inline void __init phyp_dump_reserve_mem(void) {} #endif /* CONFIG_PHYP_DUMP && CONFIG_PPC_RTAS */ -static void set_boot_memory_limit(void) -{ -#ifdef CONFIG_PPC32 - /* 601 can only access 16MB at the moment */ - if (PVR_VER(mfspr(SPRN_PVR)) == 1) - lmb_set_current_limit(0x01000000); - /* 8xx can only access 8MB at the moment */ - else if (PVR_VER(mfspr(SPRN_PVR)) == 0x50) - lmb_set_current_limit(0x00800000); - else - lmb_set_current_limit(0x10000000); -#else - lmb_set_current_limit(lmb.rmo_size); -#endif -} - void __init early_init_devtree(void *params) { phys_addr_t limit; @@ -734,8 +723,6 @@ void __init early_init_devtree(void *params) DBG("Phys. mem: %llx\n", lmb_phys_mem_size()); - set_boot_memory_limit(); - /* We may need to relocate the flat tree, do it now. * FIXME .. and the initrd too? */ move_device_tree(); diff --git a/arch/powerpc/kernel/rtas.c b/arch/powerpc/kernel/rtas.c index 0e1ec6f..fbe8fe7 100644 --- a/arch/powerpc/kernel/rtas.c +++ b/arch/powerpc/kernel/rtas.c @@ -934,7 +934,7 @@ void __init rtas_initialize(void) */ #ifdef CONFIG_PPC64 if (machine_is(pseries) && firmware_has_feature(FW_FEATURE_LPAR)) { - rtas_region = min(lmb.rmo_size, RTAS_INSTANTIATE_MAX); + rtas_region = min(ppc64_rma_size, RTAS_INSTANTIATE_MAX); ibm_suspend_me_token = rtas_token("ibm,suspend-me"); } #endif diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c index 9143891..9690d74 100644 --- a/arch/powerpc/kernel/setup_64.c +++ b/arch/powerpc/kernel/setup_64.c @@ -482,7 +482,7 @@ static void __init emergency_stack_init(void) * bringup, we need to get at them in real mode. This means they * must also be within the RMO region. */ - limit = min(0x10000000ULL, lmb.rmo_size); + limit = min(0x10000000ULL, ppc64_rma_size); for_each_possible_cpu(i) { unsigned long sp; diff --git a/arch/powerpc/mm/40x_mmu.c b/arch/powerpc/mm/40x_mmu.c index 809f655..475932f 100644 --- a/arch/powerpc/mm/40x_mmu.c +++ b/arch/powerpc/mm/40x_mmu.c @@ -141,7 +141,19 @@ unsigned long __init mmu_mapin_ram(unsigned long top) * coverage with normal-sized pages (or other reasons) do not * attempt to allocate outside the allowed range. */ - lmb_set_current_limit(memstart_addr + mapped); + lmb_set_current_limit(mapped); return mapped; } + +void setup_initial_memory_limit(phys_addr_t first_lmb_base, + phys_addr_t first_lmb_size) +{ + /* We don't currently support the first LMB not mapping 0 + * physical on those processors + */ + BUG_ON(first_lmb_base != 0); + + /* 40x can only access 16MB at the moment (see head_40x.S) */ + lmb_set_current_limit(min_t(u64, first_lmb_size, 0x00800000)); +} diff --git a/arch/powerpc/mm/44x_mmu.c b/arch/powerpc/mm/44x_mmu.c index d8c6efb..a2d3aa9 100644 --- a/arch/powerpc/mm/44x_mmu.c +++ b/arch/powerpc/mm/44x_mmu.c @@ -24,6 +24,8 @@ */ #include <linux/init.h> +#include <linux/lmb.h> + #include <asm/mmu.h> #include <asm/system.h> #include <asm/page.h> @@ -213,6 +215,18 @@ unsigned long __init mmu_mapin_ram(unsigned long top) return total_lowmem; } +void setup_initial_memory_limit(phys_addr_t first_lmb_base, + phys_addr_t first_lmb_size) +{ + /* We don't currently support the first LMB not mapping 0 + * physical on those processors + */ + BUG_ON(first_lmb_base != 0); + + /* 44x has a 256M TLB entry pinned at boot */ + lmb_set_current_limit(min_t(u64, first_lmb_size, PPC_PIN_SIZE)); +} + #ifdef CONFIG_SMP void __cpuinit mmu_init_secondary(int cpu) { diff --git a/arch/powerpc/mm/fsl_booke_mmu.c b/arch/powerpc/mm/fsl_booke_mmu.c index 038cb29..e652560 100644 --- a/arch/powerpc/mm/fsl_booke_mmu.c +++ b/arch/powerpc/mm/fsl_booke_mmu.c @@ -234,3 +234,12 @@ void __init adjust_total_lowmem(void) lmb_set_current_limit(memstart_addr + __max_low_memory); } + +void setup_initial_memory_limit(phys_addr_t first_lmb_base, + phys_addr_t first_lmb_size) +{ + phys_addr_t limit = first_lmb_base + first_lmb_size; + + /* 64M mapped initially according to head_fsl_booke.S */ + lmb_set_current_limit(min_t(u64, limit, 0x04000000)); +} diff --git a/arch/powerpc/mm/hash_utils_64.c b/arch/powerpc/mm/hash_utils_64.c index ae7a8f1..2b748c5 100644 --- a/arch/powerpc/mm/hash_utils_64.c +++ b/arch/powerpc/mm/hash_utils_64.c @@ -649,7 +649,7 @@ static void __init htab_initialize(void) #ifdef CONFIG_DEBUG_PAGEALLOC linear_map_hash_count = lmb_end_of_DRAM() >> PAGE_SHIFT; linear_map_hash_slots = __va(lmb_alloc_base(linear_map_hash_count, - 1, lmb.rmo_size)); + 1, ppc64_rma_size)); memset(linear_map_hash_slots, 0, linear_map_hash_count); #endif /* CONFIG_DEBUG_PAGEALLOC */ @@ -1221,3 +1221,23 @@ void kernel_map_pages(struct page *page, int numpages, int enable) local_irq_restore(flags); } #endif /* CONFIG_DEBUG_PAGEALLOC */ + +void setup_initial_memory_limit(phys_addr_t first_lmb_base, + phys_addr_t first_lmb_size) +{ + /* We don't currently support the first LMB not mapping 0 + * physical on those processors + */ + BUG_ON(first_lmb_base != 0); + + /* On LPAR systems, the first entry is our RMA region, + * non-LPAR 64-bit hash MMU systems don't have a limitation + * on real mode access, but using the first entry works well + * enough. We also clamp it to 1G to avoid some funky things + * such as RTAS bugs etc... + */ + ppc64_rma_size = min_t(u64, first_lmb_size, 0x40000000); + + /* Finally limit subsequent allocations */ + lmb_set_current_limit(ppc64_rma_size); +} diff --git a/arch/powerpc/mm/init_32.c b/arch/powerpc/mm/init_32.c index 62d2242..218869a 100644 --- a/arch/powerpc/mm/init_32.c +++ b/arch/powerpc/mm/init_32.c @@ -237,3 +237,17 @@ void free_initrd_mem(unsigned long start, unsigned long end) } #endif + +#ifdef CONFIG_8xx /* No 8xx specific .c file to put that in ... */ +void setup_initial_memory_limit(phys_addr_t first_lmb_base, + phys_addr_t first_lmb_size) +{ + /* We don't currently support the first LMB not mapping 0 + * physical on those processors + */ + BUG_ON(first_lmb_base != 0); + + /* 8xx can only access 8MB at the moment */ + lmb_set_current_limit(min_t(u64, first_lmb_size, 0x00800000)); +} +#endif /* CONFIG_8xx */ diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c index e267f22..433922a 100644 --- a/arch/powerpc/mm/init_64.c +++ b/arch/powerpc/mm/init_64.c @@ -328,3 +328,4 @@ int __meminit vmemmap_populate(struct page *start_page, return 0; } #endif /* CONFIG_SPARSEMEM_VMEMMAP */ + diff --git a/arch/powerpc/mm/ppc_mmu_32.c b/arch/powerpc/mm/ppc_mmu_32.c index fe6af92..ad8a36c 100644 --- a/arch/powerpc/mm/ppc_mmu_32.c +++ b/arch/powerpc/mm/ppc_mmu_32.c @@ -271,3 +271,18 @@ void __init MMU_init_hw(void) if ( ppc_md.progress ) ppc_md.progress("hash:done", 0x205); } + +void setup_initial_memory_limit(phys_addr_t first_lmb_base, + phys_addr_t first_lmb_size) +{ + /* We don't currently support the first LMB not mapping 0 + * physical on those processors + */ + BUG_ON(first_lmb_base != 0); + + /* 601 can only access 16MB at the moment */ + if (PVR_VER(mfspr(SPRN_PVR)) == 1) + lmb_set_current_limit(min_t(u64, first_lmb_size, 0x01000000)); + else /* Anything else has 256M mapped */ + lmb_set_current_limit(min_t(u64, first_lmb_size, 0x10000000)); +} diff --git a/arch/powerpc/mm/tlb_nohash.c b/arch/powerpc/mm/tlb_nohash.c index 4a09475..a771e62 100644 --- a/arch/powerpc/mm/tlb_nohash.c +++ b/arch/powerpc/mm/tlb_nohash.c @@ -446,4 +446,18 @@ void __cpuinit early_init_mmu_secondary(void) __early_init_mmu(0); } +void setup_initial_memory_limit(phys_addr_t first_lmb_base, + phys_addr_t first_lmb_size) +{ + /* On Embedded 64-bit, we adjust the RMA size to match + * the bolted TLB entry. We know for now that only 1G + * entries are supported though that may eventually + * change. We crop it to the size of the first LMB to + * avoid going over total available memory just in case... + */ + ppc64_rma_size = min_t(u64, first_lmb_size, 0x40000000); + + /* Finally limit subsequent allocations */ + lmb_set_current_limit(ppc64_lmb_base + ppc64_rma_size); +} #endif /* CONFIG_PPC64 */ diff --git a/include/linux/lmb.h b/include/linux/lmb.h index 3b950c3..6912ae2 100644 --- a/include/linux/lmb.h +++ b/include/linux/lmb.h @@ -33,7 +33,6 @@ struct lmb_type { struct lmb { unsigned long debug; - u64 rmo_size; u64 current_limit; struct lmb_type memory; struct lmb_type reserved; diff --git a/lib/lmb.c b/lib/lmb.c index 34a558c..e7a7842 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -49,7 +49,6 @@ void lmb_dump_all(void) return; pr_info("LMB configuration:\n"); - pr_info(" rmo_size = 0x%llx\n", (unsigned long long)lmb.rmo_size); pr_info(" memory.size = 0x%llx\n", (unsigned long long)lmb.memory.size); lmb_dump(&lmb.memory, "memory"); @@ -195,10 +194,6 @@ static long lmb_add_region(struct lmb_type *type, u64 base, u64 size) long lmb_add(u64 base, u64 size) { - /* On pSeries LPAR systems, the first LMB is our RMO region. */ - if (base == 0) - lmb.rmo_size = size; - return lmb_add_region(&lmb.memory, base, size); } @@ -459,9 +454,6 @@ void __init lmb_enforce_memory_limit(u64 memory_limit) break; } - if (lmb.memory.regions[0].size < lmb.rmo_size) - lmb.rmo_size = lmb.memory.regions[0].size; - memory_limit = lmb_end_of_DRAM(); /* And truncate any reserves above the limit also. */ -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 09/25] lmb: Change u64 to phys_addr_t 2010-05-10 9:38 ` [PATCH 08/25] lmb: Remove rmo_size, burry it in arch/powerpc where it belongs Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 10/25] lmb: Remove unused lmb.debug struct member Benjamin Herrenschmidt 0 siblings, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt Let's not waste space and cycles on archs that don't support >32-bit physical address space. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- include/linux/lmb.h | 44 ++++++++++---------- lib/lmb.c | 114 ++++++++++++++++++++++++++------------------------- 2 files changed, 80 insertions(+), 78 deletions(-) diff --git a/include/linux/lmb.h b/include/linux/lmb.h index 6912ae2..042250c 100644 --- a/include/linux/lmb.h +++ b/include/linux/lmb.h @@ -21,19 +21,19 @@ #define MAX_LMB_REGIONS 128 struct lmb_region { - u64 base; - u64 size; + phys_addr_t base; + phys_addr_t size; }; struct lmb_type { unsigned long cnt; - u64 size; + phys_addr_t size; struct lmb_region regions[MAX_LMB_REGIONS+1]; }; struct lmb { unsigned long debug; - u64 current_limit; + phys_addr_t current_limit; struct lmb_type memory; struct lmb_type reserved; }; @@ -42,32 +42,32 @@ extern struct lmb lmb; extern void __init lmb_init(void); extern void __init lmb_analyze(void); -extern long lmb_add(u64 base, u64 size); -extern long lmb_remove(u64 base, u64 size); -extern long __init lmb_free(u64 base, u64 size); -extern long __init lmb_reserve(u64 base, u64 size); +extern long lmb_add(phys_addr_t base, phys_addr_t size); +extern long lmb_remove(phys_addr_t base, phys_addr_t size); +extern long __init lmb_free(phys_addr_t base, phys_addr_t size); +extern long __init lmb_reserve(phys_addr_t base, phys_addr_t size); -extern u64 __init lmb_alloc_nid(u64 size, u64 align, int nid); -extern u64 __init lmb_alloc(u64 size, u64 align); +extern phys_addr_t __init lmb_alloc_nid(phys_addr_t size, phys_addr_t align, int nid); +extern phys_addr_t __init lmb_alloc(phys_addr_t size, phys_addr_t align); /* Flags for lmb_alloc_base() amd __lmb_alloc_base() */ -#define LMB_ALLOC_ANYWHERE (~(u64)0) +#define LMB_ALLOC_ANYWHERE (~(phys_addr_t)0) #define LMB_ALLOC_ACCESSIBLE 0 -extern u64 __init lmb_alloc_base(u64 size, - u64, u64 max_addr); -extern u64 __init __lmb_alloc_base(u64 size, - u64 align, u64 max_addr); -extern u64 __init lmb_phys_mem_size(void); -extern u64 lmb_end_of_DRAM(void); -extern void __init lmb_enforce_memory_limit(u64 memory_limit); -extern int __init lmb_is_reserved(u64 addr); -extern int lmb_is_region_reserved(u64 base, u64 size); +extern phys_addr_t __init lmb_alloc_base(phys_addr_t size, + phys_addr_t, phys_addr_t max_addr); +extern phys_addr_t __init __lmb_alloc_base(phys_addr_t size, + phys_addr_t align, phys_addr_t max_addr); +extern phys_addr_t __init lmb_phys_mem_size(void); +extern phys_addr_t lmb_end_of_DRAM(void); +extern void __init lmb_enforce_memory_limit(phys_addr_t memory_limit); +extern int __init lmb_is_reserved(phys_addr_t addr); +extern int lmb_is_region_reserved(phys_addr_t base, phys_addr_t size); extern void lmb_dump_all(void); /* Provided by the architecture */ -extern u64 lmb_nid_range(u64 start, u64 end, int *nid); +extern phys_addr_t lmb_nid_range(phys_addr_t start, phys_addr_t end, int *nid); /** * lmb_set_current_limit - Set the current allocation limit to allow @@ -75,7 +75,7 @@ extern u64 lmb_nid_range(u64 start, u64 end, int *nid); * accessible during boot * @limit: New limit value (physical address) */ -extern void lmb_set_current_limit(u64 limit); +extern void lmb_set_current_limit(phys_addr_t limit); /* diff --git a/lib/lmb.c b/lib/lmb.c index e7a7842..2995673 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -55,13 +55,14 @@ void lmb_dump_all(void) lmb_dump(&lmb.reserved, "reserved"); } -static unsigned long lmb_addrs_overlap(u64 base1, u64 size1, u64 base2, - u64 size2) +static unsigned long lmb_addrs_overlap(phys_addr_t base1, phys_addr_t size1, + phys_addr_t base2, phys_addr_t size2) { return ((base1 < (base2 + size2)) && (base2 < (base1 + size1))); } -static long lmb_addrs_adjacent(u64 base1, u64 size1, u64 base2, u64 size2) +static long lmb_addrs_adjacent(phys_addr_t base1, phys_addr_t size1, + phys_addr_t base2, phys_addr_t size2) { if (base2 == base1 + size1) return 1; @@ -72,12 +73,12 @@ static long lmb_addrs_adjacent(u64 base1, u64 size1, u64 base2, u64 size2) } static long lmb_regions_adjacent(struct lmb_type *type, - unsigned long r1, unsigned long r2) + unsigned long r1, unsigned long r2) { - u64 base1 = type->regions[r1].base; - u64 size1 = type->regions[r1].size; - u64 base2 = type->regions[r2].base; - u64 size2 = type->regions[r2].size; + phys_addr_t base1 = type->regions[r1].base; + phys_addr_t size1 = type->regions[r1].size; + phys_addr_t base2 = type->regions[r2].base; + phys_addr_t size2 = type->regions[r2].size; return lmb_addrs_adjacent(base1, size1, base2, size2); } @@ -128,7 +129,7 @@ void __init lmb_analyze(void) lmb.memory.size += lmb.memory.regions[i].size; } -static long lmb_add_region(struct lmb_type *type, u64 base, u64 size) +static long lmb_add_region(struct lmb_type *type, phys_addr_t base, phys_addr_t size) { unsigned long coalesced = 0; long adjacent, i; @@ -141,8 +142,8 @@ static long lmb_add_region(struct lmb_type *type, u64 base, u64 size) /* First try and coalesce this LMB with another. */ for (i = 0; i < type->cnt; i++) { - u64 rgnbase = type->regions[i].base; - u64 rgnsize = type->regions[i].size; + phys_addr_t rgnbase = type->regions[i].base; + phys_addr_t rgnsize = type->regions[i].size; if ((rgnbase == base) && (rgnsize == size)) /* Already have this region, so we're done */ @@ -192,16 +193,16 @@ static long lmb_add_region(struct lmb_type *type, u64 base, u64 size) return 0; } -long lmb_add(u64 base, u64 size) +long lmb_add(phys_addr_t base, phys_addr_t size) { return lmb_add_region(&lmb.memory, base, size); } -static long __lmb_remove(struct lmb_type *type, u64 base, u64 size) +static long __lmb_remove(struct lmb_type *type, phys_addr_t base, phys_addr_t size) { - u64 rgnbegin, rgnend; - u64 end = base + size; + phys_addr_t rgnbegin, rgnend; + phys_addr_t end = base + size; int i; rgnbegin = rgnend = 0; /* supress gcc warnings */ @@ -246,17 +247,17 @@ static long __lmb_remove(struct lmb_type *type, u64 base, u64 size) return lmb_add_region(type, end, rgnend - end); } -long lmb_remove(u64 base, u64 size) +long lmb_remove(phys_addr_t base, phys_addr_t size) { return __lmb_remove(&lmb.memory, base, size); } -long __init lmb_free(u64 base, u64 size) +long __init lmb_free(phys_addr_t base, phys_addr_t size) { return __lmb_remove(&lmb.reserved, base, size); } -long __init lmb_reserve(u64 base, u64 size) +long __init lmb_reserve(phys_addr_t base, phys_addr_t size) { struct lmb_type *_rgn = &lmb.reserved; @@ -265,13 +266,13 @@ long __init lmb_reserve(u64 base, u64 size) return lmb_add_region(_rgn, base, size); } -long lmb_overlaps_region(struct lmb_type *type, u64 base, u64 size) +long lmb_overlaps_region(struct lmb_type *type, phys_addr_t base, phys_addr_t size) { unsigned long i; for (i = 0; i < type->cnt; i++) { - u64 rgnbase = type->regions[i].base; - u64 rgnsize = type->regions[i].size; + phys_addr_t rgnbase = type->regions[i].base; + phys_addr_t rgnsize = type->regions[i].size; if (lmb_addrs_overlap(base, size, rgnbase, rgnsize)) break; } @@ -279,20 +280,20 @@ long lmb_overlaps_region(struct lmb_type *type, u64 base, u64 size) return (i < type->cnt) ? i : -1; } -static u64 lmb_align_down(u64 addr, u64 size) +static phys_addr_t lmb_align_down(phys_addr_t addr, phys_addr_t size) { return addr & ~(size - 1); } -static u64 lmb_align_up(u64 addr, u64 size) +static phys_addr_t lmb_align_up(phys_addr_t addr, phys_addr_t size) { return (addr + (size - 1)) & ~(size - 1); } -static u64 __init lmb_alloc_region(u64 start, u64 end, - u64 size, u64 align) +static phys_addr_t __init lmb_alloc_region(phys_addr_t start, phys_addr_t end, + phys_addr_t size, phys_addr_t align) { - u64 base, res_base; + phys_addr_t base, res_base; long j; base = lmb_align_down((end - size), align); @@ -301,7 +302,7 @@ static u64 __init lmb_alloc_region(u64 start, u64 end, if (j < 0) { /* this area isn't reserved, take it */ if (lmb_add_region(&lmb.reserved, base, size) < 0) - base = ~(u64)0; + base = ~(phys_addr_t)0; return base; } res_base = lmb.reserved.regions[j].base; @@ -310,42 +311,43 @@ static u64 __init lmb_alloc_region(u64 start, u64 end, base = lmb_align_down(res_base - size, align); } - return ~(u64)0; + return ~(phys_addr_t)0; } -u64 __weak __init lmb_nid_range(u64 start, u64 end, int *nid) +phys_addr_t __weak __init lmb_nid_range(phys_addr_t start, phys_addr_t end, int *nid) { *nid = 0; return end; } -static u64 __init lmb_alloc_nid_region(struct lmb_region *mp, - u64 size, u64 align, int nid) +static phys_addr_t __init lmb_alloc_nid_region(struct lmb_region *mp, + phys_addr_t size, + phys_addr_t align, int nid) { - u64 start, end; + phys_addr_t start, end; start = mp->base; end = start + mp->size; start = lmb_align_up(start, align); while (start < end) { - u64 this_end; + phys_addr_t this_end; int this_nid; this_end = lmb_nid_range(start, end, &this_nid); if (this_nid == nid) { - u64 ret = lmb_alloc_region(start, this_end, size, align); - if (ret != ~(u64)0) + phys_addr_t ret = lmb_alloc_region(start, this_end, size, align); + if (ret != ~(phys_addr_t)0) return ret; } start = this_end; } - return ~(u64)0; + return ~(phys_addr_t)0; } -u64 __init lmb_alloc_nid(u64 size, u64 align, int nid) +phys_addr_t __init lmb_alloc_nid(phys_addr_t size, phys_addr_t align, int nid) { struct lmb_type *mem = &lmb.memory; int i; @@ -359,23 +361,23 @@ u64 __init lmb_alloc_nid(u64 size, u64 align, int nid) size = lmb_align_up(size, align); for (i = 0; i < mem->cnt; i++) { - u64 ret = lmb_alloc_nid_region(&mem->regions[i], + phys_addr_t ret = lmb_alloc_nid_region(&mem->regions[i], size, align, nid); - if (ret != ~(u64)0) + if (ret != ~(phys_addr_t)0) return ret; } return lmb_alloc(size, align); } -u64 __init lmb_alloc(u64 size, u64 align) +phys_addr_t __init lmb_alloc(phys_addr_t size, phys_addr_t align) { return lmb_alloc_base(size, align, LMB_ALLOC_ACCESSIBLE); } -u64 __init lmb_alloc_base(u64 size, u64 align, u64 max_addr) +phys_addr_t __init lmb_alloc_base(phys_addr_t size, phys_addr_t align, phys_addr_t max_addr) { - u64 alloc; + phys_addr_t alloc; alloc = __lmb_alloc_base(size, align, max_addr); @@ -386,11 +388,11 @@ u64 __init lmb_alloc_base(u64 size, u64 align, u64 max_addr) return alloc; } -u64 __init __lmb_alloc_base(u64 size, u64 align, u64 max_addr) +phys_addr_t __init __lmb_alloc_base(phys_addr_t size, phys_addr_t align, phys_addr_t max_addr) { long i; - u64 base = 0; - u64 res_base; + phys_addr_t base = 0; + phys_addr_t res_base; BUG_ON(0 == size); @@ -405,26 +407,26 @@ u64 __init __lmb_alloc_base(u64 size, u64 align, u64 max_addr) * top of memory */ for (i = lmb.memory.cnt - 1; i >= 0; i--) { - u64 lmbbase = lmb.memory.regions[i].base; - u64 lmbsize = lmb.memory.regions[i].size; + phys_addr_t lmbbase = lmb.memory.regions[i].base; + phys_addr_t lmbsize = lmb.memory.regions[i].size; if (lmbsize < size) continue; base = min(lmbbase + lmbsize, max_addr); res_base = lmb_alloc_region(lmbbase, base, size, align); - if (res_base != ~(u64)0) + if (res_base != ~(phys_addr_t)0) return res_base; } return 0; } /* You must call lmb_analyze() before this. */ -u64 __init lmb_phys_mem_size(void) +phys_addr_t __init lmb_phys_mem_size(void) { return lmb.memory.size; } -u64 lmb_end_of_DRAM(void) +phys_addr_t lmb_end_of_DRAM(void) { int idx = lmb.memory.cnt - 1; @@ -432,10 +434,10 @@ u64 lmb_end_of_DRAM(void) } /* You must call lmb_analyze() after this. */ -void __init lmb_enforce_memory_limit(u64 memory_limit) +void __init lmb_enforce_memory_limit(phys_addr_t memory_limit) { unsigned long i; - u64 limit; + phys_addr_t limit; struct lmb_region *p; if (!memory_limit) @@ -472,12 +474,12 @@ void __init lmb_enforce_memory_limit(u64 memory_limit) } } -int __init lmb_is_reserved(u64 addr) +int __init lmb_is_reserved(phys_addr_t addr) { int i; for (i = 0; i < lmb.reserved.cnt; i++) { - u64 upper = lmb.reserved.regions[i].base + + phys_addr_t upper = lmb.reserved.regions[i].base + lmb.reserved.regions[i].size - 1; if ((addr >= lmb.reserved.regions[i].base) && (addr <= upper)) return 1; @@ -485,13 +487,13 @@ int __init lmb_is_reserved(u64 addr) return 0; } -int lmb_is_region_reserved(u64 base, u64 size) +int lmb_is_region_reserved(phys_addr_t base, phys_addr_t size) { return lmb_overlaps_region(&lmb.reserved, base, size); } -void __init lmb_set_current_limit(u64 limit) +void __init lmb_set_current_limit(phys_addr_t limit) { lmb.current_limit = limit; } -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 10/25] lmb: Remove unused lmb.debug struct member 2010-05-10 9:38 ` [PATCH 09/25] lmb: Change u64 to phys_addr_t Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 11/25] lmb: Remove lmb_type.size and add lmb.memory_size instead Benjamin Herrenschmidt 0 siblings, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- include/linux/lmb.h | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/include/linux/lmb.h b/include/linux/lmb.h index 042250c..5fdd900 100644 --- a/include/linux/lmb.h +++ b/include/linux/lmb.h @@ -32,7 +32,6 @@ struct lmb_type { }; struct lmb { - unsigned long debug; phys_addr_t current_limit; struct lmb_type memory; struct lmb_type reserved; @@ -55,9 +54,11 @@ extern phys_addr_t __init lmb_alloc(phys_addr_t size, phys_addr_t align); #define LMB_ALLOC_ACCESSIBLE 0 extern phys_addr_t __init lmb_alloc_base(phys_addr_t size, - phys_addr_t, phys_addr_t max_addr); + phys_addr_t align, + phys_addr_t max_addr); extern phys_addr_t __init __lmb_alloc_base(phys_addr_t size, - phys_addr_t align, phys_addr_t max_addr); + phys_addr_t align, + phys_addr_t max_addr); extern phys_addr_t __init lmb_phys_mem_size(void); extern phys_addr_t lmb_end_of_DRAM(void); extern void __init lmb_enforce_memory_limit(phys_addr_t memory_limit); -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 11/25] lmb: Remove lmb_type.size and add lmb.memory_size instead 2010-05-10 9:38 ` [PATCH 10/25] lmb: Remove unused lmb.debug struct member Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 12/25] lmb: Move lmb arrays to static storage in lmb.c and make their size a variable Benjamin Herrenschmidt 0 siblings, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt Right now, both the "memory" and "reserved" lmb_type structures have a "size" member. It's unused in the later case, and represent the calculated memory size in the later case. This moves it out to the main lmb structure instead Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- arch/powerpc/mm/mem.c | 2 +- include/linux/lmb.h | 2 +- lib/lmb.c | 8 ++++---- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c index 17a8027..87e122e 100644 --- a/arch/powerpc/mm/mem.c +++ b/arch/powerpc/mm/mem.c @@ -301,7 +301,7 @@ void __init mem_init(void) swiotlb_init(1); #endif - num_physpages = lmb.memory.size >> PAGE_SHIFT; + num_physpages = lmb_phys_mem_size() >> PAGE_SHIFT; high_memory = (void *) __va(max_low_pfn * PAGE_SIZE); #ifdef CONFIG_NEED_MULTIPLE_NODES diff --git a/include/linux/lmb.h b/include/linux/lmb.h index 5fdd900..27c2386 100644 --- a/include/linux/lmb.h +++ b/include/linux/lmb.h @@ -27,12 +27,12 @@ struct lmb_region { struct lmb_type { unsigned long cnt; - phys_addr_t size; struct lmb_region regions[MAX_LMB_REGIONS+1]; }; struct lmb { phys_addr_t current_limit; + phys_addr_t memory_size; /* Updated by lmb_analyze() */ struct lmb_type memory; struct lmb_type reserved; }; diff --git a/lib/lmb.c b/lib/lmb.c index 2995673..41cee3b 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -49,7 +49,7 @@ void lmb_dump_all(void) return; pr_info("LMB configuration:\n"); - pr_info(" memory.size = 0x%llx\n", (unsigned long long)lmb.memory.size); + pr_info(" memory size = 0x%llx\n", (unsigned long long)lmb.memory_size); lmb_dump(&lmb.memory, "memory"); lmb_dump(&lmb.reserved, "reserved"); @@ -123,10 +123,10 @@ void __init lmb_analyze(void) { int i; - lmb.memory.size = 0; + lmb.memory_size = 0; for (i = 0; i < lmb.memory.cnt; i++) - lmb.memory.size += lmb.memory.regions[i].size; + lmb.memory_size += lmb.memory.regions[i].size; } static long lmb_add_region(struct lmb_type *type, phys_addr_t base, phys_addr_t size) @@ -423,7 +423,7 @@ phys_addr_t __init __lmb_alloc_base(phys_addr_t size, phys_addr_t align, phys_ad /* You must call lmb_analyze() before this. */ phys_addr_t __init lmb_phys_mem_size(void) { - return lmb.memory.size; + return lmb.memory_size; } phys_addr_t lmb_end_of_DRAM(void) -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 12/25] lmb: Move lmb arrays to static storage in lmb.c and make their size a variable 2010-05-10 9:38 ` [PATCH 11/25] lmb: Remove lmb_type.size and add lmb.memory_size instead Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 13/25] lmb: Add debug markers at the end of the array Benjamin Herrenschmidt 2010-05-10 10:44 ` [PATCH 12/25] lmb: Move lmb arrays to static storage in lmb.c and make their size a variable Paul Mundt 0 siblings, 2 replies; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt This is in preparation for having resizable arrays. Note that we still allocate one more than needed, this is unchanged from the previous implementation. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- include/linux/lmb.h | 7 ++++--- lib/lmb.c | 10 +++++++++- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/include/linux/lmb.h b/include/linux/lmb.h index 27c2386..e575801 100644 --- a/include/linux/lmb.h +++ b/include/linux/lmb.h @@ -18,7 +18,7 @@ #include <asm/lmb.h> -#define MAX_LMB_REGIONS 128 +#define INIT_LMB_REGIONS 128 struct lmb_region { phys_addr_t base; @@ -26,8 +26,9 @@ struct lmb_region { }; struct lmb_type { - unsigned long cnt; - struct lmb_region regions[MAX_LMB_REGIONS+1]; + unsigned long cnt; /* number of regions */ + unsigned long max; /* size of the allocated array */ + struct lmb_region *regions; }; struct lmb { diff --git a/lib/lmb.c b/lib/lmb.c index 41cee3b..27dbb9c 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -18,6 +18,8 @@ struct lmb lmb; static int lmb_debug; +static struct lmb_region lmb_memory_init_regions[INIT_LMB_REGIONS + 1]; +static struct lmb_region lmb_reserved_init_regions[INIT_LMB_REGIONS + 1]; static int __init early_lmb(char *p) { @@ -104,6 +106,12 @@ static void lmb_coalesce_regions(struct lmb_type *type, void __init lmb_init(void) { + /* Hookup the initial arrays */ + lmb.memory.regions = lmb_memory_init_regions; + lmb.memory.max = INIT_LMB_REGIONS; + lmb.reserved.regions = lmb_reserved_init_regions; + lmb.reserved.max = INIT_LMB_REGIONS; + /* Create a dummy zero size LMB which will get coalesced away later. * This simplifies the lmb_add() code below... */ @@ -169,7 +177,7 @@ static long lmb_add_region(struct lmb_type *type, phys_addr_t base, phys_addr_t if (coalesced) return coalesced; - if (type->cnt >= MAX_LMB_REGIONS) + if (type->cnt >= type->max) return -1; /* Couldn't coalesce the LMB, so add it to the sorted table. */ -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 13/25] lmb: Add debug markers at the end of the array 2010-05-10 9:38 ` [PATCH 12/25] lmb: Move lmb arrays to static storage in lmb.c and make their size a variable Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 14/25] lmb: Make lmb_find_region() out of lmb_alloc_region() Benjamin Herrenschmidt 2010-05-10 10:44 ` [PATCH 12/25] lmb: Move lmb arrays to static storage in lmb.c and make their size a variable Paul Mundt 1 sibling, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt Since we allocate one more than needed, why not do a bit of sanity checking here to ensure we don't walk past the end of the array ? Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- lib/lmb.c | 11 +++++++++++ 1 files changed, 11 insertions(+), 0 deletions(-) diff --git a/lib/lmb.c b/lib/lmb.c index 27dbb9c..6765a3a 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -13,6 +13,7 @@ #include <linux/kernel.h> #include <linux/init.h> #include <linux/bitops.h> +#include <linux/poison.h> #include <linux/lmb.h> struct lmb lmb; @@ -112,6 +113,10 @@ void __init lmb_init(void) lmb.reserved.regions = lmb_reserved_init_regions; lmb.reserved.max = INIT_LMB_REGIONS; + /* Write a marker in the unused last array entry */ + lmb.memory.regions[INIT_LMB_REGIONS].base = (phys_addr_t)RED_INACTIVE; + lmb.reserved.regions[INIT_LMB_REGIONS].base = (phys_addr_t)RED_INACTIVE; + /* Create a dummy zero size LMB which will get coalesced away later. * This simplifies the lmb_add() code below... */ @@ -131,6 +136,12 @@ void __init lmb_analyze(void) { int i; + /* Check marker in the unused last array entry */ + WARN_ON(lmb_memory_init_regions[INIT_LMB_REGIONS].base + != (phys_addr_t)RED_INACTIVE); + WARN_ON(lmb_reserved_init_regions[INIT_LMB_REGIONS].base + != (phys_addr_t)RED_INACTIVE); + lmb.memory_size = 0; for (i = 0; i < lmb.memory.cnt; i++) -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 14/25] lmb: Make lmb_find_region() out of lmb_alloc_region() 2010-05-10 9:38 ` [PATCH 13/25] lmb: Add debug markers at the end of the array Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 15/25] lmb: Define LMB_ERROR internally instead of using ~(phys_addr_t)0 Benjamin Herrenschmidt 0 siblings, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt This function will be used to locate a free area to put the new lmb arrays when attempting to resize them. lmb_alloc_region() is gone, the two callsites now call lmb_add_region(). Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- lib/lmb.c | 20 +++++++++----------- 1 files changed, 9 insertions(+), 11 deletions(-) diff --git a/lib/lmb.c b/lib/lmb.c index 6765a3a..4882e9a 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -309,8 +309,8 @@ static phys_addr_t lmb_align_up(phys_addr_t addr, phys_addr_t size) return (addr + (size - 1)) & ~(size - 1); } -static phys_addr_t __init lmb_alloc_region(phys_addr_t start, phys_addr_t end, - phys_addr_t size, phys_addr_t align) +static phys_addr_t __init lmb_find_region(phys_addr_t start, phys_addr_t end, + phys_addr_t size, phys_addr_t align) { phys_addr_t base, res_base; long j; @@ -318,12 +318,8 @@ static phys_addr_t __init lmb_alloc_region(phys_addr_t start, phys_addr_t end, base = lmb_align_down((end - size), align); while (start <= base) { j = lmb_overlaps_region(&lmb.reserved, base, size); - if (j < 0) { - /* this area isn't reserved, take it */ - if (lmb_add_region(&lmb.reserved, base, size) < 0) - base = ~(phys_addr_t)0; + if (j < 0) return base; - } res_base = lmb.reserved.regions[j].base; if (res_base < size) break; @@ -356,8 +352,9 @@ static phys_addr_t __init lmb_alloc_nid_region(struct lmb_region *mp, this_end = lmb_nid_range(start, end, &this_nid); if (this_nid == nid) { - phys_addr_t ret = lmb_alloc_region(start, this_end, size, align); - if (ret != ~(phys_addr_t)0) + phys_addr_t ret = lmb_find_region(start, this_end, size, align); + if (ret != ~(phys_addr_t)0 && + lmb_add_region(&lmb.reserved, start, size) >= 0) return ret; } start = this_end; @@ -432,8 +429,9 @@ phys_addr_t __init __lmb_alloc_base(phys_addr_t size, phys_addr_t align, phys_ad if (lmbsize < size) continue; base = min(lmbbase + lmbsize, max_addr); - res_base = lmb_alloc_region(lmbbase, base, size, align); - if (res_base != ~(phys_addr_t)0) + res_base = lmb_find_region(lmbbase, base, size, align); + if (res_base != ~(phys_addr_t)0 && + lmb_add_region(&lmb.reserved, res_base, size) >= 0) return res_base; } return 0; -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 15/25] lmb: Define LMB_ERROR internally instead of using ~(phys_addr_t)0 2010-05-10 9:38 ` [PATCH 14/25] lmb: Make lmb_find_region() out of lmb_alloc_region() Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 16/25] lmb: Move lmb_init() to the bottom of the file Benjamin Herrenschmidt 0 siblings, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- lib/lmb.c | 12 +++++++----- 1 files changed, 7 insertions(+), 5 deletions(-) diff --git a/lib/lmb.c b/lib/lmb.c index 4882e9a..9fd0145 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -22,6 +22,8 @@ static int lmb_debug; static struct lmb_region lmb_memory_init_regions[INIT_LMB_REGIONS + 1]; static struct lmb_region lmb_reserved_init_regions[INIT_LMB_REGIONS + 1]; +#define LMB_ERROR (~(phys_addr_t)0) + static int __init early_lmb(char *p) { if (p && strstr(p, "debug")) @@ -326,7 +328,7 @@ static phys_addr_t __init lmb_find_region(phys_addr_t start, phys_addr_t end, base = lmb_align_down(res_base - size, align); } - return ~(phys_addr_t)0; + return LMB_ERROR; } phys_addr_t __weak __init lmb_nid_range(phys_addr_t start, phys_addr_t end, int *nid) @@ -353,14 +355,14 @@ static phys_addr_t __init lmb_alloc_nid_region(struct lmb_region *mp, this_end = lmb_nid_range(start, end, &this_nid); if (this_nid == nid) { phys_addr_t ret = lmb_find_region(start, this_end, size, align); - if (ret != ~(phys_addr_t)0 && + if (ret != LMB_ERROR && lmb_add_region(&lmb.reserved, start, size) >= 0) return ret; } start = this_end; } - return ~(phys_addr_t)0; + return LMB_ERROR; } phys_addr_t __init lmb_alloc_nid(phys_addr_t size, phys_addr_t align, int nid) @@ -379,7 +381,7 @@ phys_addr_t __init lmb_alloc_nid(phys_addr_t size, phys_addr_t align, int nid) for (i = 0; i < mem->cnt; i++) { phys_addr_t ret = lmb_alloc_nid_region(&mem->regions[i], size, align, nid); - if (ret != ~(phys_addr_t)0) + if (ret != LMB_ERROR) return ret; } @@ -430,7 +432,7 @@ phys_addr_t __init __lmb_alloc_base(phys_addr_t size, phys_addr_t align, phys_ad continue; base = min(lmbbase + lmbsize, max_addr); res_base = lmb_find_region(lmbbase, base, size, align); - if (res_base != ~(phys_addr_t)0 && + if (res_base != LMB_ERROR && lmb_add_region(&lmb.reserved, res_base, size) >= 0) return res_base; } -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 16/25] lmb: Move lmb_init() to the bottom of the file 2010-05-10 9:38 ` [PATCH 15/25] lmb: Define LMB_ERROR internally instead of using ~(phys_addr_t)0 Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 17/25] lmb: split lmb_find_base() out of __lmb_alloc_base() Benjamin Herrenschmidt 0 siblings, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt It's a real PITA to have to search for it in the middle Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- lib/lmb.c | 54 +++++++++++++++++++++++++++--------------------------- 1 files changed, 27 insertions(+), 27 deletions(-) diff --git a/lib/lmb.c b/lib/lmb.c index 9fd0145..141d4ab 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -107,33 +107,6 @@ static void lmb_coalesce_regions(struct lmb_type *type, lmb_remove_region(type, r2); } -void __init lmb_init(void) -{ - /* Hookup the initial arrays */ - lmb.memory.regions = lmb_memory_init_regions; - lmb.memory.max = INIT_LMB_REGIONS; - lmb.reserved.regions = lmb_reserved_init_regions; - lmb.reserved.max = INIT_LMB_REGIONS; - - /* Write a marker in the unused last array entry */ - lmb.memory.regions[INIT_LMB_REGIONS].base = (phys_addr_t)RED_INACTIVE; - lmb.reserved.regions[INIT_LMB_REGIONS].base = (phys_addr_t)RED_INACTIVE; - - /* Create a dummy zero size LMB which will get coalesced away later. - * This simplifies the lmb_add() code below... - */ - lmb.memory.regions[0].base = 0; - lmb.memory.regions[0].size = 0; - lmb.memory.cnt = 1; - - /* Ditto. */ - lmb.reserved.regions[0].base = 0; - lmb.reserved.regions[0].size = 0; - lmb.reserved.cnt = 1; - - lmb.current_limit = LMB_ALLOC_ANYWHERE; -} - void __init lmb_analyze(void) { int i; @@ -517,3 +490,30 @@ void __init lmb_set_current_limit(phys_addr_t limit) lmb.current_limit = limit; } +void __init lmb_init(void) +{ + /* Hookup the initial arrays */ + lmb.memory.regions = lmb_memory_init_regions; + lmb.memory.max = INIT_LMB_REGIONS; + lmb.reserved.regions = lmb_reserved_init_regions; + lmb.reserved.max = INIT_LMB_REGIONS; + + /* Write a marker in the unused last array entry */ + lmb.memory.regions[INIT_LMB_REGIONS].base = (phys_addr_t)RED_INACTIVE; + lmb.reserved.regions[INIT_LMB_REGIONS].base = (phys_addr_t)RED_INACTIVE; + + /* Create a dummy zero size LMB which will get coalesced away later. + * This simplifies the lmb_add() code below... + */ + lmb.memory.regions[0].base = 0; + lmb.memory.regions[0].size = 0; + lmb.memory.cnt = 1; + + /* Ditto. */ + lmb.reserved.regions[0].base = 0; + lmb.reserved.regions[0].size = 0; + lmb.reserved.cnt = 1; + + lmb.current_limit = LMB_ALLOC_ANYWHERE; +} + -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 17/25] lmb: split lmb_find_base() out of __lmb_alloc_base() 2010-05-10 9:38 ` [PATCH 16/25] lmb: Move lmb_init() to the bottom of the file Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 18/25] lmb: Move functions around into a more sensible order Benjamin Herrenschmidt 0 siblings, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt This will be used by the array resize code and might prove useful to some arch code as well at which point it can be made non-static. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- lib/lmb.c | 43 +++++++++++++++++++++++++++---------------- 1 files changed, 27 insertions(+), 16 deletions(-) diff --git a/lib/lmb.c b/lib/lmb.c index 141d4ab..95ef5b6 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -366,20 +366,7 @@ phys_addr_t __init lmb_alloc(phys_addr_t size, phys_addr_t align) return lmb_alloc_base(size, align, LMB_ALLOC_ACCESSIBLE); } -phys_addr_t __init lmb_alloc_base(phys_addr_t size, phys_addr_t align, phys_addr_t max_addr) -{ - phys_addr_t alloc; - - alloc = __lmb_alloc_base(size, align, max_addr); - - if (alloc == 0) - panic("ERROR: Failed to allocate 0x%llx bytes below 0x%llx.\n", - (unsigned long long) size, (unsigned long long) max_addr); - - return alloc; -} - -phys_addr_t __init __lmb_alloc_base(phys_addr_t size, phys_addr_t align, phys_addr_t max_addr) +static phys_addr_t __init lmb_find_base(phys_addr_t size, phys_addr_t align, phys_addr_t max_addr) { long i; phys_addr_t base = 0; @@ -405,13 +392,37 @@ phys_addr_t __init __lmb_alloc_base(phys_addr_t size, phys_addr_t align, phys_ad continue; base = min(lmbbase + lmbsize, max_addr); res_base = lmb_find_region(lmbbase, base, size, align); - if (res_base != LMB_ERROR && - lmb_add_region(&lmb.reserved, res_base, size) >= 0) + if (res_base != LMB_ERROR) return res_base; } return 0; } +phys_addr_t __init __lmb_alloc_base(phys_addr_t size, phys_addr_t align, phys_addr_t max_addr) +{ + phys_addr_t found = lmb_find_base(size, align, max_addr); + + if (found != LMB_ERROR && + lmb_add_region(&lmb.reserved, found, size) >= 0) + return found; + + return 0; +} + +phys_addr_t __init lmb_alloc_base(phys_addr_t size, phys_addr_t align, phys_addr_t max_addr) +{ + phys_addr_t alloc; + + alloc = __lmb_alloc_base(size, align, max_addr); + + if (alloc == 0) + panic("ERROR: Failed to allocate 0x%llx bytes below 0x%llx.\n", + (unsigned long long) size, (unsigned long long) max_addr); + + return alloc; +} + + /* You must call lmb_analyze() before this. */ phys_addr_t __init lmb_phys_mem_size(void) { -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 18/25] lmb: Move functions around into a more sensible order 2010-05-10 9:38 ` [PATCH 17/25] lmb: split lmb_find_base() out of __lmb_alloc_base() Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 19/25] lmb: Add array resizing support Benjamin Herrenschmidt 0 siblings, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt Some shuffling is needed for doing array resize so we may as well put some sense into the ordering of the functions in the whole lmb.c file. No code change. Added some comments. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- lib/lmb.c | 295 ++++++++++++++++++++++++++++++++---------------------------- 1 files changed, 157 insertions(+), 138 deletions(-) diff --git a/lib/lmb.c b/lib/lmb.c index 95ef5b6..4977888 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -24,40 +24,18 @@ static struct lmb_region lmb_reserved_init_regions[INIT_LMB_REGIONS + 1]; #define LMB_ERROR (~(phys_addr_t)0) -static int __init early_lmb(char *p) -{ - if (p && strstr(p, "debug")) - lmb_debug = 1; - return 0; -} -early_param("lmb", early_lmb); +/* + * Address comparison utilities + */ -static void lmb_dump(struct lmb_type *region, char *name) +static phys_addr_t lmb_align_down(phys_addr_t addr, phys_addr_t size) { - unsigned long long base, size; - int i; - - pr_info(" %s.cnt = 0x%lx\n", name, region->cnt); - - for (i = 0; i < region->cnt; i++) { - base = region->regions[i].base; - size = region->regions[i].size; - - pr_info(" %s[0x%x]\t0x%016llx - 0x%016llx, 0x%llx bytes\n", - name, i, base, base + size - 1, size); - } + return addr & ~(size - 1); } -void lmb_dump_all(void) +static phys_addr_t lmb_align_up(phys_addr_t addr, phys_addr_t size) { - if (!lmb_debug) - return; - - pr_info("LMB configuration:\n"); - pr_info(" memory size = 0x%llx\n", (unsigned long long)lmb.memory_size); - - lmb_dump(&lmb.memory, "memory"); - lmb_dump(&lmb.reserved, "reserved"); + return (addr + (size - 1)) & ~(size - 1); } static unsigned long lmb_addrs_overlap(phys_addr_t base1, phys_addr_t size1, @@ -88,6 +66,77 @@ static long lmb_regions_adjacent(struct lmb_type *type, return lmb_addrs_adjacent(base1, size1, base2, size2); } +long lmb_overlaps_region(struct lmb_type *type, phys_addr_t base, phys_addr_t size) +{ + unsigned long i; + + for (i = 0; i < type->cnt; i++) { + phys_addr_t rgnbase = type->regions[i].base; + phys_addr_t rgnsize = type->regions[i].size; + if (lmb_addrs_overlap(base, size, rgnbase, rgnsize)) + break; + } + + return (i < type->cnt) ? i : -1; +} + +/* + * Find, allocate, deallocate or reserve unreserved regions. All allocations + * are top-down. + */ + +static phys_addr_t __init lmb_find_region(phys_addr_t start, phys_addr_t end, + phys_addr_t size, phys_addr_t align) +{ + phys_addr_t base, res_base; + long j; + + base = lmb_align_down((end - size), align); + while (start <= base) { + j = lmb_overlaps_region(&lmb.reserved, base, size); + if (j < 0) + return base; + res_base = lmb.reserved.regions[j].base; + if (res_base < size) + break; + base = lmb_align_down(res_base - size, align); + } + + return LMB_ERROR; +} + +static phys_addr_t __init lmb_find_base(phys_addr_t size, phys_addr_t align, phys_addr_t max_addr) +{ + long i; + phys_addr_t base = 0; + phys_addr_t res_base; + + BUG_ON(0 == size); + + size = lmb_align_up(size, align); + + /* Pump up max_addr */ + if (max_addr == LMB_ALLOC_ACCESSIBLE) + max_addr = lmb.current_limit; + + /* We do a top-down search, this tends to limit memory + * fragmentation by keeping early boot allocs near the + * top of memory + */ + for (i = lmb.memory.cnt - 1; i >= 0; i--) { + phys_addr_t lmbbase = lmb.memory.regions[i].base; + phys_addr_t lmbsize = lmb.memory.regions[i].size; + + if (lmbsize < size) + continue; + base = min(lmbbase + lmbsize, max_addr); + res_base = lmb_find_region(lmbbase, base, size, align); + if (res_base != LMB_ERROR) + return res_base; + } + return 0; +} + static void lmb_remove_region(struct lmb_type *type, unsigned long r) { unsigned long i; @@ -107,22 +156,6 @@ static void lmb_coalesce_regions(struct lmb_type *type, lmb_remove_region(type, r2); } -void __init lmb_analyze(void) -{ - int i; - - /* Check marker in the unused last array entry */ - WARN_ON(lmb_memory_init_regions[INIT_LMB_REGIONS].base - != (phys_addr_t)RED_INACTIVE); - WARN_ON(lmb_reserved_init_regions[INIT_LMB_REGIONS].base - != (phys_addr_t)RED_INACTIVE); - - lmb.memory_size = 0; - - for (i = 0; i < lmb.memory.cnt; i++) - lmb.memory_size += lmb.memory.regions[i].size; -} - static long lmb_add_region(struct lmb_type *type, phys_addr_t base, phys_addr_t size) { unsigned long coalesced = 0; @@ -260,50 +293,42 @@ long __init lmb_reserve(phys_addr_t base, phys_addr_t size) return lmb_add_region(_rgn, base, size); } -long lmb_overlaps_region(struct lmb_type *type, phys_addr_t base, phys_addr_t size) +phys_addr_t __init __lmb_alloc_base(phys_addr_t size, phys_addr_t align, phys_addr_t max_addr) { - unsigned long i; + phys_addr_t found = lmb_find_base(size, align, max_addr); - for (i = 0; i < type->cnt; i++) { - phys_addr_t rgnbase = type->regions[i].base; - phys_addr_t rgnsize = type->regions[i].size; - if (lmb_addrs_overlap(base, size, rgnbase, rgnsize)) - break; - } + if (found != LMB_ERROR && + lmb_add_region(&lmb.reserved, found, size) >= 0) + return found; - return (i < type->cnt) ? i : -1; + return 0; } -static phys_addr_t lmb_align_down(phys_addr_t addr, phys_addr_t size) +phys_addr_t __init lmb_alloc_base(phys_addr_t size, phys_addr_t align, phys_addr_t max_addr) { - return addr & ~(size - 1); -} + phys_addr_t alloc; -static phys_addr_t lmb_align_up(phys_addr_t addr, phys_addr_t size) -{ - return (addr + (size - 1)) & ~(size - 1); -} + alloc = __lmb_alloc_base(size, align, max_addr); -static phys_addr_t __init lmb_find_region(phys_addr_t start, phys_addr_t end, - phys_addr_t size, phys_addr_t align) -{ - phys_addr_t base, res_base; - long j; + if (alloc == 0) + panic("ERROR: Failed to allocate 0x%llx bytes below 0x%llx.\n", + (unsigned long long) size, (unsigned long long) max_addr); - base = lmb_align_down((end - size), align); - while (start <= base) { - j = lmb_overlaps_region(&lmb.reserved, base, size); - if (j < 0) - return base; - res_base = lmb.reserved.regions[j].base; - if (res_base < size) - break; - base = lmb_align_down(res_base - size, align); - } + return alloc; +} - return LMB_ERROR; +phys_addr_t __init lmb_alloc(phys_addr_t size, phys_addr_t align) +{ + return lmb_alloc_base(size, align, LMB_ALLOC_ACCESSIBLE); } + +/* + * Additional node-local allocators. Search for node memory is bottom up + * and walks lmb regions within that node bottom-up as well, but allocation + * within an lmb region is top-down. + */ + phys_addr_t __weak __init lmb_nid_range(phys_addr_t start, phys_addr_t end, int *nid) { *nid = 0; @@ -361,67 +386,9 @@ phys_addr_t __init lmb_alloc_nid(phys_addr_t size, phys_addr_t align, int nid) return lmb_alloc(size, align); } -phys_addr_t __init lmb_alloc(phys_addr_t size, phys_addr_t align) -{ - return lmb_alloc_base(size, align, LMB_ALLOC_ACCESSIBLE); -} - -static phys_addr_t __init lmb_find_base(phys_addr_t size, phys_addr_t align, phys_addr_t max_addr) -{ - long i; - phys_addr_t base = 0; - phys_addr_t res_base; - - BUG_ON(0 == size); - - size = lmb_align_up(size, align); - - /* Pump up max_addr */ - if (max_addr == LMB_ALLOC_ACCESSIBLE) - max_addr = lmb.current_limit; - - /* We do a top-down search, this tends to limit memory - * fragmentation by keeping early boot allocs near the - * top of memory - */ - for (i = lmb.memory.cnt - 1; i >= 0; i--) { - phys_addr_t lmbbase = lmb.memory.regions[i].base; - phys_addr_t lmbsize = lmb.memory.regions[i].size; - - if (lmbsize < size) - continue; - base = min(lmbbase + lmbsize, max_addr); - res_base = lmb_find_region(lmbbase, base, size, align); - if (res_base != LMB_ERROR) - return res_base; - } - return 0; -} - -phys_addr_t __init __lmb_alloc_base(phys_addr_t size, phys_addr_t align, phys_addr_t max_addr) -{ - phys_addr_t found = lmb_find_base(size, align, max_addr); - - if (found != LMB_ERROR && - lmb_add_region(&lmb.reserved, found, size) >= 0) - return found; - - return 0; -} - -phys_addr_t __init lmb_alloc_base(phys_addr_t size, phys_addr_t align, phys_addr_t max_addr) -{ - phys_addr_t alloc; - - alloc = __lmb_alloc_base(size, align, max_addr); - - if (alloc == 0) - panic("ERROR: Failed to allocate 0x%llx bytes below 0x%llx.\n", - (unsigned long long) size, (unsigned long long) max_addr); - - return alloc; -} - +/* + * Remaining API functions + */ /* You must call lmb_analyze() before this. */ phys_addr_t __init lmb_phys_mem_size(void) @@ -501,6 +468,50 @@ void __init lmb_set_current_limit(phys_addr_t limit) lmb.current_limit = limit; } +static void lmb_dump(struct lmb_type *region, char *name) +{ + unsigned long long base, size; + int i; + + pr_info(" %s.cnt = 0x%lx\n", name, region->cnt); + + for (i = 0; i < region->cnt; i++) { + base = region->regions[i].base; + size = region->regions[i].size; + + pr_info(" %s[0x%x]\t0x%016llx - 0x%016llx, 0x%llx bytes\n", + name, i, base, base + size - 1, size); + } +} + +void lmb_dump_all(void) +{ + if (!lmb_debug) + return; + + pr_info("LMB configuration:\n"); + pr_info(" memory size = 0x%llx\n", (unsigned long long)lmb.memory_size); + + lmb_dump(&lmb.memory, "memory"); + lmb_dump(&lmb.reserved, "reserved"); +} + +void __init lmb_analyze(void) +{ + int i; + + /* Check marker in the unused last array entry */ + WARN_ON(lmb_memory_init_regions[INIT_LMB_REGIONS].base + != (phys_addr_t)RED_INACTIVE); + WARN_ON(lmb_reserved_init_regions[INIT_LMB_REGIONS].base + != (phys_addr_t)RED_INACTIVE); + + lmb.memory_size = 0; + + for (i = 0; i < lmb.memory.cnt; i++) + lmb.memory_size += lmb.memory.regions[i].size; +} + void __init lmb_init(void) { /* Hookup the initial arrays */ @@ -528,3 +539,11 @@ void __init lmb_init(void) lmb.current_limit = LMB_ALLOC_ANYWHERE; } +static int __init early_lmb(char *p) +{ + if (p && strstr(p, "debug")) + lmb_debug = 1; + return 0; +} +early_param("lmb", early_lmb); + -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 19/25] lmb: Add array resizing support 2010-05-10 9:38 ` [PATCH 18/25] lmb: Move functions around into a more sensible order Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 20/25] lmb: Add arch function to control coalescing of lmb memory regions Benjamin Herrenschmidt 2010-05-10 23:59 ` [PATCH 19/25] lmb: Add array resizing support Yinghai Lu 0 siblings, 2 replies; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt When one of the array gets full, we resize it. After much thinking and a few iterations of that code, I went back to on-demand resizing using the (new) internal lmb_find_base() function, which is pretty much what Yinghai initially proposed, though there some differences in the details. To work this relies on the default alloc limit being set sensibly by the architecture. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- lib/lmb.c | 93 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 92 insertions(+), 1 deletions(-) diff --git a/lib/lmb.c b/lib/lmb.c index 4977888..2602683 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -11,6 +11,7 @@ */ #include <linux/kernel.h> +#include <linux/slab.h> #include <linux/init.h> #include <linux/bitops.h> #include <linux/poison.h> @@ -24,6 +25,17 @@ static struct lmb_region lmb_reserved_init_regions[INIT_LMB_REGIONS + 1]; #define LMB_ERROR (~(phys_addr_t)0) +/* inline so we don't get a warning when pr_debug is compiled out */ +static inline const char *lmb_type_name(struct lmb_type *type) +{ + if (type == &lmb.memory) + return "memory"; + else if (type == &lmb.reserved) + return "reserved"; + else + return "unknown"; +} + /* * Address comparison utilities */ @@ -156,6 +168,73 @@ static void lmb_coalesce_regions(struct lmb_type *type, lmb_remove_region(type, r2); } +/* Defined below but needed now */ +static long lmb_add_region(struct lmb_type *type, phys_addr_t base, phys_addr_t size); + +static int lmb_double_array(struct lmb_type *type) +{ + struct lmb_region *new_array, *old_array; + phys_addr_t old_size, new_size, addr; + int use_slab = slab_is_available(); + + pr_debug("lmb: %s array full, doubling...", lmb_type_name(type)); + + /* Calculate new doubled size */ + old_size = type->max * sizeof(struct lmb_region); + new_size = old_size << 1; + + /* Try to find some space for it. + * + * WARNING: We assume that either slab_is_available() and we use it or + * we use LMB for allocations. That means that this is unsafe to use + * when bootmem is currently active (unless bootmem itself is implemented + * on top of LMB which isn't the case yet) + * + * This should however not be an issue for now, as we currently only + * call into LMB while it's still active, or much later when slab is + * active for memory hotplug operations + */ + if (use_slab) { + new_array = kmalloc(new_size, GFP_KERNEL); + addr = new_array == NULL ? LMB_ERROR : __pa(new_array); + } else + addr = lmb_find_base(new_size, sizeof(phys_addr_t), LMB_ALLOC_ACCESSIBLE); + if (addr == LMB_ERROR) { + pr_err("lmb: Failed to double %s array from %ld to %ld entries !\n", + lmb_type_name(type), type->max, type->max * 2); + return -1; + } + new_array = __va(addr); + + /* Found space, we now need to move the array over before + * we add the reserved region since it may be our reserved + * array itself that is full. + */ + memcpy(new_array, type->regions, old_size); + memset(new_array + type->max, 0, old_size); + old_array = type->regions; + type->regions = new_array; + type->max <<= 1; + + /* If we use SLAB that's it, we are done */ + if (use_slab) + return 0; + + /* Add the new reserved region now. Should not fail ! */ + BUG_ON(lmb_add_region(&lmb.reserved, addr, new_size) < 0); + + /* If the array wasn't our static init one, then free it. We only do + * that before SLAB is available as later on, we don't know whether + * to use kfree or free_bootmem_pages(). Shouldn't be a big deal + * anyways + */ + if (old_array != lmb_memory_init_regions && + old_array != lmb_reserved_init_regions) + lmb_free(__pa(old_array), old_size); + + return 0; +} + static long lmb_add_region(struct lmb_type *type, phys_addr_t base, phys_addr_t size) { unsigned long coalesced = 0; @@ -196,7 +275,11 @@ static long lmb_add_region(struct lmb_type *type, phys_addr_t base, phys_addr_t if (coalesced) return coalesced; - if (type->cnt >= type->max) + + /* If we are out of space, we fail. It's too late to resize the array + * but then this shouldn't have happened in the first place. + */ + if (WARN_ON(type->cnt >= type->max)) return -1; /* Couldn't coalesce the LMB, so add it to the sorted table. */ @@ -217,6 +300,14 @@ static long lmb_add_region(struct lmb_type *type, phys_addr_t base, phys_addr_t } type->cnt++; + /* The array is full ? Try to resize it. If that fails, we undo + * our allocation and return an error + */ + if (type->cnt == type->max && lmb_double_array(type)) { + type->cnt--; + return -1; + } + return 0; } -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 20/25] lmb: Add arch function to control coalescing of lmb memory regions 2010-05-10 9:38 ` [PATCH 19/25] lmb: Add array resizing support Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 21/25] lmb: Add "start" argument to lmb_find_base() Benjamin Herrenschmidt 2010-05-10 23:59 ` [PATCH 19/25] lmb: Add array resizing support Yinghai Lu 1 sibling, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt Some archs such as ARM want to avoid coalescing accross things such as the lowmem/highmem boundary or similar. This provides the option to control it via an arch callback for which a weak default is provided which always allows coalescing. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- include/linux/lmb.h | 2 ++ lib/lmb.c | 19 ++++++++++++++++++- 2 files changed, 20 insertions(+), 1 deletions(-) diff --git a/include/linux/lmb.h b/include/linux/lmb.h index e575801..404b49c 100644 --- a/include/linux/lmb.h +++ b/include/linux/lmb.h @@ -70,6 +70,8 @@ extern void lmb_dump_all(void); /* Provided by the architecture */ extern phys_addr_t lmb_nid_range(phys_addr_t start, phys_addr_t end, int *nid); +extern int lmb_memory_can_coalesce(phys_addr_t addr1, phys_addr_t size1, + phys_addr_t addr2, phys_addr_t size2); /** * lmb_set_current_limit - Set the current allocation limit to allow diff --git a/lib/lmb.c b/lib/lmb.c index 2602683..84ac3a9 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -235,6 +235,12 @@ static int lmb_double_array(struct lmb_type *type) return 0; } +extern int __weak lmb_memory_can_coalesce(phys_addr_t addr1, phys_addr_t size1, + phys_addr_t addr2, phys_addr_t size2) +{ + return 1; +} + static long lmb_add_region(struct lmb_type *type, phys_addr_t base, phys_addr_t size) { unsigned long coalesced = 0; @@ -256,6 +262,10 @@ static long lmb_add_region(struct lmb_type *type, phys_addr_t base, phys_addr_t return 0; adjacent = lmb_addrs_adjacent(base, size, rgnbase, rgnsize); + /* Check if arch allows coalescing */ + if (adjacent != 0 && type == &lmb.memory && + !lmb_memory_can_coalesce(base, size, rgnbase, rgnsize)) + break; if (adjacent > 0) { type->regions[i].base -= size; type->regions[i].size += size; @@ -268,7 +278,14 @@ static long lmb_add_region(struct lmb_type *type, phys_addr_t base, phys_addr_t } } - if ((i < type->cnt - 1) && lmb_regions_adjacent(type, i, i+1)) { + /* If we plugged a hole, we may want to also coalesce with the + * next region + */ + if ((i < type->cnt - 1) && lmb_regions_adjacent(type, i, i+1) && + ((type != &lmb.memory || lmb_memory_can_coalesce(type->regions[i].base, + type->regions[i].size, + type->regions[i+1].base, + type->regions[i+1].size)))) { lmb_coalesce_regions(type, i, i+1); coalesced++; } -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 21/25] lmb: Add "start" argument to lmb_find_base() 2010-05-10 9:38 ` [PATCH 20/25] lmb: Add arch function to control coalescing of lmb memory regions Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 22/25] lmb: NUMA allocate can now use early_pfn_map Benjamin Herrenschmidt 2010-05-10 23:37 ` [PATCH 21/25] lmb: Add "start" argument to lmb_find_base() Yinghai Lu 0 siblings, 2 replies; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt To constraint the search of a region between two boundaries, which will be used by the new NUMA aware allocator among others. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- lib/lmb.c | 27 ++++++++++++++++----------- 1 files changed, 16 insertions(+), 11 deletions(-) diff --git a/lib/lmb.c b/lib/lmb.c index 84ac3a9..848f908 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -117,19 +117,18 @@ static phys_addr_t __init lmb_find_region(phys_addr_t start, phys_addr_t end, return LMB_ERROR; } -static phys_addr_t __init lmb_find_base(phys_addr_t size, phys_addr_t align, phys_addr_t max_addr) +static phys_addr_t __init lmb_find_base(phys_addr_t size, phys_addr_t align, + phys_addr_t start, phys_addr_t end) { long i; - phys_addr_t base = 0; - phys_addr_t res_base; BUG_ON(0 == size); size = lmb_align_up(size, align); /* Pump up max_addr */ - if (max_addr == LMB_ALLOC_ACCESSIBLE) - max_addr = lmb.current_limit; + if (end == LMB_ALLOC_ACCESSIBLE) + end = lmb.current_limit; /* We do a top-down search, this tends to limit memory * fragmentation by keeping early boot allocs near the @@ -138,13 +137,19 @@ static phys_addr_t __init lmb_find_base(phys_addr_t size, phys_addr_t align, phy for (i = lmb.memory.cnt - 1; i >= 0; i--) { phys_addr_t lmbbase = lmb.memory.regions[i].base; phys_addr_t lmbsize = lmb.memory.regions[i].size; + phys_addr_t bottom, top, found; if (lmbsize < size) continue; - base = min(lmbbase + lmbsize, max_addr); - res_base = lmb_find_region(lmbbase, base, size, align); - if (res_base != LMB_ERROR) - return res_base; + if ((lmbbase + lmbsize) <= start) + break; + bottom = max(lmbbase, start); + top = min(lmbbase + lmbsize, end); + if (bottom >= top) + continue; + found = lmb_find_region(lmbbase, top, size, align); + if (found != LMB_ERROR) + return found; } return 0; } @@ -198,7 +203,7 @@ static int lmb_double_array(struct lmb_type *type) new_array = kmalloc(new_size, GFP_KERNEL); addr = new_array == NULL ? LMB_ERROR : __pa(new_array); } else - addr = lmb_find_base(new_size, sizeof(phys_addr_t), LMB_ALLOC_ACCESSIBLE); + addr = lmb_find_base(new_size, sizeof(phys_addr_t), 0, LMB_ALLOC_ACCESSIBLE); if (addr == LMB_ERROR) { pr_err("lmb: Failed to double %s array from %ld to %ld entries !\n", lmb_type_name(type), type->max, type->max * 2); @@ -403,7 +408,7 @@ long __init lmb_reserve(phys_addr_t base, phys_addr_t size) phys_addr_t __init __lmb_alloc_base(phys_addr_t size, phys_addr_t align, phys_addr_t max_addr) { - phys_addr_t found = lmb_find_base(size, align, max_addr); + phys_addr_t found = lmb_find_base(size, align, 0, max_addr); if (found != LMB_ERROR && lmb_add_region(&lmb.reserved, found, size) >= 0) -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 22/25] lmb: NUMA allocate can now use early_pfn_map 2010-05-10 9:38 ` [PATCH 21/25] lmb: Add "start" argument to lmb_find_base() Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 23/25] lmb: Separate lmb_alloc_nid() and lmb_alloc_try_nid() Benjamin Herrenschmidt 2010-05-10 23:37 ` [PATCH 21/25] lmb: Add "start" argument to lmb_find_base() Yinghai Lu 1 sibling, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt We now provide a default (weak) implementation of lmb_nid_range() which uses the early_pfn_map[] if CONFIG_ARCH_POPULATES_NODE_MAP is set. Sparc still needs to use its own method due to the way the pages can be scattered between nodes. This implementation is inefficient due to our main algorithm and callback construct wanting to work on an ascending addresses bases while early_pfn_map[] would rather work with nid's (it's unsorted at that stage). But it should work and we can look into improving it subsequently, possibly using arch compile options to chose a different algorithm alltogether. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- include/linux/lmb.h | 3 +++ lib/lmb.c | 28 +++++++++++++++++++++++++++- 2 files changed, 30 insertions(+), 1 deletions(-) diff --git a/include/linux/lmb.h b/include/linux/lmb.h index 404b49c..45724a6 100644 --- a/include/linux/lmb.h +++ b/include/linux/lmb.h @@ -47,6 +47,9 @@ extern long lmb_remove(phys_addr_t base, phys_addr_t size); extern long __init lmb_free(phys_addr_t base, phys_addr_t size); extern long __init lmb_reserve(phys_addr_t base, phys_addr_t size); +/* The numa aware allocator is only available if + * CONFIG_ARCH_POPULATES_NODE_MAP is set + */ extern phys_addr_t __init lmb_alloc_nid(phys_addr_t size, phys_addr_t align, int nid); extern phys_addr_t __init lmb_alloc(phys_addr_t size, phys_addr_t align); diff --git a/lib/lmb.c b/lib/lmb.c index 848f908..f4b2f95 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -15,6 +15,7 @@ #include <linux/init.h> #include <linux/bitops.h> #include <linux/poison.h> +#include <linux/pfn.h> #include <linux/lmb.h> struct lmb lmb; @@ -439,11 +440,36 @@ phys_addr_t __init lmb_alloc(phys_addr_t size, phys_addr_t align) /* * Additional node-local allocators. Search for node memory is bottom up * and walks lmb regions within that node bottom-up as well, but allocation - * within an lmb region is top-down. + * within an lmb region is top-down. XXX I plan to fix that at some stage + * + * WARNING: Only available after early_node_map[] has been populated, + * on some architectures, that is after all the calls to add_active_range() + * have been done to populate it. */ phys_addr_t __weak __init lmb_nid_range(phys_addr_t start, phys_addr_t end, int *nid) { +#ifdef CONFIG_ARCH_POPULATES_NODE_MAP + /* + * This code originates from sparc which really wants use to walk by addresses + * and returns the nid. This is not very convenient for early_pfn_map[] users + * as the map isn't sorted yet, and it really wants to be walked by nid. + * + * For now, I implement the inefficient method below which walks the early + * map multiple times. Eventually we may want to use an ARCH config option + * to implement a completely different method for both case. + */ + unsigned long start_pfn, end_pfn; + int i; + + for (i = 0; i < MAX_NUMNODES; i++) { + get_pfn_range_for_nid(i, &start_pfn, &end_pfn); + if (start < PFN_PHYS(start_pfn) || start >= PFN_PHYS(end_pfn)) + continue; + *nid = i; + return min(end, PFN_PHYS(end_pfn)); + } +#endif *nid = 0; return end; -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH 23/25] lmb: Separate lmb_alloc_nid() and lmb_alloc_try_nid() 2010-05-10 9:38 ` [PATCH 22/25] lmb: NUMA allocate can now use early_pfn_map Benjamin Herrenschmidt @ 2010-05-10 9:38 ` Benjamin Herrenschmidt 0 siblings, 0 replies; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 9:38 UTC (permalink / raw) To: linux-mm; +Cc: linux-kernel, tglx, mingo, davem, lethal, Benjamin Herrenschmidt The former is now strict, it will fail if it cannot honor the allocation within the node, while the later implements the previous semantic which falls back to allocating anywhere. Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> --- arch/sparc/mm/init_64.c | 4 ++-- include/linux/lmb.h | 6 +++++- lib/lmb.c | 10 ++++++++++ 3 files changed, 17 insertions(+), 3 deletions(-) diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c index 88443c8..86477c5 100644 --- a/arch/sparc/mm/init_64.c +++ b/arch/sparc/mm/init_64.c @@ -820,7 +820,7 @@ static void __init allocate_node_data(int nid) struct pglist_data *p; #ifdef CONFIG_NEED_MULTIPLE_NODES - paddr = lmb_alloc_nid(sizeof(struct pglist_data), SMP_CACHE_BYTES, nid); + paddr = lmb_alloc_try_nid(sizeof(struct pglist_data), SMP_CACHE_BYTES, nid); if (!paddr) { prom_printf("Cannot allocate pglist_data for nid[%d]\n", nid); prom_halt(); @@ -840,7 +840,7 @@ static void __init allocate_node_data(int nid) if (p->node_spanned_pages) { num_pages = bootmem_bootmap_pages(p->node_spanned_pages); - paddr = lmb_alloc_nid(num_pages << PAGE_SHIFT, PAGE_SIZE, nid); + paddr = lmb_alloc_try_nid(num_pages << PAGE_SHIFT, PAGE_SIZE, nid); if (!paddr) { prom_printf("Cannot allocate bootmap for nid[%d]\n", nid); diff --git a/include/linux/lmb.h b/include/linux/lmb.h index 45724a6..4e45aa9 100644 --- a/include/linux/lmb.h +++ b/include/linux/lmb.h @@ -50,7 +50,11 @@ extern long __init lmb_reserve(phys_addr_t base, phys_addr_t size); /* The numa aware allocator is only available if * CONFIG_ARCH_POPULATES_NODE_MAP is set */ -extern phys_addr_t __init lmb_alloc_nid(phys_addr_t size, phys_addr_t align, int nid); +extern phys_addr_t __init lmb_alloc_nid(phys_addr_t size, phys_addr_t align, + int nid); +extern phys_addr_t __init lmb_alloc_try_nid(phys_addr_t size, phys_addr_t align, + int nid); + extern phys_addr_t __init lmb_alloc(phys_addr_t size, phys_addr_t align); /* Flags for lmb_alloc_base() amd __lmb_alloc_base() */ diff --git a/lib/lmb.c b/lib/lmb.c index f4b2f95..fd98261 100644 --- a/lib/lmb.c +++ b/lib/lmb.c @@ -522,9 +522,19 @@ phys_addr_t __init lmb_alloc_nid(phys_addr_t size, phys_addr_t align, int nid) return ret; } + return 0; +} + +phys_addr_t __init lmb_alloc_try_nid(phys_addr_t size, phys_addr_t align, int nid) +{ + phys_addr_t res = lmb_alloc_nid(size, align, nid); + + if (res) + return res; return lmb_alloc(size, align); } + /* * Remaining API functions */ -- 1.6.3.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* Re: [PATCH 21/25] lmb: Add "start" argument to lmb_find_base() 2010-05-10 9:38 ` [PATCH 21/25] lmb: Add "start" argument to lmb_find_base() Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 22/25] lmb: NUMA allocate can now use early_pfn_map Benjamin Herrenschmidt @ 2010-05-10 23:37 ` Yinghai Lu 2010-05-11 4:56 ` Benjamin Herrenschmidt 1 sibling, 1 reply; 37+ messages in thread From: Yinghai Lu @ 2010-05-10 23:37 UTC (permalink / raw) To: Benjamin Herrenschmidt; +Cc: linux-mm, linux-kernel, tglx, mingo, davem, lethal On Mon, May 10, 2010 at 2:38 AM, Benjamin Herrenschmidt <benh@kernel.crashing.org> wrote: > To constraint the search of a region between two boundaries, > which will be used by the new NUMA aware allocator among others. > > Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> > --- > lib/lmb.c | 27 ++++++++++++++++----------- > 1 files changed, 16 insertions(+), 11 deletions(-) > > diff --git a/lib/lmb.c b/lib/lmb.c > index 84ac3a9..848f908 100644 > --- a/lib/lmb.c > +++ b/lib/lmb.c > @@ -117,19 +117,18 @@ static phys_addr_t __init lmb_find_region(phys_addr_t start, phys_addr_t end, > return LMB_ERROR; > } > > -static phys_addr_t __init lmb_find_base(phys_addr_t size, phys_addr_t align, phys_addr_t max_addr) > +static phys_addr_t __init lmb_find_base(phys_addr_t size, phys_addr_t align, > + phys_addr_t start, phys_addr_t end) > { > long i; > - phys_addr_t base = 0; > - phys_addr_t res_base; > > BUG_ON(0 == size); > > size = lmb_align_up(size, align); > > /* Pump up max_addr */ > - if (max_addr == LMB_ALLOC_ACCESSIBLE) > - max_addr = lmb.current_limit; > + if (end == LMB_ALLOC_ACCESSIBLE) > + end = lmb.current_limit; > > /* We do a top-down search, this tends to limit memory > * fragmentation by keeping early boot allocs near the > @@ -138,13 +137,19 @@ static phys_addr_t __init lmb_find_base(phys_addr_t size, phys_addr_t align, phy > for (i = lmb.memory.cnt - 1; i >= 0; i--) { > phys_addr_t lmbbase = lmb.memory.regions[i].base; > phys_addr_t lmbsize = lmb.memory.regions[i].size; > + phys_addr_t bottom, top, found; > > if (lmbsize < size) > continue; > - base = min(lmbbase + lmbsize, max_addr); > - res_base = lmb_find_region(lmbbase, base, size, align); > - if (res_base != LMB_ERROR) > - return res_base; > + if ((lmbbase + lmbsize) <= start) > + break; > + bottom = max(lmbbase, start); > + top = min(lmbbase + lmbsize, end); > + if (bottom >= top) > + continue; > + found = lmb_find_region(lmbbase, top, size, align); ^^^^^^^^^ should use bottom here YH -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 21/25] lmb: Add "start" argument to lmb_find_base() 2010-05-10 23:37 ` [PATCH 21/25] lmb: Add "start" argument to lmb_find_base() Yinghai Lu @ 2010-05-11 4:56 ` Benjamin Herrenschmidt 0 siblings, 0 replies; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-11 4:56 UTC (permalink / raw) To: Yinghai Lu; +Cc: linux-mm, linux-kernel, tglx, mingo, davem, lethal On Mon, 2010-05-10 at 16:37 -0700, Yinghai Lu wrote: > > if (lmbsize < size) > > continue; > > - base = min(lmbbase + lmbsize, max_addr); > > - res_base = lmb_find_region(lmbbase, base, size, align); > > - if (res_base != LMB_ERROR) > > - return res_base; > > + if ((lmbbase + lmbsize) <= start) > > + break; > > + bottom = max(lmbbase, start); > > + top = min(lmbbase + lmbsize, end); > > + if (bottom >= top) > > + continue; > > + found = lmb_find_region(lmbbase, top, size, align); > ^^^^^^^^^ > should use bottom here Correct, I missed that when converting. Cheers, Ben. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 19/25] lmb: Add array resizing support 2010-05-10 9:38 ` [PATCH 19/25] lmb: Add array resizing support Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 20/25] lmb: Add arch function to control coalescing of lmb memory regions Benjamin Herrenschmidt @ 2010-05-10 23:59 ` Yinghai Lu 2010-05-11 4:56 ` Benjamin Herrenschmidt 1 sibling, 1 reply; 37+ messages in thread From: Yinghai Lu @ 2010-05-10 23:59 UTC (permalink / raw) To: Benjamin Herrenschmidt; +Cc: linux-mm, linux-kernel, tglx, mingo, davem, lethal On Mon, May 10, 2010 at 2:38 AM, Benjamin Herrenschmidt <benh@kernel.crashing.org> wrote: > When one of the array gets full, we resize it. After much thinking and > a few iterations of that code, I went back to on-demand resizing using > the (new) internal lmb_find_base() function, which is pretty much what > Yinghai initially proposed, though there some differences in the details. > > To work this relies on the default alloc limit being set sensibly by > the architecture. > > Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> > --- > lib/lmb.c | 93 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- > 1 files changed, 92 insertions(+), 1 deletions(-) > > diff --git a/lib/lmb.c b/lib/lmb.c > index 4977888..2602683 100644 > --- a/lib/lmb.c > +++ b/lib/lmb.c > @@ -11,6 +11,7 @@ > */ > > #include <linux/kernel.h> > +#include <linux/slab.h> > #include <linux/init.h> > #include <linux/bitops.h> > #include <linux/poison.h> > @@ -24,6 +25,17 @@ static struct lmb_region lmb_reserved_init_regions[INIT_LMB_REGIONS + 1]; > > #define LMB_ERROR (~(phys_addr_t)0) > > +/* inline so we don't get a warning when pr_debug is compiled out */ > +static inline const char *lmb_type_name(struct lmb_type *type) > +{ > + if (type == &lmb.memory) > + return "memory"; > + else if (type == &lmb.reserved) > + return "reserved"; > + else > + return "unknown"; > +} > + > /* > * Address comparison utilities > */ > @@ -156,6 +168,73 @@ static void lmb_coalesce_regions(struct lmb_type *type, > lmb_remove_region(type, r2); > } > > +/* Defined below but needed now */ > +static long lmb_add_region(struct lmb_type *type, phys_addr_t base, phys_addr_t size); > + > +static int lmb_double_array(struct lmb_type *type) > +{ > + struct lmb_region *new_array, *old_array; > + phys_addr_t old_size, new_size, addr; > + int use_slab = slab_is_available(); > + > + pr_debug("lmb: %s array full, doubling...", lmb_type_name(type)); > + > + /* Calculate new doubled size */ > + old_size = type->max * sizeof(struct lmb_region); > + new_size = old_size << 1; > + > + /* Try to find some space for it. > + * > + * WARNING: We assume that either slab_is_available() and we use it or > + * we use LMB for allocations. That means that this is unsafe to use > + * when bootmem is currently active (unless bootmem itself is implemented > + * on top of LMB which isn't the case yet) > + * > + * This should however not be an issue for now, as we currently only > + * call into LMB while it's still active, or much later when slab is > + * active for memory hotplug operations > + */ > + if (use_slab) { > + new_array = kmalloc(new_size, GFP_KERNEL); > + addr = new_array == NULL ? LMB_ERROR : __pa(new_array); > + } else > + addr = lmb_find_base(new_size, sizeof(phys_addr_t), LMB_ALLOC_ACCESSIBLE); > + if (addr == LMB_ERROR) { > + pr_err("lmb: Failed to double %s array from %ld to %ld entries !\n", > + lmb_type_name(type), type->max, type->max * 2); > + return -1; > + } > + new_array = __va(addr); > + > + /* Found space, we now need to move the array over before > + * we add the reserved region since it may be our reserved > + * array itself that is full. > + */ > + memcpy(new_array, type->regions, old_size); > + memset(new_array + type->max, 0, old_size); > + old_array = type->regions; > + type->regions = new_array; > + type->max <<= 1; > + > + /* If we use SLAB that's it, we are done */ > + if (use_slab) > + return 0; > + > + /* Add the new reserved region now. Should not fail ! */ > + BUG_ON(lmb_add_region(&lmb.reserved, addr, new_size) < 0); > + > + /* If the array wasn't our static init one, then free it. We only do > + * that before SLAB is available as later on, we don't know whether > + * to use kfree or free_bootmem_pages(). Shouldn't be a big deal > + * anyways > + */ > + if (old_array != lmb_memory_init_regions && > + old_array != lmb_reserved_init_regions) > + lmb_free(__pa(old_array), old_size); > + > + return 0; > +} > + > static long lmb_add_region(struct lmb_type *type, phys_addr_t base, phys_addr_t size) > { > unsigned long coalesced = 0; > @@ -196,7 +275,11 @@ static long lmb_add_region(struct lmb_type *type, phys_addr_t base, phys_addr_t > > if (coalesced) > return coalesced; > - if (type->cnt >= type->max) > + > + /* If we are out of space, we fail. It's too late to resize the array > + * but then this shouldn't have happened in the first place. > + */ > + if (WARN_ON(type->cnt >= type->max)) > return -1; > > /* Couldn't coalesce the LMB, so add it to the sorted table. */ > @@ -217,6 +300,14 @@ static long lmb_add_region(struct lmb_type *type, phys_addr_t base, phys_addr_t > } > type->cnt++; > > + /* The array is full ? Try to resize it. If that fails, we undo > + * our allocation and return an error > + */ > + if (type->cnt == type->max && lmb_double_array(type)) { you need to pass base, base+size with lmb_double_array() otherwise when you are using lmb_reserve(base, size), double_array() array could have chance to get new buffer that is overlapped with [base, base + size). to keep it simple, should check_double_array() after lmb_reserve, lmb_add, lmb_free (yes, that need it too). that was suggested by Michael Ellerman. YH -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 19/25] lmb: Add array resizing support 2010-05-10 23:59 ` [PATCH 19/25] lmb: Add array resizing support Yinghai Lu @ 2010-05-11 4:56 ` Benjamin Herrenschmidt 2010-05-11 17:54 ` Yinghai Lu 0 siblings, 1 reply; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-11 4:56 UTC (permalink / raw) To: Yinghai Lu; +Cc: linux-mm, linux-kernel, tglx, mingo, davem, lethal On Mon, 2010-05-10 at 16:59 -0700, Yinghai Lu wrote: > you need to pass base, base+size with lmb_double_array() > > otherwise when you are using lmb_reserve(base, size), double_array() > array could have chance to get > new buffer that is overlapped with [base, base + size). > > to keep it simple, should check_double_array() after lmb_reserve, > lmb_add, lmb_free (yes, that need it too). > that was suggested by Michael Ellerman. > No. You may notice that I addressed this problem by moving the call to lmb_double_array() to -after- we record the entry in the array, so it shouldn't be able to pickup the same one. I dislike the idea of sprinkling the check for resize everywhere at the top level. Cheers, Ben. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 19/25] lmb: Add array resizing support 2010-05-11 4:56 ` Benjamin Herrenschmidt @ 2010-05-11 17:54 ` Yinghai Lu 0 siblings, 0 replies; 37+ messages in thread From: Yinghai Lu @ 2010-05-11 17:54 UTC (permalink / raw) To: Benjamin Herrenschmidt; +Cc: linux-mm, linux-kernel, tglx, mingo, davem, lethal On Mon, May 10, 2010 at 9:56 PM, Benjamin Herrenschmidt <benh@kernel.crashing.org> wrote: > On Mon, 2010-05-10 at 16:59 -0700, Yinghai Lu wrote: >> you need to pass base, base+size with lmb_double_array() >> >> otherwise when you are using lmb_reserve(base, size), double_array() >> array could have chance to get >> new buffer that is overlapped with [base, base + size). >> >> to keep it simple, should check_double_array() after lmb_reserve, >> lmb_add, lmb_free (yes, that need it too). >> that was suggested by Michael Ellerman. >> > > No. You may notice that I addressed this problem by moving the > call to lmb_double_array() to -after- we record the entry in > the array, so it shouldn't be able to pickup the same one. oh, you are right. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 12/25] lmb: Move lmb arrays to static storage in lmb.c and make their size a variable 2010-05-10 9:38 ` [PATCH 12/25] lmb: Move lmb arrays to static storage in lmb.c and make their size a variable Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 13/25] lmb: Add debug markers at the end of the array Benjamin Herrenschmidt @ 2010-05-10 10:44 ` Paul Mundt 2010-05-10 11:46 ` Benjamin Herrenschmidt 1 sibling, 1 reply; 37+ messages in thread From: Paul Mundt @ 2010-05-10 10:44 UTC (permalink / raw) To: Benjamin Herrenschmidt; +Cc: linux-mm, linux-kernel, tglx, mingo, davem On Mon, May 10, 2010 at 07:38:46PM +1000, Benjamin Herrenschmidt wrote: > diff --git a/include/linux/lmb.h b/include/linux/lmb.h > index 27c2386..e575801 100644 > --- a/include/linux/lmb.h > +++ b/include/linux/lmb.h > @@ -18,7 +18,7 @@ > > #include <asm/lmb.h> > > -#define MAX_LMB_REGIONS 128 > +#define INIT_LMB_REGIONS 128 > > struct lmb_region { > phys_addr_t base; Perhaps it would be better to weight this against MAX_ACTIVE_REGIONS for the ARCH_POPULATES_NODE_MAP case? The early node map is already using that size, at least. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 12/25] lmb: Move lmb arrays to static storage in lmb.c and make their size a variable 2010-05-10 10:44 ` [PATCH 12/25] lmb: Move lmb arrays to static storage in lmb.c and make their size a variable Paul Mundt @ 2010-05-10 11:46 ` Benjamin Herrenschmidt 0 siblings, 0 replies; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 11:46 UTC (permalink / raw) To: Paul Mundt; +Cc: linux-mm, linux-kernel, tglx, mingo, davem On Mon, 2010-05-10 at 19:44 +0900, Paul Mundt wrote: > Perhaps it would be better to weight this against MAX_ACTIVE_REGIONS > for the ARCH_POPULATES_NODE_MAP case? The early node map is already > using that size, at least. Well, one of the next patches implement dynamic resize of the LMB array so I was actually thinking about shrinking the static one to a minimum... Cheers, Ben. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 05/25] lmb: Factor the lowest level alloc function 2010-05-10 9:38 ` [PATCH 05/25] lmb: Factor the lowest level alloc function Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 06/25] lmb: Expose LMB_ALLOC_ANYWHERE Benjamin Herrenschmidt @ 2010-05-11 12:30 ` Thomas Gleixner 2010-05-11 22:24 ` Benjamin Herrenschmidt 2010-05-12 5:24 ` Benjamin Herrenschmidt 1 sibling, 2 replies; 37+ messages in thread From: Thomas Gleixner @ 2010-05-11 12:30 UTC (permalink / raw) To: Benjamin Herrenschmidt; +Cc: linux-mm, linux-kernel, tglx, mingo, davem, lethal On Mon, 10 May 2010, Benjamin Herrenschmidt wrote: > > @@ -396,33 +406,24 @@ u64 __init __lmb_alloc_base(u64 size, u64 align, u64 max_addr) > if (max_addr == LMB_ALLOC_ANYWHERE) > max_addr = LMB_REAL_LIMIT; > > + /* Pump up max_addr */ > + if (max_addr == LMB_ALLOC_ANYWHERE) > + max_addr = ~(u64)0; > + That if is pretty useless as you set max_addr to LMB_REAL_LIMIT right above. Thanks, tglx -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 05/25] lmb: Factor the lowest level alloc function 2010-05-11 12:30 ` [PATCH 05/25] lmb: Factor the lowest level alloc function Thomas Gleixner @ 2010-05-11 22:24 ` Benjamin Herrenschmidt 2010-05-12 5:24 ` Benjamin Herrenschmidt 1 sibling, 0 replies; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-11 22:24 UTC (permalink / raw) To: Thomas Gleixner; +Cc: linux-mm, linux-kernel, tglx, mingo, davem, lethal On Tue, 2010-05-11 at 14:30 +0200, Thomas Gleixner wrote: > > @@ -396,33 +406,24 @@ u64 __init __lmb_alloc_base(u64 size, u64 > align, u64 max_addr) > > if (max_addr == LMB_ALLOC_ANYWHERE) > > max_addr = LMB_REAL_LIMIT; > > > > + /* Pump up max_addr */ > > + if (max_addr == LMB_ALLOC_ANYWHERE) > > + max_addr = ~(u64)0; > > + > > That if is pretty useless as you set max_addr to LMB_REAL_LIMIT > right above. This is a mismerge/mis-rebase of one of my patches actually. I'll dbl check what's up but I think the first hunk should go along with LMB_REAL_LIMIT in favor of the new default limit thing. But we first need to make sure the default is set sensibly and I haven't fixed all archs yet. I'll dbl check what's up there. Cheers, Benm. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 05/25] lmb: Factor the lowest level alloc function 2010-05-11 12:30 ` [PATCH 05/25] lmb: Factor the lowest level alloc function Thomas Gleixner 2010-05-11 22:24 ` Benjamin Herrenschmidt @ 2010-05-12 5:24 ` Benjamin Herrenschmidt 1 sibling, 0 replies; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-12 5:24 UTC (permalink / raw) To: Thomas Gleixner; +Cc: linux-mm, linux-kernel, tglx, mingo, davem, lethal On Tue, 2010-05-11 at 14:30 +0200, Thomas Gleixner wrote: > On Mon, 10 May 2010, Benjamin Herrenschmidt wrote: > > > > @@ -396,33 +406,24 @@ u64 __init __lmb_alloc_base(u64 size, u64 align, u64 max_addr) > > if (max_addr == LMB_ALLOC_ANYWHERE) > > max_addr = LMB_REAL_LIMIT; > > > > + /* Pump up max_addr */ > > + if (max_addr == LMB_ALLOC_ANYWHERE) > > + max_addr = ~(u64)0; > > + > > That if is pretty useless as you set max_addr to LMB_REAL_LIMIT > right above. Well, actually no :-) LMB_REAL_LIMIT can actually be == to LMB_ALLOC_ANYWHERE which at this stage in the series is still 0, so we must not miss the second bit. Subsequent patches remove this anyways as LMB_ALLOC_ANYWHERE gets turned into ~0 which makes more sense. So it's a bit weird looking, but it's correct and transitory code only. Cheers, Ben. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 03/25] lmb: Introduce for_each_lmb() and new accessors, and use it 2010-05-10 9:38 ` [PATCH 03/25] lmb: Introduce for_each_lmb() and new accessors, and use it Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 04/25] lmb: Remove nid_range argument, arch provides lmb_nid_range() instead Benjamin Herrenschmidt @ 2010-05-10 15:39 ` Thomas Gleixner 2010-05-10 21:52 ` Benjamin Herrenschmidt 2010-05-12 5:20 ` Benjamin Herrenschmidt 1 sibling, 2 replies; 37+ messages in thread From: Thomas Gleixner @ 2010-05-10 15:39 UTC (permalink / raw) To: Benjamin Herrenschmidt; +Cc: linux-mm, linux-kernel, tglx, mingo, davem, lethal On Mon, 10 May 2010, Benjamin Herrenschmidt wrote: > > +#define for_each_lmb(lmb_type, region) \ > + for (reg = lmb.lmb_type.regions; \ > + region < (lmb.lmb_type.regions + lmb.lmb_type.cnt); \ > + region++) > + Can you please make this: #define for_each_lmb(lmb_type, region, __iter) \ for (__iter = lmb.lmb_type.regions; \ region < (lmb.lmb_type.regions + lmb.lmb_type.cnt); \ region++) so we do not have the iterator name hardcoded inside the macro body. Thanks, tglx -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 03/25] lmb: Introduce for_each_lmb() and new accessors, and use it 2010-05-10 15:39 ` [PATCH 03/25] lmb: Introduce for_each_lmb() and new accessors, and use it Thomas Gleixner @ 2010-05-10 21:52 ` Benjamin Herrenschmidt 2010-05-12 5:20 ` Benjamin Herrenschmidt 1 sibling, 0 replies; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-10 21:52 UTC (permalink / raw) To: Thomas Gleixner; +Cc: linux-mm, linux-kernel, tglx, mingo, davem, lethal On Mon, 2010-05-10 at 17:39 +0200, Thomas Gleixner wrote: > Can you please make this: > > #define for_each_lmb(lmb_type, region, __iter) \ > for (__iter = lmb.lmb_type.regions; \ > region < (lmb.lmb_type.regions + lmb.lmb_type.cnt); \ > region++) > > so we do not have the iterator name hardcoded inside the macro body. Oops, you are right, that's a thinko. I'll fix that. Cheers, Ben. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 03/25] lmb: Introduce for_each_lmb() and new accessors, and use it 2010-05-10 15:39 ` [PATCH 03/25] lmb: Introduce for_each_lmb() and new accessors, and use it Thomas Gleixner 2010-05-10 21:52 ` Benjamin Herrenschmidt @ 2010-05-12 5:20 ` Benjamin Herrenschmidt 1 sibling, 0 replies; 37+ messages in thread From: Benjamin Herrenschmidt @ 2010-05-12 5:20 UTC (permalink / raw) To: Thomas Gleixner; +Cc: linux-mm, linux-kernel, tglx, mingo, davem, lethal On Mon, 2010-05-10 at 17:39 +0200, Thomas Gleixner wrote: > > +#define for_each_lmb(lmb_type, region) \ > > + for (reg = lmb.lmb_type.regions; \ > > + region < (lmb.lmb_type.regions + lmb.lmb_type.cnt); \ > > + region++) > > + > > Can you please make this: > > #define for_each_lmb(lmb_type, region, __iter) \ > for (__iter = lmb.lmb_type.regions; \ > region < (lmb.lmb_type.regions + lmb.lmb_type.cnt); \ > region++) > > so we do not have the iterator name hardcoded inside the macro body. As a matter of fact, we didn't, but I made a typo on the first one: - for (reg = lmb.lmb_type.regions; \ + for (region = lmb.lmb_type.regions; \ region < (lmb.lmb_type.regions + lmb.lmb_type.cnt); \ region++) I'll fix that in my series. Cheers, Ben. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 37+ messages in thread
end of thread, other threads:[~2010-05-12 5:24 UTC | newest] Thread overview: 37+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2010-05-10 9:38 [PATCH/WIP] lmb cleanups and additions Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 01/25] lmb: Rename lmb_region to lmb_type and lmb_property to lmb_region Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 02/25] lmb: No reason to include asm/lmb.h late Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 03/25] lmb: Introduce for_each_lmb() and new accessors, and use it Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 04/25] lmb: Remove nid_range argument, arch provides lmb_nid_range() instead Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 05/25] lmb: Factor the lowest level alloc function Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 06/25] lmb: Expose LMB_ALLOC_ANYWHERE Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 07/25] lmb: Introduce default allocation limit and use it to replace explicit ones Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 08/25] lmb: Remove rmo_size, burry it in arch/powerpc where it belongs Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 09/25] lmb: Change u64 to phys_addr_t Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 10/25] lmb: Remove unused lmb.debug struct member Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 11/25] lmb: Remove lmb_type.size and add lmb.memory_size instead Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 12/25] lmb: Move lmb arrays to static storage in lmb.c and make their size a variable Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 13/25] lmb: Add debug markers at the end of the array Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 14/25] lmb: Make lmb_find_region() out of lmb_alloc_region() Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 15/25] lmb: Define LMB_ERROR internally instead of using ~(phys_addr_t)0 Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 16/25] lmb: Move lmb_init() to the bottom of the file Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 17/25] lmb: split lmb_find_base() out of __lmb_alloc_base() Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 18/25] lmb: Move functions around into a more sensible order Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 19/25] lmb: Add array resizing support Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 20/25] lmb: Add arch function to control coalescing of lmb memory regions Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 21/25] lmb: Add "start" argument to lmb_find_base() Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 22/25] lmb: NUMA allocate can now use early_pfn_map Benjamin Herrenschmidt 2010-05-10 9:38 ` [PATCH 23/25] lmb: Separate lmb_alloc_nid() and lmb_alloc_try_nid() Benjamin Herrenschmidt 2010-05-10 23:37 ` [PATCH 21/25] lmb: Add "start" argument to lmb_find_base() Yinghai Lu 2010-05-11 4:56 ` Benjamin Herrenschmidt 2010-05-10 23:59 ` [PATCH 19/25] lmb: Add array resizing support Yinghai Lu 2010-05-11 4:56 ` Benjamin Herrenschmidt 2010-05-11 17:54 ` Yinghai Lu 2010-05-10 10:44 ` [PATCH 12/25] lmb: Move lmb arrays to static storage in lmb.c and make their size a variable Paul Mundt 2010-05-10 11:46 ` Benjamin Herrenschmidt 2010-05-11 12:30 ` [PATCH 05/25] lmb: Factor the lowest level alloc function Thomas Gleixner 2010-05-11 22:24 ` Benjamin Herrenschmidt 2010-05-12 5:24 ` Benjamin Herrenschmidt 2010-05-10 15:39 ` [PATCH 03/25] lmb: Introduce for_each_lmb() and new accessors, and use it Thomas Gleixner 2010-05-10 21:52 ` Benjamin Herrenschmidt 2010-05-12 5:20 ` Benjamin Herrenschmidt
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).