* [PATCH] mm: Do not walk all of system memory during show_mem @ 2013-10-16 10:42 Mel Gorman 2013-10-17 1:11 ` David Rientjes 2013-10-31 4:55 ` [PATCH] mm: Do not walk all of system memory during show_mem KOSAKI Motohiro 0 siblings, 2 replies; 6+ messages in thread From: Mel Gorman @ 2013-10-16 10:42 UTC (permalink / raw) To: Andrew Morton; +Cc: Linux-MM, LKML It has been reported on very large machines that show_mem is taking almost 5 minutes to display information. This is a serious problem if there is an OOM storm. The bulk of the cost is in show_mem doing a very expensive PFN walk to give us the following information Total RAM: Also available as totalram_pages Highmem pages: Also available as totalhigh_pages Reserved pages: Can be inferred from the zone structure Shared pages: PFN walk required Unshared pages: PFN walk required Quick pages: Per-cpu walk required Only the shared/unshared pages requires a full PFN walk but that information is useless. It is also inaccurate as page pins of unshared pages would be accounted for as shared. Even if the information was accurate, I'm struggling to think how the shared/unshared information could be useful for debugging OOM conditions. Maybe it was useful before rmap existed when reclaiming shared pages was costly but it is less relevant today. The PFN walk could be optimised a bit but why bother as the information is useless. This patch deletes the PFN walker and infers the total RAM, highmem and reserved pages count from struct zone. It omits the shared/unshared page usage on the grounds that it is useless. It also corrects the reporting of HighMem as HighMem/MovableOnly as ZONE_MOVABLE has similar problems to HighMem with respect to lowmem/highmem exhaustion. Signed-off-by: Mel Gorman <mgorman@suse.de> --- lib/show_mem.c | 39 +++++++++++---------------------------- 1 file changed, 11 insertions(+), 28 deletions(-) diff --git a/lib/show_mem.c b/lib/show_mem.c index b7c7231..5847a49 100644 --- a/lib/show_mem.c +++ b/lib/show_mem.c @@ -12,8 +12,7 @@ void show_mem(unsigned int filter) { pg_data_t *pgdat; - unsigned long total = 0, reserved = 0, shared = 0, - nonshared = 0, highmem = 0; + unsigned long total = 0, reserved = 0, highmem = 0; printk("Mem-Info:\n"); show_free_areas(filter); @@ -22,43 +21,27 @@ void show_mem(unsigned int filter) return; for_each_online_pgdat(pgdat) { - unsigned long i, flags; + unsigned long flags; + int zoneid; pgdat_resize_lock(pgdat, &flags); - for (i = 0; i < pgdat->node_spanned_pages; i++) { - struct page *page; - unsigned long pfn = pgdat->node_start_pfn + i; - - if (unlikely(!(i % MAX_ORDER_NR_PAGES))) - touch_nmi_watchdog(); - - if (!pfn_valid(pfn)) + for (zoneid = 0; zoneid < MAX_NR_ZONES; zoneid++) { + struct zone *zone = &pgdat->node_zones[zoneid]; + if (!populated_zone(zone)) continue; - page = pfn_to_page(pfn); - - if (PageHighMem(page)) - highmem++; + total += zone->present_pages; + reserved = zone->present_pages - zone->managed_pages; - if (PageReserved(page)) - reserved++; - else if (page_count(page) == 1) - nonshared++; - else if (page_count(page) > 1) - shared += page_count(page) - 1; - - total++; + if (is_highmem_idx(zoneid)) + highmem += zone->present_pages; } pgdat_resize_unlock(pgdat, &flags); } printk("%lu pages RAM\n", total); -#ifdef CONFIG_HIGHMEM - printk("%lu pages HighMem\n", highmem); -#endif + printk("%lu pages HighMem/MovableOnly\n", highmem); printk("%lu pages reserved\n", reserved); - printk("%lu pages shared\n", shared); - printk("%lu pages non-shared\n", nonshared); #ifdef CONFIG_QUICKLIST printk("%lu pages in pagetable cache\n", quicklist_total_size()); -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] mm: Do not walk all of system memory during show_mem 2013-10-16 10:42 [PATCH] mm: Do not walk all of system memory during show_mem Mel Gorman @ 2013-10-17 1:11 ` David Rientjes 2013-11-04 10:08 ` Mel Gorman 2013-12-03 14:57 ` [PATCH] mm, show_mem: Remove SHOW_MEM_FILTER_PAGE_COUNT Mel Gorman 2013-10-31 4:55 ` [PATCH] mm: Do not walk all of system memory during show_mem KOSAKI Motohiro 1 sibling, 2 replies; 6+ messages in thread From: David Rientjes @ 2013-10-17 1:11 UTC (permalink / raw) To: Mel Gorman; +Cc: Andrew Morton, Linux-MM, LKML On Wed, 16 Oct 2013, Mel Gorman wrote: > It has been reported on very large machines that show_mem is taking almost > 5 minutes to display information. This is a serious problem if there is > an OOM storm. The bulk of the cost is in show_mem doing a very expensive > PFN walk to give us the following information > > Total RAM: Also available as totalram_pages > Highmem pages: Also available as totalhigh_pages > Reserved pages: Can be inferred from the zone structure > Shared pages: PFN walk required > Unshared pages: PFN walk required > Quick pages: Per-cpu walk required > > Only the shared/unshared pages requires a full PFN walk but that information > is useless. It is also inaccurate as page pins of unshared pages would > be accounted for as shared. Even if the information was accurate, I'm > struggling to think how the shared/unshared information could be useful > for debugging OOM conditions. Maybe it was useful before rmap existed when > reclaiming shared pages was costly but it is less relevant today. > > The PFN walk could be optimised a bit but why bother as the information is > useless. This patch deletes the PFN walker and infers the total RAM, highmem > and reserved pages count from struct zone. It omits the shared/unshared page > usage on the grounds that it is useless. It also corrects the reporting > of HighMem as HighMem/MovableOnly as ZONE_MOVABLE has similar problems to > HighMem with respect to lowmem/highmem exhaustion. > We haven't been hit by this for the oom killer, but we did get hit with this for page allocation failure warnings as a result of having irqs disabled and passing GFP_ATOMIC to the page allocator without GFP_NOWARN. That was the intention of passing SHOW_MEM_FILTER_PAGE_COUNT into show_mem() in 4b59e6c47309 ("mm, show_mem: suppress page counts in non-blockable contexts"). With this, I assume we can just remove SHOW_MEM_FILTER_PAGE_COUNT entirely? -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] mm: Do not walk all of system memory during show_mem 2013-10-17 1:11 ` David Rientjes @ 2013-11-04 10:08 ` Mel Gorman 2013-12-03 14:57 ` [PATCH] mm, show_mem: Remove SHOW_MEM_FILTER_PAGE_COUNT Mel Gorman 1 sibling, 0 replies; 6+ messages in thread From: Mel Gorman @ 2013-11-04 10:08 UTC (permalink / raw) To: David Rientjes; +Cc: Andrew Morton, Linux-MM, LKML On Wed, Oct 16, 2013 at 06:11:56PM -0700, David Rientjes wrote: > On Wed, 16 Oct 2013, Mel Gorman wrote: > > > It has been reported on very large machines that show_mem is taking almost > > 5 minutes to display information. This is a serious problem if there is > > an OOM storm. The bulk of the cost is in show_mem doing a very expensive > > PFN walk to give us the following information > > > > Total RAM: Also available as totalram_pages > > Highmem pages: Also available as totalhigh_pages > > Reserved pages: Can be inferred from the zone structure > > Shared pages: PFN walk required > > Unshared pages: PFN walk required > > Quick pages: Per-cpu walk required > > > > Only the shared/unshared pages requires a full PFN walk but that information > > is useless. It is also inaccurate as page pins of unshared pages would > > be accounted for as shared. Even if the information was accurate, I'm > > struggling to think how the shared/unshared information could be useful > > for debugging OOM conditions. Maybe it was useful before rmap existed when > > reclaiming shared pages was costly but it is less relevant today. > > > > The PFN walk could be optimised a bit but why bother as the information is > > useless. This patch deletes the PFN walker and infers the total RAM, highmem > > and reserved pages count from struct zone. It omits the shared/unshared page > > usage on the grounds that it is useless. It also corrects the reporting > > of HighMem as HighMem/MovableOnly as ZONE_MOVABLE has similar problems to > > HighMem with respect to lowmem/highmem exhaustion. > > > > We haven't been hit by this for the oom killer, but we did get hit with > this for page allocation failure warnings as a result of having irqs > disabled and passing GFP_ATOMIC to the page allocator without GFP_NOWARN. > That was the intention of passing SHOW_MEM_FILTER_PAGE_COUNT into > show_mem() in 4b59e6c47309 ("mm, show_mem: suppress page counts in > non-blockable contexts"). > > With this, I assume we can just remove SHOW_MEM_FILTER_PAGE_COUNT > entirely? We could once all the per-arch show_mem functions were updated similar to lib/show_mem.c. I've added a todo item to do just that. Thanks. -- Mel Gorman SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH] mm, show_mem: Remove SHOW_MEM_FILTER_PAGE_COUNT 2013-10-17 1:11 ` David Rientjes 2013-11-04 10:08 ` Mel Gorman @ 2013-12-03 14:57 ` Mel Gorman 2013-12-03 23:41 ` David Rientjes 1 sibling, 1 reply; 6+ messages in thread From: Mel Gorman @ 2013-12-03 14:57 UTC (permalink / raw) To: Andrew Morton Cc: David Rientjes, Tony Luck, Russell King, James Bottomley, linux-arm-kernel, linux-ia64, linux-parisc, Linux-MM, LKML Commit 4b59e6c4 (mm, show_mem: suppress page counts in non-blockable contexts) introduced SHOW_MEM_FILTER_PAGE_COUNT to suppress PFN walks on large memory machines. Commit c78e9363 (mm: do not walk all of system memory during show_mem) avoided a PFN walk in the generic show_mem helper which removes the requirement for SHOW_MEM_FILTER_PAGE_COUNT in that case. This patch removes PFN walkers from the arch-specific implementations that report on a per-node or per-zone granularity. ARM and unicore32 still do a PFN walk as they report memory usage on each bank which is a much finer granularity where the debugging information may still be of use. As the remaining arches doing PFN walks have relatively small amounts of memory, this patch simply removes SHOW_MEM_FILTER_PAGE_COUNT. Signed-off-by: Mel Gorman <mgorman@suse.de> --- arch/arm/mm/init.c | 3 --- arch/ia64/mm/contig.c | 68 ------------------------------------------------ arch/ia64/mm/discontig.c | 63 -------------------------------------------- arch/ia64/mm/init.c | 48 ++++++++++++++++++++++++++++++++++ arch/parisc/mm/init.c | 62 ++++++++++++++----------------------------- arch/unicore32/mm/init.c | 3 --- include/linux/mm.h | 1 - lib/show_mem.c | 3 --- mm/page_alloc.c | 7 ----- 9 files changed, 68 insertions(+), 190 deletions(-) diff --git a/arch/arm/mm/init.c b/arch/arm/mm/init.c index 3e8f106..2e71e24 100644 --- a/arch/arm/mm/init.c +++ b/arch/arm/mm/init.c @@ -92,9 +92,6 @@ void show_mem(unsigned int filter) printk("Mem-info:\n"); show_free_areas(filter); - if (filter & SHOW_MEM_FILTER_PAGE_COUNT) - return; - for_each_bank (i, mi) { struct membank *bank = &mi->bank[i]; unsigned int pfn1, pfn2; diff --git a/arch/ia64/mm/contig.c b/arch/ia64/mm/contig.c index da5237d..52715a7 100644 --- a/arch/ia64/mm/contig.c +++ b/arch/ia64/mm/contig.c @@ -31,74 +31,6 @@ static unsigned long max_gap; #endif -/** - * show_mem - give short summary of memory stats - * - * Shows a simple page count of reserved and used pages in the system. - * For discontig machines, it does this on a per-pgdat basis. - */ -void show_mem(unsigned int filter) -{ - int i, total_reserved = 0; - int total_shared = 0, total_cached = 0; - unsigned long total_present = 0; - pg_data_t *pgdat; - - printk(KERN_INFO "Mem-info:\n"); - show_free_areas(filter); - printk(KERN_INFO "Node memory in pages:\n"); - if (filter & SHOW_MEM_FILTER_PAGE_COUNT) - return; - for_each_online_pgdat(pgdat) { - unsigned long present; - unsigned long flags; - int shared = 0, cached = 0, reserved = 0; - int nid = pgdat->node_id; - - if (skip_free_areas_node(filter, nid)) - continue; - pgdat_resize_lock(pgdat, &flags); - present = pgdat->node_present_pages; - for(i = 0; i < pgdat->node_spanned_pages; i++) { - struct page *page; - if (unlikely(i % MAX_ORDER_NR_PAGES == 0)) - touch_nmi_watchdog(); - if (pfn_valid(pgdat->node_start_pfn + i)) - page = pfn_to_page(pgdat->node_start_pfn + i); - else { -#ifdef CONFIG_VIRTUAL_MEM_MAP - if (max_gap < LARGE_GAP) - continue; -#endif - i = vmemmap_find_next_valid_pfn(nid, i) - 1; - continue; - } - if (PageReserved(page)) - reserved++; - else if (PageSwapCache(page)) - cached++; - else if (page_count(page)) - shared += page_count(page)-1; - } - pgdat_resize_unlock(pgdat, &flags); - total_present += present; - total_reserved += reserved; - total_cached += cached; - total_shared += shared; - printk(KERN_INFO "Node %4d: RAM: %11ld, rsvd: %8d, " - "shrd: %10d, swpd: %10d\n", nid, - present, reserved, shared, cached); - } - printk(KERN_INFO "%ld pages of RAM\n", total_present); - printk(KERN_INFO "%d reserved pages\n", total_reserved); - printk(KERN_INFO "%d pages shared\n", total_shared); - printk(KERN_INFO "%d pages swap cached\n", total_cached); - printk(KERN_INFO "Total of %ld pages in page table cache\n", - quicklist_total_size()); - printk(KERN_INFO "%ld free buffer pages\n", nr_free_buffer_pages()); -} - - /* physical address where the bootmem map is located */ unsigned long bootmap_start; diff --git a/arch/ia64/mm/discontig.c b/arch/ia64/mm/discontig.c index 2de08f4..8786268 100644 --- a/arch/ia64/mm/discontig.c +++ b/arch/ia64/mm/discontig.c @@ -608,69 +608,6 @@ void *per_cpu_init(void) #endif /* CONFIG_SMP */ /** - * show_mem - give short summary of memory stats - * - * Shows a simple page count of reserved and used pages in the system. - * For discontig machines, it does this on a per-pgdat basis. - */ -void show_mem(unsigned int filter) -{ - int i, total_reserved = 0; - int total_shared = 0, total_cached = 0; - unsigned long total_present = 0; - pg_data_t *pgdat; - - printk(KERN_INFO "Mem-info:\n"); - show_free_areas(filter); - if (filter & SHOW_MEM_FILTER_PAGE_COUNT) - return; - printk(KERN_INFO "Node memory in pages:\n"); - for_each_online_pgdat(pgdat) { - unsigned long present; - unsigned long flags; - int shared = 0, cached = 0, reserved = 0; - int nid = pgdat->node_id; - - if (skip_free_areas_node(filter, nid)) - continue; - pgdat_resize_lock(pgdat, &flags); - present = pgdat->node_present_pages; - for(i = 0; i < pgdat->node_spanned_pages; i++) { - struct page *page; - if (unlikely(i % MAX_ORDER_NR_PAGES == 0)) - touch_nmi_watchdog(); - if (pfn_valid(pgdat->node_start_pfn + i)) - page = pfn_to_page(pgdat->node_start_pfn + i); - else { - i = vmemmap_find_next_valid_pfn(nid, i) - 1; - continue; - } - if (PageReserved(page)) - reserved++; - else if (PageSwapCache(page)) - cached++; - else if (page_count(page)) - shared += page_count(page)-1; - } - pgdat_resize_unlock(pgdat, &flags); - total_present += present; - total_reserved += reserved; - total_cached += cached; - total_shared += shared; - printk(KERN_INFO "Node %4d: RAM: %11ld, rsvd: %8d, " - "shrd: %10d, swpd: %10d\n", nid, - present, reserved, shared, cached); - } - printk(KERN_INFO "%ld pages of RAM\n", total_present); - printk(KERN_INFO "%d reserved pages\n", total_reserved); - printk(KERN_INFO "%d pages shared\n", total_shared); - printk(KERN_INFO "%d pages swap cached\n", total_cached); - printk(KERN_INFO "Total of %ld pages in page table cache\n", - quicklist_total_size()); - printk(KERN_INFO "%ld free buffer pages\n", nr_free_buffer_pages()); -} - -/** * call_pernode_memory - use SRAT to call callback functions with node info * @start: physical start of range * @len: length of range diff --git a/arch/ia64/mm/init.c b/arch/ia64/mm/init.c index 88504ab..25c3502 100644 --- a/arch/ia64/mm/init.c +++ b/arch/ia64/mm/init.c @@ -684,3 +684,51 @@ per_linux32_init(void) } __initcall(per_linux32_init); + +/** + * show_mem - give short summary of memory stats + * + * Shows a simple page count of reserved and used pages in the system. + * For discontig machines, it does this on a per-pgdat basis. + */ +void show_mem(unsigned int filter) +{ + int total_reserved = 0; + unsigned long total_present = 0; + pg_data_t *pgdat; + + printk(KERN_INFO "Mem-info:\n"); + show_free_areas(filter); + printk(KERN_INFO "Node memory in pages:\n"); + for_each_online_pgdat(pgdat) { + unsigned long present; + unsigned long flags; + int reserved = 0; + int nid = pgdat->node_id; + int zoneid; + + if (skip_free_areas_node(filter, nid)) + continue; + pgdat_resize_lock(pgdat, &flags); + + for (zoneid = 0; zoneid < MAX_NR_ZONES; zoneid++) { + struct zone *zone = &pgdat->node_zones[zoneid]; + if (!populated_zone(zone)) + continue; + + reserved += zone->present_pages - zone->managed_pages; + } + present = pgdat->node_present_pages; + + pgdat_resize_unlock(pgdat, &flags); + total_present += present; + total_reserved += reserved; + printk(KERN_INFO "Node %4d: RAM: %11ld, rsvd: %8d, ", + nid, present, reserved); + } + printk(KERN_INFO "%ld pages of RAM\n", total_present); + printk(KERN_INFO "%d reserved pages\n", total_reserved); + printk(KERN_INFO "Total of %ld pages in page table cache\n", + quicklist_total_size()); + printk(KERN_INFO "%ld free buffer pages\n", nr_free_buffer_pages()); +} diff --git a/arch/parisc/mm/init.c b/arch/parisc/mm/init.c index b0f96c0..e12d5a7 100644 --- a/arch/parisc/mm/init.c +++ b/arch/parisc/mm/init.c @@ -632,55 +632,33 @@ EXPORT_SYMBOL(empty_zero_page); void show_mem(unsigned int filter) { - int i,free = 0,total = 0,reserved = 0; - int shared = 0, cached = 0; + int total = 0,reserved = 0; + pg_data_t *pgdat; printk(KERN_INFO "Mem-info:\n"); show_free_areas(filter); - if (filter & SHOW_MEM_FILTER_PAGE_COUNT) - return; -#ifndef CONFIG_DISCONTIGMEM - i = max_mapnr; - while (i-- > 0) { - total++; - if (PageReserved(mem_map+i)) - reserved++; - else if (PageSwapCache(mem_map+i)) - cached++; - else if (!page_count(&mem_map[i])) - free++; - else - shared += page_count(&mem_map[i]) - 1; - } -#else - for (i = 0; i < npmem_ranges; i++) { - int j; - for (j = node_start_pfn(i); j < node_end_pfn(i); j++) { - struct page *p; - unsigned long flags; - - pgdat_resize_lock(NODE_DATA(i), &flags); - p = nid_page_nr(i, j) - node_start_pfn(i); - - total++; - if (PageReserved(p)) - reserved++; - else if (PageSwapCache(p)) - cached++; - else if (!page_count(p)) - free++; - else - shared += page_count(p) - 1; - pgdat_resize_unlock(NODE_DATA(i), &flags); - } + for_each_online_pgdat(pgdat) { + unsigned long flags; + int zoneid; + + pgdat_resize_lock(pgdat, &flags); + for (zoneid = 0; zoneid < MAX_NR_ZONES; zoneid++) { + struct zone *zone = &pgdat->node_zones[zoneid]; + if (!populated_zone(zone)) + continue; + + total += zone->present_pages; + reserved = zone->present_pages - zone->managed_pages; + + if (is_highmem_idx(zoneid)) + highmem += zone->present_pages; + } + pgdat_resize_unlock(pgdat, &flags); } -#endif + printk(KERN_INFO "%d pages of RAM\n", total); printk(KERN_INFO "%d reserved pages\n", reserved); - printk(KERN_INFO "%d pages shared\n", shared); - printk(KERN_INFO "%d pages swap cached\n", cached); - #ifdef CONFIG_DISCONTIGMEM { diff --git a/arch/unicore32/mm/init.c b/arch/unicore32/mm/init.c index ae6bc03..be2bde9 100644 --- a/arch/unicore32/mm/init.c +++ b/arch/unicore32/mm/init.c @@ -66,9 +66,6 @@ void show_mem(unsigned int filter) printk(KERN_DEFAULT "Mem-info:\n"); show_free_areas(filter); - if (filter & SHOW_MEM_FILTER_PAGE_COUNT) - return; - for_each_bank(i, mi) { struct membank *bank = &mi->bank[i]; unsigned int pfn1, pfn2; diff --git a/include/linux/mm.h b/include/linux/mm.h index 1cedd00..42b2486 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -984,7 +984,6 @@ extern void pagefault_out_of_memory(void); * various contexts. */ #define SHOW_MEM_FILTER_NODES (0x0001u) /* disallowed nodes */ -#define SHOW_MEM_FILTER_PAGE_COUNT (0x0002u) /* page type count */ extern void show_free_areas(unsigned int flags); extern bool skip_free_areas_node(unsigned int flags, int nid); diff --git a/lib/show_mem.c b/lib/show_mem.c index 5847a49..f58689f 100644 --- a/lib/show_mem.c +++ b/lib/show_mem.c @@ -17,9 +17,6 @@ void show_mem(unsigned int filter) printk("Mem-Info:\n"); show_free_areas(filter); - if (filter & SHOW_MEM_FILTER_PAGE_COUNT) - return; - for_each_online_pgdat(pgdat) { unsigned long flags; int zoneid; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 580a5f0..dcb61e8 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2073,13 +2073,6 @@ void warn_alloc_failed(gfp_t gfp_mask, int order, const char *fmt, ...) return; /* - * Walking all memory to count page types is very expensive and should - * be inhibited in non-blockable contexts. - */ - if (!(gfp_mask & __GFP_WAIT)) - filter |= SHOW_MEM_FILTER_PAGE_COUNT; - - /* * This documents exceptions given to allocations in certain * contexts that are allowed to allocate outside current's set * of allowed nodes. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] mm, show_mem: Remove SHOW_MEM_FILTER_PAGE_COUNT 2013-12-03 14:57 ` [PATCH] mm, show_mem: Remove SHOW_MEM_FILTER_PAGE_COUNT Mel Gorman @ 2013-12-03 23:41 ` David Rientjes 0 siblings, 0 replies; 6+ messages in thread From: David Rientjes @ 2013-12-03 23:41 UTC (permalink / raw) To: Mel Gorman Cc: Andrew Morton, Tony Luck, Russell King, James Bottomley, linux-arm-kernel, linux-ia64, linux-parisc, Linux-MM, LKML On Tue, 3 Dec 2013, Mel Gorman wrote: > Commit 4b59e6c4 (mm, show_mem: suppress page counts in non-blockable > contexts) introduced SHOW_MEM_FILTER_PAGE_COUNT to suppress PFN walks > on large memory machines. Commit c78e9363 (mm: do not walk all of system > memory during show_mem) avoided a PFN walk in the generic show_mem helper > which removes the requirement for SHOW_MEM_FILTER_PAGE_COUNT in that case. > > This patch removes PFN walkers from the arch-specific implementations that > report on a per-node or per-zone granularity. ARM and unicore32 still do > a PFN walk as they report memory usage on each bank which is a much finer > granularity where the debugging information may still be of use. As the > remaining arches doing PFN walks have relatively small amounts of memory, > this patch simply removes SHOW_MEM_FILTER_PAGE_COUNT. > > Signed-off-by: Mel Gorman <mgorman@suse.de> Acked-by: David Rientjes <rientjes@google.com> Thanks! -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] mm: Do not walk all of system memory during show_mem 2013-10-16 10:42 [PATCH] mm: Do not walk all of system memory during show_mem Mel Gorman 2013-10-17 1:11 ` David Rientjes @ 2013-10-31 4:55 ` KOSAKI Motohiro 1 sibling, 0 replies; 6+ messages in thread From: KOSAKI Motohiro @ 2013-10-31 4:55 UTC (permalink / raw) To: Mel Gorman, Andrew Morton; +Cc: kosaki.motohiro, Linux-MM, LKML (10/16/13 6:42 AM), Mel Gorman wrote: > It has been reported on very large machines that show_mem is taking almost > 5 minutes to display information. This is a serious problem if there is > an OOM storm. The bulk of the cost is in show_mem doing a very expensive > PFN walk to give us the following information > > Total RAM: Also available as totalram_pages > Highmem pages: Also available as totalhigh_pages > Reserved pages: Can be inferred from the zone structure > Shared pages: PFN walk required > Unshared pages: PFN walk required > Quick pages: Per-cpu walk required > > Only the shared/unshared pages requires a full PFN walk but that information > is useless. It is also inaccurate as page pins of unshared pages would > be accounted for as shared. Even if the information was accurate, I'm > struggling to think how the shared/unshared information could be useful > for debugging OOM conditions. Maybe it was useful before rmap existed when > reclaiming shared pages was costly but it is less relevant today. > > The PFN walk could be optimised a bit but why bother as the information is > useless. This patch deletes the PFN walker and infers the total RAM, highmem > and reserved pages count from struct zone. It omits the shared/unshared page > usage on the grounds that it is useless. It also corrects the reporting > of HighMem as HighMem/MovableOnly as ZONE_MOVABLE has similar problems to > HighMem with respect to lowmem/highmem exhaustion. > > Signed-off-by: Mel Gorman <mgorman@suse.de> That's ok. I haven't used such information on my long oom debugging history. Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-12-03 23:41 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-10-16 10:42 [PATCH] mm: Do not walk all of system memory during show_mem Mel Gorman 2013-10-17 1:11 ` David Rientjes 2013-11-04 10:08 ` Mel Gorman 2013-12-03 14:57 ` [PATCH] mm, show_mem: Remove SHOW_MEM_FILTER_PAGE_COUNT Mel Gorman 2013-12-03 23:41 ` David Rientjes 2013-10-31 4:55 ` [PATCH] mm: Do not walk all of system memory during show_mem KOSAKI Motohiro
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).