From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail202.messagelabs.com (mail202.messagelabs.com [216.82.254.227]) by kanga.kvack.org (Postfix) with SMTP id 325EA6B004F for ; Mon, 13 Jul 2009 01:36:46 -0400 (EDT) Received: from m4.gw.fujitsu.co.jp ([10.0.50.74]) by fgwmail5.fujitsu.co.jp (Fujitsu Gateway) with ESMTP id n6D5vURA022818 for (envelope-from kosaki.motohiro@jp.fujitsu.com); Mon, 13 Jul 2009 14:57:32 +0900 Received: from smail (m4 [127.0.0.1]) by outgoing.m4.gw.fujitsu.co.jp (Postfix) with ESMTP id 2465345DE6E for ; Mon, 13 Jul 2009 14:57:30 +0900 (JST) Received: from s4.gw.fujitsu.co.jp (s4.gw.fujitsu.co.jp [10.0.50.94]) by m4.gw.fujitsu.co.jp (Postfix) with ESMTP id A2E4D45DE70 for ; Mon, 13 Jul 2009 14:57:29 +0900 (JST) Received: from s4.gw.fujitsu.co.jp (localhost.localdomain [127.0.0.1]) by s4.gw.fujitsu.co.jp (Postfix) with ESMTP id DC886E0800E for ; Mon, 13 Jul 2009 14:57:26 +0900 (JST) Received: from m106.s.css.fujitsu.com (m106.s.css.fujitsu.com [10.249.87.106]) by s4.gw.fujitsu.co.jp (Postfix) with ESMTP id D13651DB8043 for ; Mon, 13 Jul 2009 14:57:25 +0900 (JST) From: KOSAKI Motohiro Subject: [PATCH 0/4] OOM analysis helper patch series v3 Message-Id: <20090713144924.6257.A69D9226@jp.fujitsu.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Date: Mon, 13 Jul 2009 14:57:25 +0900 (JST) Sender: owner-linux-mm@kvack.org To: LKML Cc: kosaki.motohiro@jp.fujitsu.com, linux-mm , Andrew Morton , Wu Fengguang , Christoph Lameter , Rik van Riel List-ID: ChangeLog Since v2 - Dropped "[4/5] add isolate pages vmstat" temporary because it become slightly big. Then, I plan to submit it as another patchset. - Rewrote many patch description (Thanks! Christoph) Since v1 - Dropped "[5/5] add NR_ANON_PAGES to OOM log" patch - Instead, introduce "[5/5] add shmem vmstat" patch - Fixed unit bug (Thanks Minchan) - Separated isolated vmstat to two field (Thanks Minchan and Wu) - Fixed isolated page and lumpy reclaim issue (Thanks Minchan) - Rewrote some patch description (Thanks Christoph) This patch series are tested on 2.6.31-rc2 + mm-show_free_areas-display-slab-pages-in-two-separate-fields.patch ========================== Current OOM log doesn't provide sufficient memory usage information. it cause make confusion to lkml MM guys. this patch series add some memory usage information to OOM log output. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail138.messagelabs.com (mail138.messagelabs.com [216.82.249.35]) by kanga.kvack.org (Postfix) with SMTP id 243F66B004F for ; Mon, 13 Jul 2009 01:39:23 -0400 (EDT) Received: from m1.gw.fujitsu.co.jp ([10.0.50.71]) by fgwmail7.fujitsu.co.jp (Fujitsu Gateway) with ESMTP id n6D60AMm017031 for (envelope-from kosaki.motohiro@jp.fujitsu.com); Mon, 13 Jul 2009 15:00:10 +0900 Received: from smail (m1 [127.0.0.1]) by outgoing.m1.gw.fujitsu.co.jp (Postfix) with ESMTP id A916945DE53 for ; Mon, 13 Jul 2009 15:00:09 +0900 (JST) Received: from s1.gw.fujitsu.co.jp (s1.gw.fujitsu.co.jp [10.0.50.91]) by m1.gw.fujitsu.co.jp (Postfix) with ESMTP id 82B3B45DE54 for ; Mon, 13 Jul 2009 15:00:09 +0900 (JST) Received: from s1.gw.fujitsu.co.jp (localhost.localdomain [127.0.0.1]) by s1.gw.fujitsu.co.jp (Postfix) with ESMTP id 29A6A1DB8046 for ; Mon, 13 Jul 2009 15:00:09 +0900 (JST) Received: from ml13.s.css.fujitsu.com (ml13.s.css.fujitsu.com [10.249.87.103]) by s1.gw.fujitsu.co.jp (Postfix) with ESMTP id 1FAB1E3800C for ; Mon, 13 Jul 2009 15:00:08 +0900 (JST) From: KOSAKI Motohiro Subject: [PATCH 1/4][resend] add per-zone statistics to show_free_areas() In-Reply-To: <20090713144924.6257.A69D9226@jp.fujitsu.com> References: <20090713144924.6257.A69D9226@jp.fujitsu.com> Message-Id: <20090713145732.625A.A69D9226@jp.fujitsu.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Date: Mon, 13 Jul 2009 15:00:07 +0900 (JST) Sender: owner-linux-mm@kvack.org To: LKML Cc: kosaki.motohiro@jp.fujitsu.com, linux-mm , Andrew Morton , Wu Fengguang , Christoph Lameter , Rik van Riel List-ID: Subject: [PATCH] add per-zone statistics to show_free_areas() show_free_areas() displays only a limited amount of zone counters. This patch includes additional counters in the display to allow easier debugging. This may be especially useful if an OOM is due to running out of DMA memory. Signed-off-by: KOSAKI Motohiro Reviewed-by: Christoph Lameter Acked-by: Wu Fengguang Reviewed-by: Minchan Kim Reviewed-by: Rik van Riel --- mm/page_alloc.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) Index: b/mm/page_alloc.c =================================================================== --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2151,6 +2151,16 @@ void show_free_areas(void) " inactive_file:%lukB" " unevictable:%lukB" " present:%lukB" + " mlocked:%lukB" + " dirty:%lukB" + " writeback:%lukB" + " mapped:%lukB" + " slab_reclaimable:%lukB" + " slab_unreclaimable:%lukB" + " pagetables:%lukB" + " unstable:%lukB" + " bounce:%lukB" + " writeback_tmp:%lukB" " pages_scanned:%lu" " all_unreclaimable? %s" "\n", @@ -2165,6 +2175,16 @@ void show_free_areas(void) K(zone_page_state(zone, NR_INACTIVE_FILE)), K(zone_page_state(zone, NR_UNEVICTABLE)), K(zone->present_pages), + K(zone_page_state(zone, NR_MLOCK)), + K(zone_page_state(zone, NR_FILE_DIRTY)), + K(zone_page_state(zone, NR_WRITEBACK)), + K(zone_page_state(zone, NR_FILE_MAPPED)), + K(zone_page_state(zone, NR_SLAB_RECLAIMABLE)), + K(zone_page_state(zone, NR_SLAB_UNRECLAIMABLE)), + K(zone_page_state(zone, NR_PAGETABLE)), + K(zone_page_state(zone, NR_UNSTABLE_NFS)), + K(zone_page_state(zone, NR_BOUNCE)), + K(zone_page_state(zone, NR_WRITEBACK_TEMP)), zone->pages_scanned, (zone_is_all_unreclaimable(zone) ? "yes" : "no") ); -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail143.messagelabs.com (mail143.messagelabs.com [216.82.254.35]) by kanga.kvack.org (Postfix) with SMTP id 4751B6B005A for ; Mon, 13 Jul 2009 01:40:26 -0400 (EDT) Received: from m2.gw.fujitsu.co.jp ([10.0.50.72]) by fgwmail5.fujitsu.co.jp (Fujitsu Gateway) with ESMTP id n6D61ENg024671 for (envelope-from kosaki.motohiro@jp.fujitsu.com); Mon, 13 Jul 2009 15:01:14 +0900 Received: from smail (m2 [127.0.0.1]) by outgoing.m2.gw.fujitsu.co.jp (Postfix) with ESMTP id D3EF545DE57 for ; Mon, 13 Jul 2009 15:01:13 +0900 (JST) Received: from s2.gw.fujitsu.co.jp (s2.gw.fujitsu.co.jp [10.0.50.92]) by m2.gw.fujitsu.co.jp (Postfix) with ESMTP id A8C6E45DD75 for ; Mon, 13 Jul 2009 15:01:13 +0900 (JST) Received: from s2.gw.fujitsu.co.jp (localhost.localdomain [127.0.0.1]) by s2.gw.fujitsu.co.jp (Postfix) with ESMTP id 821DF1DB803A for ; Mon, 13 Jul 2009 15:01:13 +0900 (JST) Received: from m105.s.css.fujitsu.com (m105.s.css.fujitsu.com [10.249.87.105]) by s2.gw.fujitsu.co.jp (Postfix) with ESMTP id 2C684E38004 for ; Mon, 13 Jul 2009 15:01:13 +0900 (JST) From: KOSAKI Motohiro Subject: [PATCH 2/4][resend] add buffer cache information to show_free_areas() In-Reply-To: <20090713144924.6257.A69D9226@jp.fujitsu.com> References: <20090713144924.6257.A69D9226@jp.fujitsu.com> Message-Id: <20090713150021.625D.A69D9226@jp.fujitsu.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Date: Mon, 13 Jul 2009 15:01:12 +0900 (JST) Sender: owner-linux-mm@kvack.org To: LKML Cc: kosaki.motohiro@jp.fujitsu.com, linux-mm , Andrew Morton , Wu Fengguang , Christoph Lameter , Rik van Riel List-ID: ChangeLog Since v3 - Rewrote the descriptin (Thanks Christoph!) Since v2 - Changed display order, now, "buffer" field display right after unstable Since v1 - Fixed showing the number with kilobyte unit issue ================ Subject: [PATCH] add buffer cache information to show_free_areas() It is often useful to know the statistics for all pages that are handled like page cache pages when looking at OOM log output. Therefore show_free_areas() should also display buffer cache statistics. Signed-off-by: KOSAKI Motohiro Acked-by: Wu Fengguang Reviewed-by: Rik van Riel --- mm/page_alloc.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) Index: b/mm/page_alloc.c =================================================================== --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2118,7 +2118,7 @@ void show_free_areas(void) printk("Active_anon:%lu active_file:%lu inactive_anon:%lu\n" " inactive_file:%lu" " unevictable:%lu" - " dirty:%lu writeback:%lu unstable:%lu\n" + " dirty:%lu writeback:%lu unstable:%lu buffer:%lu\n" " free:%lu slab_reclaimable:%lu slab_unreclaimable:%lu\n" " mapped:%lu pagetables:%lu bounce:%lu\n", global_page_state(NR_ACTIVE_ANON), @@ -2129,6 +2129,7 @@ void show_free_areas(void) global_page_state(NR_FILE_DIRTY), global_page_state(NR_WRITEBACK), global_page_state(NR_UNSTABLE_NFS), + nr_blockdev_pages(), global_page_state(NR_FREE_PAGES), global_page_state(NR_SLAB_RECLAIMABLE), global_page_state(NR_SLAB_UNRECLAIMABLE), -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail190.messagelabs.com (mail190.messagelabs.com [216.82.249.51]) by kanga.kvack.org (Postfix) with SMTP id CA64E6B005D for ; Mon, 13 Jul 2009 01:41:39 -0400 (EDT) Received: from m4.gw.fujitsu.co.jp ([10.0.50.74]) by fgwmail5.fujitsu.co.jp (Fujitsu Gateway) with ESMTP id n6D62SMh025192 for (envelope-from kosaki.motohiro@jp.fujitsu.com); Mon, 13 Jul 2009 15:02:28 +0900 Received: from smail (m4 [127.0.0.1]) by outgoing.m4.gw.fujitsu.co.jp (Postfix) with ESMTP id B4BF045DE79 for ; Mon, 13 Jul 2009 15:02:27 +0900 (JST) Received: from s4.gw.fujitsu.co.jp (s4.gw.fujitsu.co.jp [10.0.50.94]) by m4.gw.fujitsu.co.jp (Postfix) with ESMTP id 924C845DE6F for ; Mon, 13 Jul 2009 15:02:27 +0900 (JST) Received: from s4.gw.fujitsu.co.jp (localhost.localdomain [127.0.0.1]) by s4.gw.fujitsu.co.jp (Postfix) with ESMTP id 516111DB803E for ; Mon, 13 Jul 2009 15:02:27 +0900 (JST) Received: from m108.s.css.fujitsu.com (m108.s.css.fujitsu.com [10.249.87.108]) by s4.gw.fujitsu.co.jp (Postfix) with ESMTP id BA2EF1DB803A for ; Mon, 13 Jul 2009 15:02:26 +0900 (JST) From: KOSAKI Motohiro Subject: [PATCH 3/4][resend] Show kernel stack usage in /proc/meminfo and OOM log output In-Reply-To: <20090713144924.6257.A69D9226@jp.fujitsu.com> References: <20090713144924.6257.A69D9226@jp.fujitsu.com> Message-Id: <20090713150114.6260.A69D9226@jp.fujitsu.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Date: Mon, 13 Jul 2009 15:02:25 +0900 (JST) Sender: owner-linux-mm@kvack.org To: LKML Cc: kosaki.motohiro@jp.fujitsu.com, linux-mm , Andrew Morton , Wu Fengguang , Christoph Lameter , Rik van Riel List-ID: ChangeLog Since v1 - Rewrote the descriptin (Thanks Christoph!) ===================== Subject: [PATCH] Show kernel stack usage in /proc/meminfo and OOM log output The amount of memory allocated to kernel stacks can become significant and cause OOM conditions. However, we do not display the amount of memory consumed by stacks. Add code to display the amount of memory used for stacks in /proc/meminfo. Signed-off-by: KOSAKI Motohiro Reviewed-by: Christoph Lameter Reviewed-by: Minchan Kim Reviewed-by: Rik van Riel --- drivers/base/node.c | 3 +++ fs/proc/meminfo.c | 2 ++ include/linux/mmzone.h | 3 ++- kernel/fork.c | 11 +++++++++++ mm/page_alloc.c | 3 +++ mm/vmstat.c | 1 + 6 files changed, 22 insertions(+), 1 deletion(-) Index: b/fs/proc/meminfo.c =================================================================== --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -84,6 +84,7 @@ static int meminfo_proc_show(struct seq_ "Slab: %8lu kB\n" "SReclaimable: %8lu kB\n" "SUnreclaim: %8lu kB\n" + "KernelStack: %8lu kB\n" "PageTables: %8lu kB\n" #ifdef CONFIG_QUICKLIST "Quicklists: %8lu kB\n" @@ -128,6 +129,7 @@ static int meminfo_proc_show(struct seq_ global_page_state(NR_SLAB_UNRECLAIMABLE)), K(global_page_state(NR_SLAB_RECLAIMABLE)), K(global_page_state(NR_SLAB_UNRECLAIMABLE)), + global_page_state(NR_KERNEL_STACK) * THREAD_SIZE / 1024, K(global_page_state(NR_PAGETABLE)), #ifdef CONFIG_QUICKLIST K(quicklist_total_size()), Index: b/include/linux/mmzone.h =================================================================== --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -94,10 +94,11 @@ enum zone_stat_item { NR_SLAB_RECLAIMABLE, NR_SLAB_UNRECLAIMABLE, NR_PAGETABLE, /* used for pagetables */ + NR_KERNEL_STACK, + /* Second 128 byte cacheline */ NR_UNSTABLE_NFS, /* NFS unstable pages */ NR_BOUNCE, NR_VMSCAN_WRITE, - /* Second 128 byte cacheline */ NR_WRITEBACK_TEMP, /* Writeback using temporary buffers */ #ifdef CONFIG_NUMA NUMA_HIT, /* allocated in intended node */ Index: b/kernel/fork.c =================================================================== --- a/kernel/fork.c +++ b/kernel/fork.c @@ -137,9 +137,17 @@ struct kmem_cache *vm_area_cachep; /* SLAB cache for mm_struct structures (tsk->mm) */ static struct kmem_cache *mm_cachep; +static void account_kernel_stack(struct thread_info *ti, int account) +{ + struct zone *zone = page_zone(virt_to_page(ti)); + + mod_zone_page_state(zone, NR_KERNEL_STACK, account); +} + void free_task(struct task_struct *tsk) { prop_local_destroy_single(&tsk->dirties); + account_kernel_stack(tsk->stack, -1); free_thread_info(tsk->stack); rt_mutex_debug_task_free(tsk); ftrace_graph_exit_task(tsk); @@ -255,6 +263,9 @@ static struct task_struct *dup_task_stru tsk->btrace_seq = 0; #endif tsk->splice_pipe = NULL; + + account_kernel_stack(ti, 1); + return tsk; out: Index: b/mm/page_alloc.c =================================================================== --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2158,6 +2158,7 @@ void show_free_areas(void) " mapped:%lukB" " slab_reclaimable:%lukB" " slab_unreclaimable:%lukB" + " kernel_stack:%lukB" " pagetables:%lukB" " unstable:%lukB" " bounce:%lukB" @@ -2182,6 +2183,8 @@ void show_free_areas(void) K(zone_page_state(zone, NR_FILE_MAPPED)), K(zone_page_state(zone, NR_SLAB_RECLAIMABLE)), K(zone_page_state(zone, NR_SLAB_UNRECLAIMABLE)), + zone_page_state(zone, NR_KERNEL_STACK) * + THREAD_SIZE / 1024, K(zone_page_state(zone, NR_PAGETABLE)), K(zone_page_state(zone, NR_UNSTABLE_NFS)), K(zone_page_state(zone, NR_BOUNCE)), Index: b/mm/vmstat.c =================================================================== --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -639,6 +639,7 @@ static const char * const vmstat_text[] "nr_slab_reclaimable", "nr_slab_unreclaimable", "nr_page_table_pages", + "nr_kernel_stack", "nr_unstable", "nr_bounce", "nr_vmscan_write", Index: b/drivers/base/node.c =================================================================== --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -85,6 +85,7 @@ static ssize_t node_read_meminfo(struct "Node %d FilePages: %8lu kB\n" "Node %d Mapped: %8lu kB\n" "Node %d AnonPages: %8lu kB\n" + "Node %d KernelStack: %8lu kB\n" "Node %d PageTables: %8lu kB\n" "Node %d NFS_Unstable: %8lu kB\n" "Node %d Bounce: %8lu kB\n" @@ -116,6 +117,8 @@ static ssize_t node_read_meminfo(struct nid, K(node_page_state(nid, NR_FILE_PAGES)), nid, K(node_page_state(nid, NR_FILE_MAPPED)), nid, K(node_page_state(nid, NR_ANON_PAGES)), + nid, node_page_state(nid, NR_KERNEL_STACK) * + THREAD_SIZE / 1024, nid, K(node_page_state(nid, NR_PAGETABLE)), nid, K(node_page_state(nid, NR_UNSTABLE_NFS)), nid, K(node_page_state(nid, NR_BOUNCE)), -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail191.messagelabs.com (mail191.messagelabs.com [216.82.242.19]) by kanga.kvack.org (Postfix) with SMTP id 144C16B0062 for ; Mon, 13 Jul 2009 01:44:22 -0400 (EDT) Received: from m3.gw.fujitsu.co.jp ([10.0.50.73]) by fgwmail7.fujitsu.co.jp (Fujitsu Gateway) with ESMTP id n6D65Ben019280 for (envelope-from kosaki.motohiro@jp.fujitsu.com); Mon, 13 Jul 2009 15:05:11 +0900 Received: from smail (m3 [127.0.0.1]) by outgoing.m3.gw.fujitsu.co.jp (Postfix) with ESMTP id D268F45DE53 for ; Mon, 13 Jul 2009 15:05:10 +0900 (JST) Received: from s3.gw.fujitsu.co.jp (s3.gw.fujitsu.co.jp [10.0.50.93]) by m3.gw.fujitsu.co.jp (Postfix) with ESMTP id 8A03A45DE52 for ; Mon, 13 Jul 2009 15:05:10 +0900 (JST) Received: from s3.gw.fujitsu.co.jp (localhost.localdomain [127.0.0.1]) by s3.gw.fujitsu.co.jp (Postfix) with ESMTP id 34337E1800C for ; Mon, 13 Jul 2009 15:05:10 +0900 (JST) Received: from m106.s.css.fujitsu.com (m106.s.css.fujitsu.com [10.249.87.106]) by s3.gw.fujitsu.co.jp (Postfix) with ESMTP id A3198E18006 for ; Mon, 13 Jul 2009 15:05:09 +0900 (JST) From: KOSAKI Motohiro Subject: [PATCH 4/4][resend] add shmem vmstat In-Reply-To: <20090713144924.6257.A69D9226@jp.fujitsu.com> References: <20090713144924.6257.A69D9226@jp.fujitsu.com> Message-Id: <20090713150226.6263.A69D9226@jp.fujitsu.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Date: Mon, 13 Jul 2009 15:05:08 +0900 (JST) Sender: owner-linux-mm@kvack.org To: LKML Cc: kosaki.motohiro@jp.fujitsu.com, linux-mm , Andrew Morton , Wu Fengguang , Christoph Lameter , Rik van Riel List-ID: ChangeLog Since v2 - Rewrote the description Since v1 - Fixed misaccounting bug on page migration ======================== Subject: [PATCH] add shmem vmstat Recently we encountered OOM problems due to memory use of the GEM cache. Generally a large amuont of Shmem/Tmpfs pages tend to create a memory shortage problem. We often use the following calculation to determine the amount of shmem pages: shmem = NR_ACTIVE_ANON + NR_INACTIVE_ANON - NR_ANON_PAGES however the expression does not consider isoalted and mlocked pages. This patch adds explicit accounting for pages used by shmem and tmpfs. Signed-off-by: KOSAKI Motohiro Acked-by: Rik van Riel Reviewed-by: Christoph Lameter Acked-by: Wu Fengguang --- drivers/base/node.c | 2 ++ fs/proc/meminfo.c | 2 ++ include/linux/mmzone.h | 1 + mm/filemap.c | 4 ++++ mm/migrate.c | 5 ++++- mm/page_alloc.c | 5 ++++- mm/vmstat.c | 2 +- 7 files changed, 18 insertions(+), 3 deletions(-) Index: b/drivers/base/node.c =================================================================== --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -85,6 +85,7 @@ static ssize_t node_read_meminfo(struct "Node %d FilePages: %8lu kB\n" "Node %d Mapped: %8lu kB\n" "Node %d AnonPages: %8lu kB\n" + "Node %d Shmem: %8lu kB\n" "Node %d KernelStack: %8lu kB\n" "Node %d PageTables: %8lu kB\n" "Node %d NFS_Unstable: %8lu kB\n" @@ -117,6 +118,7 @@ static ssize_t node_read_meminfo(struct nid, K(node_page_state(nid, NR_FILE_PAGES)), nid, K(node_page_state(nid, NR_FILE_MAPPED)), nid, K(node_page_state(nid, NR_ANON_PAGES)), + nid, K(node_page_state(nid, NR_SHMEM)), nid, node_page_state(nid, NR_KERNEL_STACK) * THREAD_SIZE / 1024, nid, K(node_page_state(nid, NR_PAGETABLE)), Index: b/fs/proc/meminfo.c =================================================================== --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -81,6 +81,7 @@ static int meminfo_proc_show(struct seq_ "Writeback: %8lu kB\n" "AnonPages: %8lu kB\n" "Mapped: %8lu kB\n" + "Shmem: %8lu kB\n" "Slab: %8lu kB\n" "SReclaimable: %8lu kB\n" "SUnreclaim: %8lu kB\n" @@ -125,6 +126,7 @@ static int meminfo_proc_show(struct seq_ K(global_page_state(NR_WRITEBACK)), K(global_page_state(NR_ANON_PAGES)), K(global_page_state(NR_FILE_MAPPED)), + K(global_page_state(NR_SHMEM)), K(global_page_state(NR_SLAB_RECLAIMABLE) + global_page_state(NR_SLAB_UNRECLAIMABLE)), K(global_page_state(NR_SLAB_RECLAIMABLE)), Index: b/include/linux/mmzone.h =================================================================== --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -100,6 +100,7 @@ enum zone_stat_item { NR_BOUNCE, NR_VMSCAN_WRITE, NR_WRITEBACK_TEMP, /* Writeback using temporary buffers */ + NR_SHMEM, /* shmem pages (included tmpfs/GEM pages) */ #ifdef CONFIG_NUMA NUMA_HIT, /* allocated in intended node */ NUMA_MISS, /* allocated in non intended node */ Index: b/mm/filemap.c =================================================================== --- a/mm/filemap.c +++ b/mm/filemap.c @@ -120,6 +120,8 @@ void __remove_from_page_cache(struct pag page->mapping = NULL; mapping->nrpages--; __dec_zone_page_state(page, NR_FILE_PAGES); + if (PageSwapBacked(page)) + __dec_zone_page_state(page, NR_SHMEM); BUG_ON(page_mapped(page)); /* @@ -476,6 +478,8 @@ int add_to_page_cache_locked(struct page if (likely(!error)) { mapping->nrpages++; __inc_zone_page_state(page, NR_FILE_PAGES); + if (PageSwapBacked(page)) + __inc_zone_page_state(page, NR_SHMEM); spin_unlock_irq(&mapping->tree_lock); } else { page->mapping = NULL; Index: b/mm/vmstat.c =================================================================== --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -644,7 +644,7 @@ static const char * const vmstat_text[] "nr_bounce", "nr_vmscan_write", "nr_writeback_temp", - + "nr_shmem", #ifdef CONFIG_NUMA "numa_hit", "numa_miss", Index: b/mm/page_alloc.c =================================================================== --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2120,7 +2120,7 @@ void show_free_areas(void) " unevictable:%lu" " dirty:%lu writeback:%lu unstable:%lu buffer:%lu\n" " free:%lu slab_reclaimable:%lu slab_unreclaimable:%lu\n" - " mapped:%lu pagetables:%lu bounce:%lu\n", + " mapped:%lu shmem:%lu pagetables:%lu bounce:%lu\n", global_page_state(NR_ACTIVE_ANON), global_page_state(NR_ACTIVE_FILE), global_page_state(NR_INACTIVE_ANON), @@ -2134,6 +2134,7 @@ void show_free_areas(void) global_page_state(NR_SLAB_RECLAIMABLE), global_page_state(NR_SLAB_UNRECLAIMABLE), global_page_state(NR_FILE_MAPPED), + global_page_state(NR_SHMEM), global_page_state(NR_PAGETABLE), global_page_state(NR_BOUNCE)); @@ -2156,6 +2157,7 @@ void show_free_areas(void) " dirty:%lukB" " writeback:%lukB" " mapped:%lukB" + " shmem:%lukB" " slab_reclaimable:%lukB" " slab_unreclaimable:%lukB" " kernel_stack:%lukB" @@ -2181,6 +2183,7 @@ void show_free_areas(void) K(zone_page_state(zone, NR_FILE_DIRTY)), K(zone_page_state(zone, NR_WRITEBACK)), K(zone_page_state(zone, NR_FILE_MAPPED)), + K(zone_page_state(zone, NR_SHMEM)), K(zone_page_state(zone, NR_SLAB_RECLAIMABLE)), K(zone_page_state(zone, NR_SLAB_UNRECLAIMABLE)), zone_page_state(zone, NR_KERNEL_STACK) * Index: b/mm/migrate.c =================================================================== --- a/mm/migrate.c +++ b/mm/migrate.c @@ -312,7 +312,10 @@ static int migrate_page_move_mapping(str */ __dec_zone_page_state(page, NR_FILE_PAGES); __inc_zone_page_state(newpage, NR_FILE_PAGES); - + if (PageSwapBacked(page)) { + __dec_zone_page_state(page, NR_SHMEM); + __inc_zone_page_state(newpage, NR_SHMEM); + } spin_unlock_irq(&mapping->tree_lock); return 0; -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail203.messagelabs.com (mail203.messagelabs.com [216.82.254.243]) by kanga.kvack.org (Postfix) with SMTP id 00B026B004F for ; Mon, 13 Jul 2009 16:21:36 -0400 (EDT) Message-ID: <4A5B9D52.3050703@redhat.com> Date: Mon, 13 Jul 2009 16:47:14 -0400 From: Rik van Riel MIME-Version: 1.0 Subject: Re: [PATCH 0/4] OOM analysis helper patch series v3 References: <20090713144924.6257.A69D9226@jp.fujitsu.com> In-Reply-To: <20090713144924.6257.A69D9226@jp.fujitsu.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org To: KOSAKI Motohiro Cc: LKML , linux-mm , Andrew Morton , Wu Fengguang , Christoph Lameter List-ID: KOSAKI Motohiro wrote: > ChangeLog > Since v2 > - Dropped "[4/5] add isolate pages vmstat" temporary because it become > slightly big. Then, I plan to submit it as another patchset. Shame, I really liked that patch :) -- All rights reversed. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail191.messagelabs.com (mail191.messagelabs.com [216.82.242.19]) by kanga.kvack.org (Postfix) with ESMTP id 5F2256B004F for ; Mon, 13 Jul 2009 18:03:56 -0400 (EDT) Date: Mon, 13 Jul 2009 15:29:52 -0700 From: Andrew Morton Subject: Re: [PATCH 3/4][resend] Show kernel stack usage in /proc/meminfo and OOM log output Message-Id: <20090713152952.9b1f6388.akpm@linux-foundation.org> In-Reply-To: <20090713150114.6260.A69D9226@jp.fujitsu.com> References: <20090713144924.6257.A69D9226@jp.fujitsu.com> <20090713150114.6260.A69D9226@jp.fujitsu.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org To: KOSAKI Motohiro Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, fengguang.wu@intel.com, cl@linux-foundation.org, riel@redhat.com List-ID: On Mon, 13 Jul 2009 15:02:25 +0900 (JST) KOSAKI Motohiro wrote: > ChangeLog > Since v1 > - Rewrote the descriptin (Thanks Christoph!) > > ===================== > Subject: [PATCH] Show kernel stack usage in /proc/meminfo and OOM log output > > The amount of memory allocated to kernel stacks can become significant and > cause OOM conditions. However, we do not display the amount of memory > consumed by stacks. > > Add code to display the amount of memory used for stacks in /proc/meminfo. > > ... > > +static void account_kernel_stack(struct thread_info *ti, int account) > +{ > + struct zone *zone = page_zone(virt_to_page(ti)); > + > + mod_zone_page_state(zone, NR_KERNEL_STACK, account); > +} > + > void free_task(struct task_struct *tsk) > { > prop_local_destroy_single(&tsk->dirties); > + account_kernel_stack(tsk->stack, -1); But surely there are other less expensive ways of calculating this. The number we want is small-known-constant * number-of-tasks. number-of-tasks probably isn't tracked, but can be calculated along the lines of nr_running(), nr_uninterruptible() and nr_iowait(). number-of-tasks is also equal to number-of-task_structs and number-of_thread_infos which can be obtained from slab (if the arch implemented these via slab - uglier). -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail172.messagelabs.com (mail172.messagelabs.com [216.82.254.3]) by kanga.kvack.org (Postfix) with SMTP id 84EE76B004F for ; Mon, 13 Jul 2009 19:09:44 -0400 (EDT) Received: from m6.gw.fujitsu.co.jp ([10.0.50.76]) by fgwmail6.fujitsu.co.jp (Fujitsu Gateway) with ESMTP id n6DNaOxZ028061 for (envelope-from kosaki.motohiro@jp.fujitsu.com); Tue, 14 Jul 2009 08:36:24 +0900 Received: from smail (m6 [127.0.0.1]) by outgoing.m6.gw.fujitsu.co.jp (Postfix) with ESMTP id 0C50A45DE50 for ; Tue, 14 Jul 2009 08:36:24 +0900 (JST) Received: from s6.gw.fujitsu.co.jp (s6.gw.fujitsu.co.jp [10.0.50.96]) by m6.gw.fujitsu.co.jp (Postfix) with ESMTP id E402545DE4E for ; Tue, 14 Jul 2009 08:36:23 +0900 (JST) Received: from s6.gw.fujitsu.co.jp (localhost.localdomain [127.0.0.1]) by s6.gw.fujitsu.co.jp (Postfix) with ESMTP id B2CAA1DB8037 for ; Tue, 14 Jul 2009 08:36:23 +0900 (JST) Received: from m108.s.css.fujitsu.com (m108.s.css.fujitsu.com [10.249.87.108]) by s6.gw.fujitsu.co.jp (Postfix) with ESMTP id 64BA61DB803E for ; Tue, 14 Jul 2009 08:36:20 +0900 (JST) From: KOSAKI Motohiro Subject: Re: [PATCH 3/4][resend] Show kernel stack usage in /proc/meminfo and OOM log output In-Reply-To: <20090713152952.9b1f6388.akpm@linux-foundation.org> References: <20090713150114.6260.A69D9226@jp.fujitsu.com> <20090713152952.9b1f6388.akpm@linux-foundation.org> Message-Id: <20090714083344.626C.A69D9226@jp.fujitsu.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Date: Tue, 14 Jul 2009 08:36:19 +0900 (JST) Sender: owner-linux-mm@kvack.org To: Andrew Morton Cc: kosaki.motohiro@jp.fujitsu.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, fengguang.wu@intel.com, cl@linux-foundation.org, riel@redhat.com List-ID: > On Mon, 13 Jul 2009 15:02:25 +0900 (JST) > KOSAKI Motohiro wrote: > > > ChangeLog > > Since v1 > > - Rewrote the descriptin (Thanks Christoph!) > > > > ===================== > > Subject: [PATCH] Show kernel stack usage in /proc/meminfo and OOM log output > > > > The amount of memory allocated to kernel stacks can become significant and > > cause OOM conditions. However, we do not display the amount of memory > > consumed by stacks. > > > > Add code to display the amount of memory used for stacks in /proc/meminfo. > > > > ... > > > > +static void account_kernel_stack(struct thread_info *ti, int account) > > +{ > > + struct zone *zone = page_zone(virt_to_page(ti)); > > + > > + mod_zone_page_state(zone, NR_KERNEL_STACK, account); > > +} > > + > > void free_task(struct task_struct *tsk) > > { > > prop_local_destroy_single(&tsk->dirties); > > + account_kernel_stack(tsk->stack, -1); > > But surely there are other less expensive ways of calculating this. > The number we want is small-known-constant * number-of-tasks. > > number-of-tasks probably isn't tracked, but can be calculated along the > lines of nr_running(), nr_uninterruptible() and nr_iowait(). But, nr_running() don't know zone information. we really need per-zone tracking IMHO. > number-of-tasks is also equal to number-of-task_structs and > number-of_thread_infos which can be obtained from slab (if the arch > implemented these via slab - uglier). You know, Almost architecture doesn't use slab for kernel-stack. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail190.messagelabs.com (mail190.messagelabs.com [216.82.249.51]) by kanga.kvack.org (Postfix) with SMTP id 0A3716B004F for ; Mon, 13 Jul 2009 19:12:18 -0400 (EDT) Received: from m6.gw.fujitsu.co.jp ([10.0.50.76]) by fgwmail7.fujitsu.co.jp (Fujitsu Gateway) with ESMTP id n6DNcv0w020520 for (envelope-from kosaki.motohiro@jp.fujitsu.com); Tue, 14 Jul 2009 08:38:59 +0900 Received: from smail (m6 [127.0.0.1]) by outgoing.m6.gw.fujitsu.co.jp (Postfix) with ESMTP id 661BA2AEA81 for ; Tue, 14 Jul 2009 08:38:57 +0900 (JST) Received: from s6.gw.fujitsu.co.jp (s6.gw.fujitsu.co.jp [10.0.50.96]) by m6.gw.fujitsu.co.jp (Postfix) with ESMTP id 1070745DE4F for ; Tue, 14 Jul 2009 08:38:57 +0900 (JST) Received: from s6.gw.fujitsu.co.jp (localhost.localdomain [127.0.0.1]) by s6.gw.fujitsu.co.jp (Postfix) with ESMTP id 867711DB803F for ; Tue, 14 Jul 2009 08:38:56 +0900 (JST) Received: from ml13.s.css.fujitsu.com (ml13.s.css.fujitsu.com [10.249.87.103]) by s6.gw.fujitsu.co.jp (Postfix) with ESMTP id 274841DB8041 for ; Tue, 14 Jul 2009 08:38:56 +0900 (JST) From: KOSAKI Motohiro Subject: Re: [PATCH 0/4] OOM analysis helper patch series v3 In-Reply-To: <4A5B9D52.3050703@redhat.com> References: <20090713144924.6257.A69D9226@jp.fujitsu.com> <4A5B9D52.3050703@redhat.com> Message-Id: <20090714083119.6269.A69D9226@jp.fujitsu.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Date: Tue, 14 Jul 2009 08:38:55 +0900 (JST) Sender: owner-linux-mm@kvack.org To: Rik van Riel Cc: kosaki.motohiro@jp.fujitsu.com, LKML , linux-mm , Andrew Morton , Wu Fengguang , Christoph Lameter List-ID: > KOSAKI Motohiro wrote: > > ChangeLog > > Since v2 > > - Dropped "[4/5] add isolate pages vmstat" temporary because it become > > slightly big. Then, I plan to submit it as another patchset. > > Shame, I really liked that patch :) Sorry, the modification for migration is slightly big. it is still under testing on my stress workload testing environment. I expect it come back at 2-3 days after. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail138.messagelabs.com (mail138.messagelabs.com [216.82.249.35]) by kanga.kvack.org (Postfix) with SMTP id 64B486B004F for ; Tue, 14 Jul 2009 13:56:22 -0400 (EDT) Date: Tue, 14 Jul 2009 11:47:51 -0700 (PDT) From: "Li, Ming Chun" Subject: Re: [PATCH 0/4] OOM analysis helper patch series v3 In-Reply-To: <20090713144924.6257.A69D9226@jp.fujitsu.com> Message-ID: References: <20090713144924.6257.A69D9226@jp.fujitsu.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-linux-mm@kvack.org To: KOSAKI Motohiro Cc: linux-mm List-ID: On Mon, 13 Jul 2009, KOSAKI Motohiro wrote: > ChangeLog > Since v2 > - Dropped "[4/5] add isolate pages vmstat" temporary because it become > slightly big. Then, I plan to submit it as another patchset. > - Rewrote many patch description (Thanks! Christoph) > Since v1 > - Dropped "[5/5] add NR_ANON_PAGES to OOM log" patch > - Instead, introduce "[5/5] add shmem vmstat" patch > - Fixed unit bug (Thanks Minchan) > - Separated isolated vmstat to two field (Thanks Minchan and Wu) > - Fixed isolated page and lumpy reclaim issue (Thanks Minchan) > - Rewrote some patch description (Thanks Christoph) > > This patch series are tested on 2.6.31-rc2 + mm-show_free_areas-display-slab-pages-in-two-separate-fields.patch For your information, I tested the patches on 2.6.31-rc3 + mm-show_free_areas-display-slab-pages-in-two-separate-fields.patch. It is fine. Vincent Li Biomedical Research Center University of British Columbia -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail138.messagelabs.com (mail138.messagelabs.com [216.82.249.35]) by kanga.kvack.org (Postfix) with SMTP id 9AC086B004F for ; Tue, 14 Jul 2009 19:04:00 -0400 (EDT) Received: from m6.gw.fujitsu.co.jp ([10.0.50.76]) by fgwmail5.fujitsu.co.jp (Fujitsu Gateway) with ESMTP id n6ENcfEg031518 for (envelope-from kosaki.motohiro@jp.fujitsu.com); Wed, 15 Jul 2009 08:38:41 +0900 Received: from smail (m6 [127.0.0.1]) by outgoing.m6.gw.fujitsu.co.jp (Postfix) with ESMTP id 3190F45DE52 for ; Wed, 15 Jul 2009 08:38:41 +0900 (JST) Received: from s6.gw.fujitsu.co.jp (s6.gw.fujitsu.co.jp [10.0.50.96]) by m6.gw.fujitsu.co.jp (Postfix) with ESMTP id 159EE45DE51 for ; Wed, 15 Jul 2009 08:38:41 +0900 (JST) Received: from s6.gw.fujitsu.co.jp (localhost.localdomain [127.0.0.1]) by s6.gw.fujitsu.co.jp (Postfix) with ESMTP id EED1E1DB803A for ; Wed, 15 Jul 2009 08:38:40 +0900 (JST) Received: from m107.s.css.fujitsu.com (m107.s.css.fujitsu.com [10.249.87.107]) by s6.gw.fujitsu.co.jp (Postfix) with ESMTP id AA04E1DB803E for ; Wed, 15 Jul 2009 08:38:40 +0900 (JST) From: KOSAKI Motohiro Subject: Re: [PATCH 0/4] OOM analysis helper patch series v3 In-Reply-To: References: <20090713144924.6257.A69D9226@jp.fujitsu.com> Message-Id: <20090715083815.252D.A69D9226@jp.fujitsu.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Date: Wed, 15 Jul 2009 08:38:39 +0900 (JST) Sender: owner-linux-mm@kvack.org To: "Li, Ming Chun" Cc: kosaki.motohiro@jp.fujitsu.com, linux-mm List-ID: > On Mon, 13 Jul 2009, KOSAKI Motohiro wrote: > > > ChangeLog > > Since v2 > > - Dropped "[4/5] add isolate pages vmstat" temporary because it become > > slightly big. Then, I plan to submit it as another patchset. > > - Rewrote many patch description (Thanks! Christoph) > > Since v1 > > - Dropped "[5/5] add NR_ANON_PAGES to OOM log" patch > > - Instead, introduce "[5/5] add shmem vmstat" patch > > - Fixed unit bug (Thanks Minchan) > > - Separated isolated vmstat to two field (Thanks Minchan and Wu) > > - Fixed isolated page and lumpy reclaim issue (Thanks Minchan) > > - Rewrote some patch description (Thanks Christoph) > > > > This patch series are tested on 2.6.31-rc2 + mm-show_free_areas-display-slab-pages-in-two-separate-fields.patch > > For your information, I tested the patches on 2.6.31-rc3 + > mm-show_free_areas-display-slab-pages-in-two-separate-fields.patch. It is > fine. Cute. thanks! -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org