From: David Woodhouse <dwmw2@infradead.org>
To: David Howells <dhowells@redhat.com>
Cc: Wu Fengguang <fengguang.wu@intel.com>,
Minchan Kim <minchan.kim@gmail.com>, Mel Gorman <mel@csn.ul.ie>,
KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
Johannes Weiner <hannes@cmpxchg.org>,
"riel@redhat.com" <riel@redhat.com>,
Andrew Morton <akpm@linux-foundation.org>,
LKML <linux-kernel@vger.kernel.org>,
Christoph Lameter <cl@linux-foundation.org>,
"peterz@infradead.org" <peterz@infradead.org>,
"tytso@mit.edu" <tytso@mit.edu>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
"elladan@eskimo.com" <elladan@eskimo.com>,
"npiggin@suse.de" <npiggin@suse.de>,
"Barnes, Jesse" <jesse.barnes@intel.com>
Subject: Re: Found the commit that causes the OOMs
Date: Mon, 29 Jun 2009 16:56:47 +0100 [thread overview]
Message-ID: <1246291007.663.630.camel@macbook.infradead.org> (raw)
In-Reply-To: <30071.1246290885@redhat.com>
On Mon, 2009-06-29 at 16:54 +0100, David Howells wrote:
> Wu Fengguang <fengguang.wu@intel.com> wrote:
>
> > Yes this time the OOM order/flags are much different from all previous OOMs.
> >
> > btw, I found that msgctl11 is pretty good at making a lot of SUnreclaim and
> > PageTables pages:
>
> I got David Woodhouse to run this on one of this boxes, but he doesn't see the
> problem, I think because he's got 4GB of RAM, and never comes close to running
> out.
>
> I've asked him to reboot with mem=1G to see if that helps reproduce it.
msgctl11 invoked oom-killer: gfp_mask=0xd0, order=1, oom_adj=0
Pid: 5795, comm: msgctl11 Not tainted 2.6.31-rc1 #147
Call Trace:
[<ffffffff81092c77>] oom_kill_process.clone.0+0xac/0x254
[<ffffffff81092b5c>] ? badness+0x24d/0x2bc
[<ffffffff81092f5f>] __out_of_memory+0x140/0x157
[<ffffffff8109308f>] out_of_memory+0x119/0x150
[<ffffffff81095c65>] ? drain_local_pages+0x16/0x18
[<ffffffff810967ab>] __alloc_pages_nodemask+0x45a/0x55b
[<ffffffff810a32b0>] ? __inc_zone_page_state+0x2e/0x30
[<ffffffff810bb6b9>] alloc_pages_current+0xae/0xb6
[<ffffffff810a604a>] ? do_wp_page+0x621/0x6c3
[<ffffffff81094d7e>] __get_free_pages+0xe/0x4b
[<ffffffff810403a7>] copy_process+0xab/0x11a5
[<ffffffff810327c8>] ? check_preempt_wakeup+0x11a/0x142
[<ffffffff810a7a06>] ? handle_mm_fault+0x678/0x6e9
[<ffffffff810415ec>] do_fork+0x14b/0x338
[<ffffffff8105b50a>] ? up_read+0xe/0x10
[<ffffffff814ee655>] ? do_page_fault+0x2da/0x307
[<ffffffff8100a55c>] sys_clone+0x28/0x2a
[<ffffffff8100bfc3>] stub_clone+0x13/0x20
[<ffffffff8100bcdb>] ? system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
CPU 4: hi: 0, btch: 1 usd: 0
CPU 5: hi: 0, btch: 1 usd: 0
CPU 6: hi: 0, btch: 1 usd: 0
CPU 7: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 20
CPU 2: hi: 186, btch: 31 usd: 19
CPU 3: hi: 186, btch: 31 usd: 20
CPU 4: hi: 186, btch: 31 usd: 19
CPU 5: hi: 186, btch: 31 usd: 24
CPU 6: hi: 186, btch: 31 usd: 41
CPU 7: hi: 186, btch: 31 usd: 25
Active_anon:72835 active_file:89 inactive_anon:575
inactive_file:103 unevictable:0 dirty:36 writeback:0 unstable:0
free:2467 slab:38211 mapped:229 pagetables:66918 bounce:0
Node 0 DMA free:4036kB min:60kB low:72kB high:88kB active_anon:3228kB inactive_a
non:256kB active_file:0kB inactive_file:0kB unevictable:0kB present:15356kB page
s_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 994 994 994
Node 0 DMA32 free:5832kB min:4000kB low:5000kB high:6000kB active_anon:288112kB
inactive_anon:2044kB active_file:356kB inactive_file:412kB unevictable:0kB prese
nt:1018080kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 2*8kB 1*16kB 0*32kB 1*64kB 2*128kB 0*256kB 1*512kB 1*1024kB 1*
2048kB 0*4096kB = 3940kB
Node 0 DMA32: 852*4kB 1*8kB 0*16kB 1*32kB 1*64kB 0*128kB 1*256kB 1*512kB 1*1024k
B 0*2048kB 0*4096kB = 5304kB
437 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap = 0kB
Total swap = 0kB
262144 pages RAM
6503 pages reserved
205864 pages shared
226536 pages non-shared
Out of memory: kill process 3855 (msgctl11) score 179248 or a child
Killed process 4222 (msgctl11)
--
David Woodhouse Open Source Technology Centre
David.Woodhouse@intel.com Intel Corporation
WARNING: multiple messages have this Message-ID (diff)
From: David Woodhouse <dwmw2@infradead.org>
To: David Howells <dhowells@redhat.com>
Cc: Wu Fengguang <fengguang.wu@intel.com>,
Minchan Kim <minchan.kim@gmail.com>, Mel Gorman <mel@csn.ul.ie>,
KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
Johannes Weiner <hannes@cmpxchg.org>,
"riel@redhat.com" <riel@redhat.com>,
Andrew Morton <akpm@linux-foundation.org>,
LKML <linux-kernel@vger.kernel.org>,
Christoph Lameter <cl@linux-foundation.org>,
"peterz@infradead.org" <peterz@infradead.org>,
"tytso@mit.edu" <tytso@mit.edu>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
"elladan@eskimo.com" <elladan@eskimo.com>,
"npiggin@suse.de" <npiggin@suse.de>,
"Barnes, Jesse" <jesse.barnes@intel.com>
Subject: Re: Found the commit that causes the OOMs
Date: Mon, 29 Jun 2009 16:56:47 +0100 [thread overview]
Message-ID: <1246291007.663.630.camel@macbook.infradead.org> (raw)
In-Reply-To: <30071.1246290885@redhat.com>
On Mon, 2009-06-29 at 16:54 +0100, David Howells wrote:
> Wu Fengguang <fengguang.wu@intel.com> wrote:
>
> > Yes this time the OOM order/flags are much different from all previous OOMs.
> >
> > btw, I found that msgctl11 is pretty good at making a lot of SUnreclaim and
> > PageTables pages:
>
> I got David Woodhouse to run this on one of this boxes, but he doesn't see the
> problem, I think because he's got 4GB of RAM, and never comes close to running
> out.
>
> I've asked him to reboot with mem=1G to see if that helps reproduce it.
msgctl11 invoked oom-killer: gfp_mask=0xd0, order=1, oom_adj=0
Pid: 5795, comm: msgctl11 Not tainted 2.6.31-rc1 #147
Call Trace:
[<ffffffff81092c77>] oom_kill_process.clone.0+0xac/0x254
[<ffffffff81092b5c>] ? badness+0x24d/0x2bc
[<ffffffff81092f5f>] __out_of_memory+0x140/0x157
[<ffffffff8109308f>] out_of_memory+0x119/0x150
[<ffffffff81095c65>] ? drain_local_pages+0x16/0x18
[<ffffffff810967ab>] __alloc_pages_nodemask+0x45a/0x55b
[<ffffffff810a32b0>] ? __inc_zone_page_state+0x2e/0x30
[<ffffffff810bb6b9>] alloc_pages_current+0xae/0xb6
[<ffffffff810a604a>] ? do_wp_page+0x621/0x6c3
[<ffffffff81094d7e>] __get_free_pages+0xe/0x4b
[<ffffffff810403a7>] copy_process+0xab/0x11a5
[<ffffffff810327c8>] ? check_preempt_wakeup+0x11a/0x142
[<ffffffff810a7a06>] ? handle_mm_fault+0x678/0x6e9
[<ffffffff810415ec>] do_fork+0x14b/0x338
[<ffffffff8105b50a>] ? up_read+0xe/0x10
[<ffffffff814ee655>] ? do_page_fault+0x2da/0x307
[<ffffffff8100a55c>] sys_clone+0x28/0x2a
[<ffffffff8100bfc3>] stub_clone+0x13/0x20
[<ffffffff8100bcdb>] ? system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
CPU 4: hi: 0, btch: 1 usd: 0
CPU 5: hi: 0, btch: 1 usd: 0
CPU 6: hi: 0, btch: 1 usd: 0
CPU 7: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 20
CPU 2: hi: 186, btch: 31 usd: 19
CPU 3: hi: 186, btch: 31 usd: 20
CPU 4: hi: 186, btch: 31 usd: 19
CPU 5: hi: 186, btch: 31 usd: 24
CPU 6: hi: 186, btch: 31 usd: 41
CPU 7: hi: 186, btch: 31 usd: 25
Active_anon:72835 active_file:89 inactive_anon:575
inactive_file:103 unevictable:0 dirty:36 writeback:0 unstable:0
free:2467 slab:38211 mapped:229 pagetables:66918 bounce:0
Node 0 DMA free:4036kB min:60kB low:72kB high:88kB active_anon:3228kB inactive_a
non:256kB active_file:0kB inactive_file:0kB unevictable:0kB present:15356kB page
s_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 994 994 994
Node 0 DMA32 free:5832kB min:4000kB low:5000kB high:6000kB active_anon:288112kB
inactive_anon:2044kB active_file:356kB inactive_file:412kB unevictable:0kB prese
nt:1018080kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 2*8kB 1*16kB 0*32kB 1*64kB 2*128kB 0*256kB 1*512kB 1*1024kB 1*
2048kB 0*4096kB = 3940kB
Node 0 DMA32: 852*4kB 1*8kB 0*16kB 1*32kB 1*64kB 0*128kB 1*256kB 1*512kB 1*1024k
B 0*2048kB 0*4096kB = 5304kB
437 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap = 0kB
Total swap = 0kB
262144 pages RAM
6503 pages reserved
205864 pages shared
226536 pages non-shared
Out of memory: kill process 3855 (msgctl11) score 179248 or a child
Killed process 4222 (msgctl11)
--
David Woodhouse Open Source Technology Centre
David.Woodhouse@intel.com Intel Corporation
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2009-06-29 15:57 UTC|newest]
Thread overview: 184+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-05-17 2:23 [PATCH 0/3] make mapped executable pages the first class citizen Wu Fengguang
2009-05-17 2:23 ` Wu Fengguang
2009-05-17 2:23 ` [PATCH 1/3] vmscan: report vm_flags in page_referenced() Wu Fengguang
2009-05-17 2:23 ` Wu Fengguang
2009-05-17 2:23 ` [PATCH 2/3] vmscan: make mapped executable pages the first class citizen Wu Fengguang
2009-05-17 2:23 ` Wu Fengguang
2009-05-19 8:59 ` Wu Fengguang
2009-05-19 8:59 ` Wu Fengguang
2009-05-17 2:23 ` [PATCH 3/3] vmscan: merge duplicate code in shrink_active_list() Wu Fengguang
2009-05-17 2:23 ` Wu Fengguang
2009-05-18 9:16 ` Wu Fengguang
2009-05-18 9:16 ` Wu Fengguang
2009-05-19 2:43 ` Wu Fengguang
2009-05-19 2:43 ` Wu Fengguang
2009-05-19 10:18 ` Johannes Weiner
2009-05-19 10:18 ` Johannes Weiner
2009-05-19 10:32 ` Wu Fengguang
2009-05-19 10:32 ` Wu Fengguang
2009-06-18 14:46 ` [PATCH 0/3] make mapped executable pages the first class citizen David Howells
2009-06-18 14:46 ` David Howells
2009-06-18 14:51 ` David Howells
2009-06-18 14:51 ` David Howells
2009-06-18 16:18 ` David Howells
2009-06-18 16:18 ` David Howells
2009-06-18 16:57 ` Andrew Morton
2009-06-18 16:57 ` Andrew Morton
2009-06-20 4:33 ` Wu Fengguang
2009-06-20 4:33 ` Wu Fengguang
2009-06-20 8:24 ` David Howells
2009-06-20 8:24 ` David Howells
2009-06-23 14:43 ` David Howells
2009-06-23 14:43 ` David Howells
2009-06-24 1:43 ` KOSAKI Motohiro
2009-06-24 1:43 ` KOSAKI Motohiro
2009-06-24 2:32 ` Wu Fengguang
2009-06-24 2:32 ` Wu Fengguang
2009-06-24 2:43 ` KOSAKI Motohiro
2009-06-24 2:43 ` KOSAKI Motohiro
2009-06-24 2:49 ` Wu Fengguang
2009-06-24 2:49 ` Wu Fengguang
2009-06-27 11:53 ` Johannes Weiner
2009-06-27 11:53 ` Johannes Weiner
2009-06-27 18:40 ` David Howells
2009-06-27 18:40 ` David Howells
2009-06-24 13:07 ` David Howells
2009-06-24 13:07 ` David Howells
2009-06-27 7:12 ` Found the commit that causes the OOMs David Howells
2009-06-27 7:12 ` David Howells
2009-06-27 12:07 ` Minchan Kim
2009-06-27 12:07 ` Minchan Kim
2009-06-27 18:58 ` David Howells
2009-06-27 18:58 ` David Howells
2009-06-27 12:54 ` Johannes Weiner
2009-06-27 12:54 ` Johannes Weiner
2009-06-27 13:50 ` Minchan Kim
2009-06-27 13:50 ` Minchan Kim
2009-06-27 15:36 ` Johannes Weiner
2009-06-27 15:36 ` Johannes Weiner
2009-06-28 16:53 ` Minchan Kim
2009-06-28 16:53 ` Minchan Kim
2009-06-27 15:52 ` KOSAKI Motohiro
2009-06-27 15:52 ` KOSAKI Motohiro
2009-06-27 18:35 ` David Howells
2009-06-27 18:35 ` David Howells
2009-06-28 7:55 ` David Howells
2009-06-28 7:55 ` David Howells
2009-06-28 11:32 ` Wu Fengguang
2009-06-28 11:32 ` Wu Fengguang
2009-06-28 13:30 ` Minchan Kim
2009-06-28 13:30 ` Minchan Kim
2009-06-28 13:36 ` Minchan Kim
2009-06-28 13:36 ` Minchan Kim
2009-06-28 14:22 ` Wu Fengguang
2009-06-28 14:22 ` Wu Fengguang
2009-06-28 15:01 ` KOSAKI Motohiro
2009-06-28 15:01 ` KOSAKI Motohiro
2009-06-28 15:10 ` Wu Fengguang
2009-06-28 15:10 ` Wu Fengguang
2009-06-28 16:50 ` Minchan Kim
2009-06-28 16:50 ` Minchan Kim
2009-06-29 0:17 ` Minchan Kim
2009-06-29 0:17 ` Minchan Kim
2009-06-29 7:34 ` Wu Fengguang
2009-06-29 7:34 ` Wu Fengguang
2009-06-29 10:10 ` David Howells
2009-06-29 10:10 ` David Howells
2009-06-29 12:55 ` Wu Fengguang
2009-06-29 12:55 ` Wu Fengguang
2009-06-29 14:21 ` David Howells
2009-06-29 14:21 ` David Howells
2009-06-29 15:00 ` Minchan Kim
2009-06-29 15:00 ` Minchan Kim
2009-06-29 15:14 ` Wu Fengguang
2009-06-29 15:14 ` Wu Fengguang
2009-06-29 15:54 ` David Howells
2009-06-29 15:54 ` David Howells
2009-06-29 15:56 ` David Woodhouse [this message]
2009-06-29 15:56 ` David Woodhouse
2009-06-30 14:05 ` Wu Fengguang
2009-06-30 14:05 ` Wu Fengguang
2009-06-30 15:50 ` Minchan Kim
2009-06-30 15:50 ` Minchan Kim
2009-07-01 2:30 ` Wu Fengguang
2009-07-01 2:30 ` Wu Fengguang
2009-07-01 1:18 ` KOSAKI Motohiro
2009-07-01 1:18 ` KOSAKI Motohiro
2009-07-01 2:13 ` Rik van Riel
2009-07-01 2:13 ` Rik van Riel
2009-07-01 2:16 ` Wu Fengguang
2009-07-01 2:16 ` Wu Fengguang
2009-07-01 2:26 ` Wu Fengguang
2009-07-01 2:26 ` Wu Fengguang
2009-07-01 2:51 ` KOSAKI Motohiro
2009-07-01 2:51 ` KOSAKI Motohiro
2009-07-01 2:57 ` Rik van Riel
2009-07-01 2:57 ` Rik van Riel
2009-07-01 4:06 ` Wu Fengguang
2009-07-01 4:06 ` Wu Fengguang
2009-07-01 4:18 ` KOSAKI Motohiro
2009-07-01 4:18 ` KOSAKI Motohiro
2009-07-01 4:25 ` Wu Fengguang
2009-07-01 4:25 ` Wu Fengguang
2009-07-01 4:30 ` KOSAKI Motohiro
2009-07-01 4:30 ` KOSAKI Motohiro
2009-07-01 11:27 ` Wu Fengguang
2009-07-01 11:27 ` Wu Fengguang
2009-07-05 9:55 ` Wu Fengguang
2009-07-05 9:55 ` Wu Fengguang
2009-07-05 10:38 ` KOSAKI Motohiro
2009-07-05 10:38 ` KOSAKI Motohiro
2009-07-05 10:51 ` Wu Fengguang
2009-07-05 10:51 ` Wu Fengguang
2009-07-01 3:54 ` Wu Fengguang
2009-07-01 3:54 ` Wu Fengguang
2009-06-29 15:27 ` David Howells
2009-06-29 15:27 ` David Howells
2009-06-29 16:07 ` Mel Gorman
2009-06-29 16:07 ` Mel Gorman
2009-06-30 4:07 ` Minchan Kim
2009-06-30 4:07 ` Minchan Kim
2009-06-30 9:22 ` Mel Gorman
2009-06-30 9:22 ` Mel Gorman
2009-06-30 9:30 ` Minchan Kim
2009-06-30 9:30 ` Minchan Kim
2009-06-30 14:00 ` Minchan Kim
2009-06-30 14:00 ` Minchan Kim
2009-06-30 19:57 ` David Howells
2009-06-30 19:57 ` David Howells
2009-07-02 7:41 ` Minchan Kim
2009-07-02 7:41 ` Minchan Kim
2009-07-02 7:44 ` Minchan Kim
2009-07-02 7:44 ` Minchan Kim
2009-07-02 12:43 ` Wu Fengguang
2009-07-02 12:43 ` Wu Fengguang
2009-07-02 14:08 ` Minchan Kim
2009-07-02 14:08 ` Minchan Kim
2009-06-28 14:49 ` KOSAKI Motohiro
2009-06-28 14:49 ` KOSAKI Motohiro
2009-06-28 15:04 ` Wu Fengguang
2009-06-28 15:04 ` Wu Fengguang
2009-06-28 16:47 ` Minchan Kim
2009-06-28 16:47 ` Minchan Kim
2009-06-29 7:48 ` KOSAKI Motohiro
2009-06-29 7:48 ` KOSAKI Motohiro
2009-06-29 9:32 ` Minchan Kim
2009-06-29 9:32 ` Minchan Kim
2009-06-29 12:43 ` David Howells
2009-06-29 12:43 ` David Howells
2009-06-29 12:59 ` Wu Fengguang
2009-06-29 12:59 ` Wu Fengguang
2009-06-29 16:57 ` Andrew Morton
2009-06-29 16:57 ` Andrew Morton
2009-06-29 18:54 ` KOSAKI Motohiro
2009-06-29 18:54 ` KOSAKI Motohiro
2009-06-29 19:08 ` KOSAKI Motohiro
2009-06-29 19:08 ` KOSAKI Motohiro
2009-06-19 5:27 ` [PATCH 0/3] make mapped executable pages the first class citizen Wu Fengguang
2009-06-19 5:27 ` Wu Fengguang
2009-06-19 5:24 ` Wu Fengguang
2009-06-19 5:24 ` Wu Fengguang
2009-06-19 5:58 ` Wu Fengguang
2009-06-19 5:58 ` Wu Fengguang
2009-06-19 8:06 ` David Howells
2009-06-19 8:06 ` David Howells
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1246291007.663.630.camel@macbook.infradead.org \
--to=dwmw2@infradead.org \
--cc=akpm@linux-foundation.org \
--cc=cl@linux-foundation.org \
--cc=dhowells@redhat.com \
--cc=elladan@eskimo.com \
--cc=fengguang.wu@intel.com \
--cc=hannes@cmpxchg.org \
--cc=jesse.barnes@intel.com \
--cc=kosaki.motohiro@jp.fujitsu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mel@csn.ul.ie \
--cc=minchan.kim@gmail.com \
--cc=npiggin@suse.de \
--cc=peterz@infradead.org \
--cc=riel@redhat.com \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.