PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[dave b]

In 2.6.3* kernels (test case was performed on the 2.6.33.3 kernel)
when physical memory runs out and there is a large swap partition -
the system completely stalls.

I noticed that when running debian lenny using dm-crypt  with
encrypted / and swap with a  2.6.33.3 kernel (and all of the 2.6.3*
series iirc) when all physical memory is used (swapiness was left at
the default 60) the system hangs and does not respond. It can resume
normal operation some time later - however it seems to take a *very*
long time for the oom killer to come in. Obviously with swapoff this
doesn't happen - the oom killer comes in and does its job.


free -m
             total       used       free     shared    buffers     cached
Mem:          1980       1101        879          0         58        201
-/+ buffers/cache:        840       1139
Swap:        24943          0      24943


My simple test case is

dd if=/dev/zero of=/tmp/stall
and wait till /tmp fills...

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[dave b]

On 14 May 2010 22:53, dave b <db.pub.mail@gmail.com> wrote:
> In 2.6.3* kernels (test case was performed on the 2.6.33.3 kernel)
> when physical memory runs out and there is a large swap partition -
> the system completely stalls.
>
> I noticed that when running debian lenny using dm-crypt  with
> encrypted / and swap with a  2.6.33.3 kernel (and all of the 2.6.3*
> series iirc) when all physical memory is used (swapiness was left at
> the default 60) the system hangs and does not respond. It can resume
> normal operation some time later - however it seems to take a *very*
> long time for the oom killer to come in. Obviously with swapoff this
> doesn't happen - the oom killer comes in and does its job.
>
>
> free -m
>             total       used       free     shared    buffers     cached
> Mem:          1980       1101        879          0         58        201
> -/+ buffers/cache:        840       1139
> Swap:        24943          0      24943
>
>
> My simple test case is
>
> dd if=/dev/zero of=/tmp/stall
> and wait till /tmp fills...
>

Sorry - I forgot to say I am running x86-64

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[dave b]

Is there a reason - no one has taken any interesting in my email ?....
 The behaviour isn't found on the 2.6.26 debian kernel. So I was
thinking that it might be due to my intel graphics card / memory
interplay ? ....

On 14 May 2010 23:14, dave b <db.pub.mail@gmail.com> wrote:
> On 14 May 2010 22:53, dave b <db.pub.mail@gmail.com> wrote:
>> In 2.6.3* kernels (test case was performed on the 2.6.33.3 kernel)
>> when physical memory runs out and there is a large swap partition -
>> the system completely stalls.
>>
>> I noticed that when running debian lenny using dm-crypt  with
>> encrypted / and swap with a  2.6.33.3 kernel (and all of the 2.6.3*
>> series iirc) when all physical memory is used (swapiness was left at
>> the default 60) the system hangs and does not respond. It can resume
>> normal operation some time later - however it seems to take a *very*
>> long time for the oom killer to come in. Obviously with swapoff this
>> doesn't happen - the oom killer comes in and does its job.
>>
>>
>> free -m
>>             total       used       free     shared    buffers     cached
>> Mem:          1980       1101        879          0         58        201
>> -/+ buffers/cache:        840       1139
>> Swap:        24943          0      24943
>>
>>
>> My simple test case is
>>
>> dd if=/dev/zero of=/tmp/stall
>> and wait till /tmp fills...
>>
>
> Sorry - I forgot to say I am running x86-64
>

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[Hugh Dickins]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 4103 bytes --]

On Thu, 20 May 2010, dave b wrote:

> Is there a reason - no one has taken any interesting in my email ?....
>  The behaviour isn't found on the 2.6.26 debian kernel. So I was
> thinking that it might be due to my intel graphics card / memory
> interplay ? ....

It's nothing personal: the usual reason is that people are very busy.

> 
> On 14 May 2010 23:14, dave b <db.pub.mail@gmail.com> wrote:
> > On 14 May 2010 22:53, dave b <db.pub.mail@gmail.com> wrote:
> >> In 2.6.3* kernels (test case was performed on the 2.6.33.3 kernel)
> >> when physical memory runs out and there is a large swap partition -
> >> the system completely stalls.
> >>
> >> I noticed that when running debian lenny using dm-crypt  with
> >> encrypted / and swap with a  2.6.33.3 kernel (and all of the 2.6.3*
> >> series iirc) when all physical memory is used (swapiness was left at
> >> the default 60) the system hangs and does not respond. It can resume
> >> normal operation some time later - however it seems to take a *very*
> >> long time for the oom killer to come in. Obviously with swapoff this
> >> doesn't happen - the oom killer comes in and does its job.
> >>
> >>
> >> free -m
> >>             total       used       free     shared    buffers     cached
> >> Mem:          1980       1101        879          0         58        201
> >> -/+ buffers/cache:        840       1139
> >> Swap:        24943          0      24943
> >>
> >>
> >> My simple test case is
> >>
> >> dd if=/dev/zero of=/tmp/stall
> >> and wait till /tmp fills...

Is that tmpfs sized the default 50% of RAM?
If you have sized it larger, then indeed filling it up might behave badly.

> >>
> >
> > Sorry - I forgot to say I am running x86-64

But I wonder if you're suffering from a bug which KOSAKI-San just
identified, and has very recently posted this patch: please try
it and let us all know - thanks.

Hugh

[PATCH] tmpfs: Insert tmpfs cache pages to inactive list at first

Shaohua Li reported parallel file copy on tmpfs can lead to
OOM killer. This is regression of caused by commit 9ff473b9a7
(vmscan: evict streaming IO first). Wow, It is 2 years old patch!

Currently, tmpfs file cache is inserted active list at first. It
mean the insertion doesn't only increase numbers of pages in anon LRU,
but also reduce anon scanning ratio. Therefore, vmscan will get totally
confusion. It scan almost only file LRU even though the system have
plenty unused tmpfs pages.

Historically, lru_cache_add_active_anon() was used by two reasons.
1) Intend to priotize shmem page rather than regular file cache.
2) Intend to avoid reclaim priority inversion of used once pages.

But we've lost both motivation because (1) Now we have separate
anon and file LRU list. then, to insert active list doesn't help
such priotize. (2) In past, one pte access bit will cause page
activation. then to insert inactive list with pte access bit mean
higher priority than to insert active list. Its priority inversion
may lead to uninteded lru chun. but it was already solved by commit
645747462 (vmscan: detect mapped file pages used only once).
(Thanks Hannes, you are great!)

Thus, now we can use lru_cache_add_anon() instead.

Reported-by: Shaohua Li <shaohua.li@intel.com>
Cc: Wu Fengguang <fengguang.wu@intel.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
Cc: Hugh Dickins <hughd@google.com>
Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
---
 mm/filemap.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/mm/filemap.c b/mm/filemap.c
index b941996..023ef61 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -452,7 +452,7 @@ int add_to_page_cache_lru(struct page *page, struct address_space *mapping,
 		if (page_is_file_cache(page))
 			lru_cache_add_file(page);
 		else
-			lru_cache_add_active_anon(page);
+			lru_cache_add_anon(page);
 	}
 	return ret;
 }
-- 
1.6.5.2

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[dave b]

That was just a simple test case with dd. That test case might be
invalid - but it is trying to trigger out of memory - doing this any
other way still causes the problem. I note that playing with some bios
settings I was actually able to trigger what appeared to be graphics
corruption issues when I launched kde applications ... nothing shows
up in dmesg so this might just be a conflict between xorg and the
kernel with those bios settings...

Anyway, This is no longer a 'problem' for me since I disabled
overcommit and altered the values for dirty_ratio and
dirty_background_ratio - and I cannot trigger it.

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[David Rientjes]

On Thu, 27 May 2010, dave b wrote:

> That was just a simple test case with dd. That test case might be
> invalid - but it is trying to trigger out of memory - doing this any
> other way still causes the problem. I note that playing with some bios
> settings I was actually able to trigger what appeared to be graphics
> corruption issues when I launched kde applications ... nothing shows
> up in dmesg so this might just be a conflict between xorg and the
> kernel with those bios settings...
> 
> Anyway, This is no longer a 'problem' for me since I disabled
> overcommit and altered the values for dirty_ratio and
> dirty_background_ratio - and I cannot trigger it.
> 

Disabling overcommit should always do it, but I'd be interested to know if 
restoring dirty_ratio to 40 would help your usecase.

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[dave b]

On 2 June 2010 06:52, David Rientjes <rientjes@google.com> wrote:
> On Thu, 27 May 2010, dave b wrote:
>
>> That was just a simple test case with dd. That test case might be
>> invalid - but it is trying to trigger out of memory - doing this any
>> other way still causes the problem. I note that playing with some bios
>> settings I was actually able to trigger what appeared to be graphics
>> corruption issues when I launched kde applications ... nothing shows
>> up in dmesg so this might just be a conflict between xorg and the
>> kernel with those bios settings...
>>
>> Anyway, This is no longer a 'problem' for me since I disabled
>> overcommit and altered the values for dirty_ratio and
>> dirty_background_ratio - and I cannot trigger it.
>>
>
> Disabling overcommit should always do it, but I'd be interested to know if
> restoring dirty_ratio to 40 would help your usecase.
>
Actually it turns out on 2.6.34.1 I can trigger this issue. What it
really is, is that linux doesn't invoke the oom killer when it should
and kill something off. This is *really* annoying.

I used the follow script - (on 2.6.34.1)
cat ./scripts/disable_over_commit
#!/bin/bash
echo 2 > /proc/sys/vm/overcommit_memory
echo 40 > /proc/sys/vm/dirty_ratio
echo 5 > /proc/sys/vm/dirty_background_ratio

And I was still able to reproduce this bug.
Here is some c  code to trigger the condition I am talking about.


#include <stdlib.h>
#include <stdio.h>

int main(void)
{
	while(1)
	{
		malloc(1000);
	}

	return 0;
}

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[David Rientjes]

On Tue, 27 Jul 2010, dave b wrote:

> Actually it turns out on 2.6.34.1 I can trigger this issue. What it
> really is, is that linux doesn't invoke the oom killer when it should
> and kill something off. This is *really* annoying.
> 

I'm not exactly sure what you're referring to, it's been two months and 
you're using a new kernel and now you're saying that the oom killer isn't 
being utilized when the original problem statement was that it was killing 
things inappropriately?

> I used the follow script - (on 2.6.34.1)
> cat ./scripts/disable_over_commit
> #!/bin/bash
> echo 2 > /proc/sys/vm/overcommit_memory
> echo 40 > /proc/sys/vm/dirty_ratio
> echo 5 > /proc/sys/vm/dirty_background_ratio
> 
> And I was still able to reproduce this bug.
> Here is some c  code to trigger the condition I am talking about.
> 
> 
> #include <stdlib.h>
> #include <stdio.h>
> 
> int main(void)
> {
> 	while(1)
> 	{
> 		malloc(1000);
> 	}
> 
> 	return 0;
> }
> 

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[dave b]

On 27 July 2010 08:12, David Rientjes <rientjes@google.com> wrote:
> On Tue, 27 Jul 2010, dave b wrote:
>
>> Actually it turns out on 2.6.34.1 I can trigger this issue. What it
>> really is, is that linux doesn't invoke the oom killer when it should
>> and kill something off. This is *really* annoying.
>>
>
> I'm not exactly sure what you're referring to, it's been two months and
> you're using a new kernel and now you're saying that the oom killer isn't
> being utilized when the original problem statement was that it was killing
> things inappropriately?

Sorry about the timespan :(
Well actually it is the same issue. Originally the oom killer wasn't
being invoked and now the problem is still it isn't invoked - it
doesn't come and kill things - my desktop just sits :)
I have since replaced the hard disk - which I thought could be the
issue. I am thinking that because I have shared graphics not using KMS
- with intel graphics - this may be the root of the cause.

--
All things that are, are with more spirit chased than enjoyed.		--
Shakespeare, "Merchant of Venice"

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[KOSAKI Motohiro]

> On 27 July 2010 08:12, David Rientjes <rientjes@google.com> wrote:
> > On Tue, 27 Jul 2010, dave b wrote:
> >
> >> Actually it turns out on 2.6.34.1 I can trigger this issue. What it
> >> really is, is that linux doesn't invoke the oom killer when it should
> >> and kill something off. This is *really* annoying.
> >>
> >
> > I'm not exactly sure what you're referring to, it's been two months and
> > you're using a new kernel and now you're saying that the oom killer isn't
> > being utilized when the original problem statement was that it was killing
> > things inappropriately?
> 
> Sorry about the timespan :(
> Well actually it is the same issue. Originally the oom killer wasn't
> being invoked and now the problem is still it isn't invoked - it
> doesn't come and kill things - my desktop just sits :)
> I have since replaced the hard disk - which I thought could be the
> issue. I am thinking that because I have shared graphics not using KMS
> - with intel graphics - this may be the root of the cause.

Do you mean the issue will be gone if disabling intel graphics?
if so, we need intel graphics driver folks help. sorry, linux-mm folks don't
know intel graphics detail.

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[dave b]

On 27 July 2010 14:46, KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
>> On 27 July 2010 08:12, David Rientjes <rientjes@google.com> wrote:
>> > On Tue, 27 Jul 2010, dave b wrote:
>> >
>> >> Actually it turns out on 2.6.34.1 I can trigger this issue. What it
>> >> really is, is that linux doesn't invoke the oom killer when it should
>> >> and kill something off. This is *really* annoying.
>> >>
>> >
>> > I'm not exactly sure what you're referring to, it's been two months and
>> > you're using a new kernel and now you're saying that the oom killer isn't
>> > being utilized when the original problem statement was that it was killing
>> > things inappropriately?
>>
>> Sorry about the timespan :(
>> Well actually it is the same issue. Originally the oom killer wasn't
>> being invoked and now the problem is still it isn't invoked - it
>> doesn't come and kill things - my desktop just sits :)
>> I have since replaced the hard disk - which I thought could be the
>> issue. I am thinking that because I have shared graphics not using KMS
>> - with intel graphics - this may be the root of the cause.
>
> Do you mean the issue will be gone if disabling intel graphics?
> if so, we need intel graphics driver folks help. sorry, linux-mm folks don't
> know intel graphics detail.

Well the only other system I have running the 2.6.34.1 kernel atm is
an arm based system.
I originally sent this to the kernel list and was told I should
probably forward it to the mm list.
It may be a general issue or it could just be specific :)

--
"Not Hercules could have knock'd out his brains, for he had none."		--
Shakespeare

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[KOSAKI Motohiro]

> > Do you mean the issue will be gone if disabling intel graphics?
> > if so, we need intel graphics driver folks help. sorry, linux-mm folks don't
> > know intel graphics detail.
> 
> Well the only other system I have running the 2.6.34.1 kernel atm is
> an arm based system.
> I originally sent this to the kernel list and was told I should
> probably forward it to the mm list.
> It may be a general issue or it could just be specific :)

Hmm.. I'm puzzled 8-)

I don't understand why other all people can't reproduce your issue
even though your reproduce program is very simple.

So, I'm guessing there is hidden reproduce condition. but I have no
idea to find it.

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[dave b]

On 27 July 2010 16:09, KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
>> > Do you mean the issue will be gone if disabling intel graphics?
>> It may be a general issue or it could just be specific :)
>
> Hmm.. I'm puzzled 8-)
>
> I don't understand why other all people can't reproduce your issue
> even though your reproduce program is very simple.
>
> So, I'm guessing there is hidden reproduce condition. but I have no
> idea to find it.

I will try with the latest ubuntu and report how that goes (that will
be using fairly new xorg etc.) it is likely to be hidden issue just
with the intel graphics driver. However, my concern is that it isn't -
and it is about how shared graphics memory is handled :)

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[dave b]

On 27 July 2010 18:09, dave b <db.pub.mail@gmail.com> wrote:
> On 27 July 2010 16:09, KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
>>> > Do you mean the issue will be gone if disabling intel graphics?
>>> It may be a general issue or it could just be specific :)
>
> I will try with the latest ubuntu and report how that goes (that will
> be using fairly new xorg etc.) it is likely to be hidden issue just
> with the intel graphics driver. However, my concern is that it isn't -
> and it is about how shared graphics memory is handled :)


Ok my desktop still stalled and no oom killer was invoked when I added
swap to a live-cd of 10.04 amd64.

*Without* *swap* *on* - the oom killer was invoked - here is a copy of it.

[  298.180542] Xorg invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0
[  298.180553] Xorg cpuset=/ mems_allowed=0
[  298.180560] Pid: 3808, comm: Xorg Not tainted 2.6.32-21-generic #32-Ubuntu
[  298.180564] Call Trace:
[  298.180583]  [<ffffffff810b37cd>] ? cpuset_print_task_mems_allowed+0x9d/0xb0
[  298.180595]  [<ffffffff810f64f4>] oom_kill_process+0xd4/0x2f0
[  298.180603]  [<ffffffff810f6ab0>] ? select_bad_process+0xd0/0x110
[  298.180609]  [<ffffffff810f6b48>] __out_of_memory+0x58/0xc0
[  298.180616]  [<ffffffff810f6cde>] out_of_memory+0x12e/0x1a0
[  298.180626]  [<ffffffff81540c9e>] ? _spin_lock+0xe/0x20
[  298.180633]  [<ffffffff810f9d21>] __alloc_pages_slowpath+0x511/0x580
[  298.180641]  [<ffffffff810f9eee>] __alloc_pages_nodemask+0x15e/0x1a0
[  298.180650]  [<ffffffff8112ca57>] alloc_pages_current+0x87/0xd0
[  298.180657]  [<ffffffff810f8e0e>] __get_free_pages+0xe/0x50
[  298.180666]  [<ffffffff81154994>] __pollwait+0xb4/0xf0
[  298.180673]  [<ffffffff814e09a5>] unix_poll+0x25/0xc0
[  298.180682]  [<ffffffff81449bea>] sock_poll+0x1a/0x20
[  298.180688]  [<ffffffff811545b2>] do_select+0x3a2/0x6d0
[  298.180696]  [<ffffffff811548e0>] ? __pollwait+0x0/0xf0
[  298.180702]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  298.180708]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  298.180714]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  298.180721]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  298.180727]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  298.180732]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  298.180737]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  298.180741]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  298.180745]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  298.180749]  [<ffffffff811550ba>] core_sys_select+0x18a/0x2c0
[  298.180777]  [<ffffffffa001eced>] ? drm_ioctl+0x13d/0x480 [drm]
[  298.180784]  [<ffffffff81085320>] ? autoremove_wake_function+0x0/0x40
[  298.180790]  [<ffffffff810397a9>] ? default_spin_lock_flags+0x9/0x10
[  298.180795]  [<ffffffff81540bbf>] ? _spin_lock_irqsave+0x2f/0x40
[  298.180800]  [<ffffffff81019e89>] ? read_tsc+0x9/0x20
[  298.180805]  [<ffffffff8108f9c9>] ? ktime_get_ts+0xa9/0xe0
[  298.180810]  [<ffffffff81155447>] sys_select+0x47/0x110
[  298.180816]  [<ffffffff810131b2>] system_call_fastpath+0x16/0x1b
[  298.180819] Mem-Info:
[  298.180822] Node 0 DMA per-cpu:
[  298.180827] CPU    0: hi:    0, btch:   1 usd:   0
[  298.180830] CPU    1: hi:    0, btch:   1 usd:   0
[  298.180832] Node 0 DMA32 per-cpu:
[  298.180837] CPU    0: hi:  186, btch:  31 usd:  60
[  298.180839] CPU    1: hi:  186, btch:  31 usd: 137
[  298.180845] active_anon:374344 inactive_anon:81753 isolated_anon:0
[  298.180847]  active_file:7038 inactive_file:7089 isolated_file:0
[  298.180848]  unevictable:0 dirty:0 writeback:0 unstable:0
[  298.180849]  free:3399 slab_reclaimable:4226 slab_unreclaimable:4383
[  298.180851]  mapped:13010 shmem:45284 pagetables:5496 bounce:0
[  298.180854] Node 0 DMA free:7920kB min:40kB low:48kB high:60kB
active_anon:3880kB inactive_anon:4096kB active_file:0kB
inactive_file:0kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:15348kB mlocked:0kB dirty:0kB writeback:0kB
mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:8kB
kernel_stack:0kB pagetables:16kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[  298.180866] lowmem_reserve[]: 0 1971 1971 1971
[  298.180871] Node 0 DMA32 free:5676kB min:5660kB low:7072kB
high:8488kB active_anon:1493496kB inactive_anon:322916kB
active_file:28152kB inactive_file:28356kB unevictable:0kB
isolated(anon):0kB isolated(file):0kB present:2019172kB mlocked:0kB
dirty:0kB writeback:0kB mapped:52040kB shmem:181136kB
slab_reclaimable:16904kB slab_unreclaimable:17524kB
kernel_stack:2096kB pagetables:21968kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:41088 all_unreclaimable? no
[  298.180884] lowmem_reserve[]: 0 0 0 0
[  298.180889] Node 0 DMA: 4*4kB 2*8kB 1*16kB 2*32kB 2*64kB 2*128kB
1*256kB 2*512kB 2*1024kB 2*2048kB 0*4096kB = 7920kB
[  298.180904] Node 0 DMA32: 397*4kB 1*8kB 1*16kB 1*32kB 1*64kB
1*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 5676kB
[  298.180918] 59413 total pagecache pages
[  298.180920] 0 pages in swap cache
[  298.180923] Swap cache stats: add 0, delete 0, find 0/0
[  298.180925] Free swap  = 0kB
[  298.180927] Total swap = 0kB
[  298.188124] 515887 pages RAM
[  298.188127] 9764 pages reserved
[  298.188129] 108553 pages shared
[  298.188131] 467319 pages non-shared
[  298.188136] Out of memory: kill process 3821 (gnome-session) score
503983 or a child
[  298.188141] Killed process 3855 (ssh-agent)
[  300.280284] gnome-terminal invoked oom-killer: gfp_mask=0xd0,
order=0, oom_adj=0
[  300.280293] gnome-terminal cpuset=/ mems_allowed=0
[  300.280300] Pid: 4081, comm: gnome-terminal Not tainted
2.6.32-21-generic #32-Ubuntu
[  300.280304] Call Trace:
[  300.280322]  [<ffffffff810b37cd>] ? cpuset_print_task_mems_allowed+0x9d/0xb0
[  300.280331]  [<ffffffff810f64f4>] oom_kill_process+0xd4/0x2f0
[  300.280338]  [<ffffffff810f6ab0>] ? select_bad_process+0xd0/0x110
[  300.280345]  [<ffffffff810f6b48>] __out_of_memory+0x58/0xc0
[  300.280351]  [<ffffffff810f6cde>] out_of_memory+0x12e/0x1a0
[  300.280360]  [<ffffffff81540c9e>] ? _spin_lock+0xe/0x20
[  300.280367]  [<ffffffff810f9d21>] __alloc_pages_slowpath+0x511/0x580
[  300.280375]  [<ffffffff810f9eee>] __alloc_pages_nodemask+0x15e/0x1a0
[  300.280386]  [<ffffffff8112ca57>] alloc_pages_current+0x87/0xd0
[  300.280393]  [<ffffffff810f8e0e>] __get_free_pages+0xe/0x50
[  300.280401]  [<ffffffff81154994>] __pollwait+0xb4/0xf0
[  300.280408]  [<ffffffff814e09a5>] unix_poll+0x25/0xc0
[  300.280416]  [<ffffffff81449bea>] sock_poll+0x1a/0x20
[  300.280423]  [<ffffffff81153f95>] do_poll+0x115/0x2c0
[  300.280430]  [<ffffffff81154b85>] do_sys_poll+0x155/0x210
[  300.280436]  [<ffffffff811548e0>] ? __pollwait+0x0/0xf0
[  300.280443]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  300.280449]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  300.280456]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  300.280462]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  300.280468]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  300.280474]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  300.280480]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  300.280487]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  300.280493]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  300.280502]  [<ffffffff8133111e>] ? tty_ldisc_deref+0xe/0x10
[  300.280510]  [<ffffffff81019e89>] ? read_tsc+0x9/0x20
[  300.280518]  [<ffffffff8108f9c9>] ? ktime_get_ts+0xa9/0xe0
[  300.280525]  [<ffffffff81153e0d>] ? poll_select_set_timeout+0x8d/0xa0
[  300.280532]  [<ffffffff81154e3c>] sys_poll+0x7c/0x110
[  300.280541]  [<ffffffff810131b2>] system_call_fastpath+0x16/0x1b
[  300.280545] Mem-Info:
[  300.280549] Node 0 DMA per-cpu:
[  300.280554] CPU    0: hi:    0, btch:   1 usd:   0
[  300.280558] CPU    1: hi:    0, btch:   1 usd:   0
[  300.280561] Node 0 DMA32 per-cpu:
[  300.280566] CPU    0: hi:  186, btch:  31 usd:  31
[  300.280570] CPU    1: hi:  186, btch:  31 usd: 138
[  300.280579] active_anon:373779 inactive_anon:82443 isolated_anon:0
[  300.280581]  active_file:7002 inactive_file:7034 isolated_file:64
[  300.280583]  unevictable:0 dirty:0 writeback:0 unstable:0
[  300.280585]  free:3370 slab_reclaimable:4219 slab_unreclaimable:4383
[  300.280587]  mapped:12988 shmem:44524 pagetables:5489 bounce:0
[  300.280592] Node 0 DMA free:7920kB min:40kB low:48kB high:60kB
active_anon:3880kB inactive_anon:4096kB active_file:0kB
inactive_file:0kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:15348kB mlocked:0kB dirty:0kB writeback:0kB
mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:8kB
kernel_stack:0kB pagetables:16kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[  300.280611] lowmem_reserve[]: 0 1971 1971 1971
[  300.280619] Node 0 DMA32 free:5560kB min:5660kB low:7072kB
high:8488kB active_anon:1491236kB inactive_anon:325676kB
active_file:28008kB inactive_file:28136kB unevictable:0kB
isolated(anon):0kB isolated(file):256kB present:2019172kB mlocked:0kB
dirty:0kB writeback:0kB mapped:51952kB shmem:178096kB
slab_reclaimable:16876kB slab_unreclaimable:17524kB
kernel_stack:2096kB pagetables:21940kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:58305 all_unreclaimable? no
[  300.280639] lowmem_reserve[]: 0 0 0 0
[  300.280647] Node 0 DMA: 4*4kB 2*8kB 1*16kB 2*32kB 2*64kB 2*128kB
1*256kB 2*512kB 2*1024kB 2*2048kB 0*4096kB = 7920kB
[  300.280668] Node 0 DMA32: 366*4kB 0*8kB 0*16kB 0*32kB 0*64kB
0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 5560kB
[  300.280687] 58623 total pagecache pages
[  300.280691] 0 pages in swap cache
[  300.280695] Swap cache stats: add 0, delete 0, find 0/0
[  300.280698] Free swap  = 0kB
[  300.280700] Total swap = 0kB
[  300.289576] 515887 pages RAM
[  300.289581] 9764 pages reserved
[  300.289583] 107753 pages shared
[  300.289585] 468177 pages non-shared
[  300.289590] Out of memory: kill process 3821 (gnome-session) score
502491 or a child
[  300.289595] Killed process 3883 (bluetooth-apple)
[  302.180340] a.out invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
[  302.180347] a.out cpuset=/ mems_allowed=0
[  302.180352] Pid: 4210, comm: a.out Not tainted 2.6.32-21-generic #32-Ubuntu
[  302.180355] Call Trace:
[  302.180367]  [<ffffffff810b37cd>] ? cpuset_print_task_mems_allowed+0x9d/0xb0
[  302.180374]  [<ffffffff810f64f4>] oom_kill_process+0xd4/0x2f0
[  302.180379]  [<ffffffff810f6ab0>] ? select_bad_process+0xd0/0x110
[  302.180383]  [<ffffffff810f6b48>] __out_of_memory+0x58/0xc0
[  302.180388]  [<ffffffff810f6cde>] out_of_memory+0x12e/0x1a0
[  302.180395]  [<ffffffff81540c9e>] ? _spin_lock+0xe/0x20
[  302.180399]  [<ffffffff810f9d21>] __alloc_pages_slowpath+0x511/0x580
[  302.180405]  [<ffffffff810f9eee>] __alloc_pages_nodemask+0x15e/0x1a0
[  302.180411]  [<ffffffff8112cb12>] alloc_page_vma+0x72/0xf0
[  302.180416]  [<ffffffff810fe741>] ? lru_cache_add_lru+0x21/0x40
[  302.180422]  [<ffffffff81110580>] do_anonymous_page+0x100/0x260
[  302.180426]  [<ffffffff811150ff>] handle_mm_fault+0x31f/0x3c0
[  302.180433]  [<ffffffff810397a9>] ? default_spin_lock_flags+0x9/0x10
[  302.180439]  [<ffffffff8154380a>] do_page_fault+0x12a/0x3b0
[  302.180443]  [<ffffffff81541165>] page_fault+0x25/0x30
[  302.180447] Mem-Info:
[  302.180450] Node 0 DMA per-cpu:
[  302.180453] CPU    0: hi:    0, btch:   1 usd:   0
[  302.180456] CPU    1: hi:    0, btch:   1 usd:   0
[  302.180458] Node 0 DMA32 per-cpu:
[  302.180462] CPU    0: hi:  186, btch:  31 usd:  68
[  302.180464] CPU    1: hi:  186, btch:  31 usd: 158
[  302.180470] active_anon:374176 inactive_anon:82789 isolated_anon:0
[  302.180472]  active_file:6734 inactive_file:6846 isolated_file:31
[  302.180473]  unevictable:0 dirty:0 writeback:0 unstable:0
[  302.180474]  free:3413 slab_reclaimable:3970 slab_unreclaimable:4374
[  302.180476]  mapped:12935 shmem:44505 pagetables:5405 bounce:0
[  302.180479] Node 0 DMA free:7920kB min:40kB low:48kB high:60kB
active_anon:3880kB inactive_anon:4096kB active_file:0kB
inactive_file:0kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:15348kB mlocked:0kB dirty:0kB writeback:0kB
mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:8kB
kernel_stack:0kB pagetables:16kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[  302.180491] lowmem_reserve[]: 0 1971 1971 1971
[  302.180497] Node 0 DMA32 free:5732kB min:5660kB low:7072kB
high:8488kB active_anon:1492824kB inactive_anon:327060kB
active_file:26936kB inactive_file:27384kB unevictable:0kB
isolated(anon):0kB isolated(file):124kB present:2019172kB mlocked:0kB
dirty:0kB writeback:0kB mapped:51740kB shmem:178020kB
slab_reclaimable:15880kB slab_unreclaimable:17488kB
kernel_stack:2080kB pagetables:21604kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:25314 all_unreclaimable? no
[  302.180510] lowmem_reserve[]: 0 0 0 0
[  302.180515] Node 0 DMA: 4*4kB 2*8kB 1*16kB 2*32kB 2*64kB 2*128kB
1*256kB 2*512kB 2*1024kB 2*2048kB 0*4096kB = 7920kB
[  302.180529] Node 0 DMA32: 399*4kB 5*8kB 0*16kB 0*32kB 0*64kB
0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 5732kB
[  302.180542] 58114 total pagecache pages
[  302.180544] 0 pages in swap cache
[  302.180547] Swap cache stats: add 0, delete 0, find 0/0
[  302.180549] Free swap  = 0kB
[  302.180551] Total swap = 0kB
[  302.187756] 515887 pages RAM
[  302.187760] 9764 pages reserved
[  302.187762] 105575 pages shared
[  302.187764] 468594 pages non-shared
[  302.187768] Out of memory: kill process 3821 (gnome-session) score
409517 or a child
[  302.187774] Killed process 3886 (nm-applet)
[  303.112722] a.out invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
[  303.112729] a.out cpuset=/ mems_allowed=0
[  303.112734] Pid: 4210, comm: a.out Not tainted 2.6.32-21-generic #32-Ubuntu
[  303.112737] Call Trace:
[  303.112749]  [<ffffffff810b37cd>] ? cpuset_print_task_mems_allowed+0x9d/0xb0
[  303.112756]  [<ffffffff810f64f4>] oom_kill_process+0xd4/0x2f0
[  303.112760]  [<ffffffff810f6ab0>] ? select_bad_process+0xd0/0x110
[  303.112765]  [<ffffffff810f6b48>] __out_of_memory+0x58/0xc0
[  303.112769]  [<ffffffff810f6cde>] out_of_memory+0x12e/0x1a0
[  303.112776]  [<ffffffff81540c9e>] ? _spin_lock+0xe/0x20
[  303.112780]  [<ffffffff810f9d21>] __alloc_pages_slowpath+0x511/0x580
[  303.112786]  [<ffffffff810f9eee>] __alloc_pages_nodemask+0x15e/0x1a0
[  303.112792]  [<ffffffff8112cb12>] alloc_page_vma+0x72/0xf0
[  303.112797]  [<ffffffff810fe741>] ? lru_cache_add_lru+0x21/0x40
[  303.112803]  [<ffffffff81110580>] do_anonymous_page+0x100/0x260
[  303.112807]  [<ffffffff811150ff>] handle_mm_fault+0x31f/0x3c0
[  303.112813]  [<ffffffff810397a9>] ? default_spin_lock_flags+0x9/0x10
[  303.112819]  [<ffffffff8154380a>] do_page_fault+0x12a/0x3b0
[  303.112824]  [<ffffffff81541165>] page_fault+0x25/0x30
[  303.112827] Mem-Info:
[  303.112830] Node 0 DMA per-cpu:
[  303.112834] CPU    0: hi:    0, btch:   1 usd:   0
[  303.112836] CPU    1: hi:    0, btch:   1 usd:   0
[  303.112839] Node 0 DMA32 per-cpu:
[  303.112842] CPU    0: hi:  186, btch:  31 usd: 168
[  303.112845] CPU    1: hi:  186, btch:  31 usd: 114
[  303.112850] active_anon:374917 inactive_anon:82293 isolated_anon:0
[  303.112852]  active_file:6678 inactive_file:6756 isolated_file:33
[  303.112853]  unevictable:0 dirty:0 writeback:0 unstable:0
[  303.112854]  free:3380 slab_reclaimable:3969 slab_unreclaimable:4366
[  303.112856]  mapped:12857 shmem:44508 pagetables:5296 bounce:0
[  303.112858] Node 0 DMA free:7920kB min:40kB low:48kB high:60kB
active_anon:3880kB inactive_anon:4096kB active_file:0kB
inactive_file:0kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:15348kB mlocked:0kB dirty:0kB writeback:0kB
mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:8kB
kernel_stack:0kB pagetables:16kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[  303.112871] lowmem_reserve[]: 0 1971 1971 1971
[  303.112876] Node 0 DMA32 free:5600kB min:5660kB low:7072kB
high:8488kB active_anon:1495788kB inactive_anon:325076kB
active_file:26712kB inactive_file:27024kB unevictable:0kB
isolated(anon):0kB isolated(file):132kB present:2019172kB mlocked:0kB
dirty:0kB writeback:0kB mapped:51428kB shmem:178032kB
slab_reclaimable:15876kB slab_unreclaimable:17456kB
kernel_stack:2072kB pagetables:21168kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:63493 all_unreclaimable? no
[  303.112889] lowmem_reserve[]: 0 0 0 0
[  303.112894] Node 0 DMA: 4*4kB 2*8kB 1*16kB 2*32kB 2*64kB 2*128kB
1*256kB 2*512kB 2*1024kB 2*2048kB 0*4096kB = 7920kB
[  303.112909] Node 0 DMA32: 374*4kB 1*8kB 0*16kB 0*32kB 0*64kB
0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 5600kB
[  303.112922] 57976 total pagecache pages
[  303.112925] 0 pages in swap cache
[  303.112928] Swap cache stats: add 0, delete 0, find 0/0
[  303.112930] Free swap  = 0kB
[  303.112931] Total swap = 0kB
[  303.120149] 515887 pages RAM
[  303.120153] 9764 pages reserved
[  303.120155] 103221 pages shared
[  303.120158] 468651 pages non-shared
[  303.120162] Out of memory: kill process 3821 (gnome-session) score
380210 or a child
[  303.120168] Killed process 3889 (polkit-gnome-au)
[  303.460215] clock-applet invoked oom-killer: gfp_mask=0xd0,
order=0, oom_adj=0
[  303.460222] clock-applet cpuset=/ mems_allowed=0
[  303.460227] Pid: 4034, comm: clock-applet Not tainted
2.6.32-21-generic #32-Ubuntu
[  303.460230] Call Trace:
[  303.460243]  [<ffffffff810b37cd>] ? cpuset_print_task_mems_allowed+0x9d/0xb0
[  303.460250]  [<ffffffff810f64f4>] oom_kill_process+0xd4/0x2f0
[  303.460254]  [<ffffffff810f6ab0>] ? select_bad_process+0xd0/0x110
[  303.460259]  [<ffffffff810f6b48>] __out_of_memory+0x58/0xc0
[  303.460264]  [<ffffffff810f6cde>] out_of_memory+0x12e/0x1a0
[  303.460270]  [<ffffffff81540c9e>] ? _spin_lock+0xe/0x20
[  303.460275]  [<ffffffff810f9d21>] __alloc_pages_slowpath+0x511/0x580
[  303.460280]  [<ffffffff810f9eee>] __alloc_pages_nodemask+0x15e/0x1a0
[  303.460287]  [<ffffffff8112ca57>] alloc_pages_current+0x87/0xd0
[  303.460292]  [<ffffffff810f8e0e>] __get_free_pages+0xe/0x50
[  303.460298]  [<ffffffff81154994>] __pollwait+0xb4/0xf0
[  303.460303]  [<ffffffff814e09a5>] unix_poll+0x25/0xc0
[  303.460309]  [<ffffffff81449bea>] sock_poll+0x1a/0x20
[  303.460314]  [<ffffffff81153f95>] do_poll+0x115/0x2c0
[  303.460319]  [<ffffffff810fb3ad>] ? free_hot_page+0x2d/0x60
[  303.460324]  [<ffffffff81154b85>] do_sys_poll+0x155/0x210
[  303.460328]  [<ffffffff811548e0>] ? __pollwait+0x0/0xf0
[  303.460333]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  303.460337]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  303.460342]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  303.460346]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  303.460351]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  303.460355]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  303.460359]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  303.460364]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  303.460368]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
[  303.460375]  [<ffffffff81019e89>] ? read_tsc+0x9/0x20
[  303.460381]  [<ffffffff8108f9c9>] ? ktime_get_ts+0xa9/0xe0
[  303.460386]  [<ffffffff81153e0d>] ? poll_select_set_timeout+0x8d/0xa0
[  303.460391]  [<ffffffff81154e3c>] sys_poll+0x7c/0x110
[  303.460397]  [<ffffffff810131b2>] system_call_fastpath+0x16/0x1b
[  303.460400] Mem-Info:
[  303.460403] Node 0 DMA per-cpu:
[  303.460407] CPU    0: hi:    0, btch:   1 usd:   0
[  303.460410] CPU    1: hi:    0, btch:   1 usd:   0
[  303.460412] Node 0 DMA32 per-cpu:
[  303.460416] CPU    0: hi:  186, btch:  31 usd: 177
[  303.460419] CPU    1: hi:  186, btch:  31 usd:  72
[  303.460425] active_anon:375232 inactive_anon:82145 isolated_anon:0
[  303.460426]  active_file:6651 inactive_file:6755 isolated_file:33
[  303.460427]  unevictable:0 dirty:0 writeback:0 unstable:0
[  303.460429]  free:3380 slab_reclaimable:3969 slab_unreclaimable:4366
[  303.460430]  mapped:12826 shmem:44508 pagetables:5203 bounce:0
[  303.460433] Node 0 DMA free:7920kB min:40kB low:48kB high:60kB
active_anon:3880kB inactive_anon:4096kB active_file:0kB
inactive_file:0kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:15348kB mlocked:0kB dirty:0kB writeback:0kB
mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:8kB
kernel_stack:0kB pagetables:16kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[  303.460445] lowmem_reserve[]: 0 1971 1971 1971
[  303.460451] Node 0 DMA32 free:5600kB min:5660kB low:7072kB
high:8488kB active_anon:1497048kB inactive_anon:324484kB
active_file:26604kB inactive_file:27020kB unevictable:0kB
isolated(anon):0kB isolated(file):132kB present:2019172kB mlocked:0kB
dirty:0kB writeback:0kB mapped:51304kB shmem:178032kB
slab_reclaimable:15876kB slab_unreclaimable:17456kB
kernel_stack:2072kB pagetables:20796kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:38786 all_unreclaimable? no
[  303.460464] lowmem_reserve[]: 0 0 0 0
[  303.460469] Node 0 DMA: 4*4kB 2*8kB 1*16kB 2*32kB 2*64kB 2*128kB
1*256kB 2*512kB 2*1024kB 2*2048kB 0*4096kB = 7920kB
[  303.460483] Node 0 DMA32: 378*4kB 1*8kB 0*16kB 0*32kB 0*64kB
0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 5616kB
[  303.460497] 57976 total pagecache pages
[  303.460500] 0 pages in swap cache
[  303.460502] Swap cache stats: add 0, delete 0, find 0/0
[  303.460505] Free swap  = 0kB
[  303.460506] Total swap = 0kB
[  303.467705] 515887 pages RAM
[  303.467709] 9764 pages reserved
[  303.467711] 101853 pages shared
[  303.467713] 468702 pages non-shared
[  303.467717] Out of memory: kill process 4210 (a.out) score 365712 or a child
[  303.467723] Killed process 4210 (a.out)
ubuntu@ubuntu:~/Desktop$




and here is the dmesg for the system:


[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version 2.6.32-21-generic (buildd@yellow) (gcc
version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #32-Ubuntu SMP Fri Apr 16
08:09:38 UTC 2010 (Ubuntu 2.6.32-21.32-generic 2.6.32.11+drm33.2)
[    0.000000] Command line: BOOT_IMAGE=/casper/vmlinuz
file=/cdrom/preseed/ubuntu.seed boot=casper initrd=/casper/initrd.lz
quiet splash --
[    0.000000] KERNEL supported cpus:
[    0.000000]   Intel GenuineIntel
[    0.000000]   AMD AuthenticAMD
[    0.000000]   Centaur CentaurHauls
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
[    0.000000]  BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000e6000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 000000007df2f800 (usable)
[    0.000000]  BIOS-e820: 000000007df2f800 - 000000007df30000 (ACPI NVS)
[    0.000000]  BIOS-e820: 000000007df30000 - 000000007df40000 (ACPI data)
[    0.000000]  BIOS-e820: 000000007df40000 - 000000007dff0000 (ACPI NVS)
[    0.000000]  BIOS-e820: 000000007dff0000 - 000000007e000000 (reserved)
[    0.000000]  BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
[    0.000000]  BIOS-e820: 00000000fed13000 - 00000000fed1a000 (reserved)
[    0.000000]  BIOS-e820: 00000000fed1c000 - 00000000feda0000 (reserved)
[    0.000000] DMI 2.3 present.
[    0.000000] last_pfn = 0x7df2f max_arch_pfn = 0x400000000
[    0.000000] MTRR default type: uncachable
[    0.000000] MTRR fixed ranges enabled:
[    0.000000]   00000-9FFFF write-back
[    0.000000]   A0000-EFFFF uncachable
[    0.000000]   F0000-FFFFF write-protect
[    0.000000] MTRR variable ranges enabled:
[    0.000000]   0 base 000000000 mask F80000000 write-back
[    0.000000]   1 base 07E000000 mask FFE000000 uncachable
[    0.000000]   2 disabled
[    0.000000]   3 disabled
[    0.000000]   4 disabled
[    0.000000]   5 disabled
[    0.000000]   6 disabled
[    0.000000]   7 disabled
[    0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
[    0.000000] e820 update range: 0000000000001000 - 0000000000006000
(usable) ==> (reserved)
[    0.000000] Scanning 1 areas for low memory corruption
[    0.000000] modified physical RAM map:
[    0.000000]  modified: 0000000000000000 - 0000000000001000 (usable)
[    0.000000]  modified: 0000000000001000 - 0000000000006000 (reserved)
[    0.000000]  modified: 0000000000006000 - 000000000009fc00 (usable)
[    0.000000]  modified: 000000000009fc00 - 00000000000a0000 (reserved)
[    0.000000]  modified: 00000000000e6000 - 0000000000100000 (reserved)
[    0.000000]  modified: 0000000000100000 - 000000007df2f800 (usable)
[    0.000000]  modified: 000000007df2f800 - 000000007df30000 (ACPI NVS)
[    0.000000]  modified: 000000007df30000 - 000000007df40000 (ACPI data)
[    0.000000]  modified: 000000007df40000 - 000000007dff0000 (ACPI NVS)
[    0.000000]  modified: 000000007dff0000 - 000000007e000000 (reserved)
[    0.000000]  modified: 00000000e0000000 - 00000000f0000000 (reserved)
[    0.000000]  modified: 00000000fed13000 - 00000000fed1a000 (reserved)
[    0.000000]  modified: 00000000fed1c000 - 00000000feda0000 (reserved)
[    0.000000] initial memory mapped : 0 - 20000000
[    0.000000] init_memory_mapping: 0000000000000000-000000007df2f000
[    0.000000] NX (Execute Disable) protection: active
[    0.000000]  0000000000 - 007de00000 page 2M
[    0.000000]  007de00000 - 007df2f000 page 4k
[    0.000000] kernel direct mapping tables up to 7df2f000 @ 8000-c000
[    0.000000] RAMDISK: 7d59e000 - 7deffcaf
[    0.000000] ACPI: RSDP 00000000000f4eb0 00014 (v00 ACPIAM)
[    0.000000] ACPI: RSDT 000000007df30000 0003C (v01 INTEL  D915GAG
20060222 MSFT 00000097)
[    0.000000] ACPI: FACP 000000007df30200 00081 (v02 INTEL  D915GAG
20060222 MSFT 00000097)
[    0.000000] ACPI: DSDT 000000007df30440 05C05 (v01 INTEL  D915GAG
00000001 INTL 02002026)
[    0.000000] ACPI: FACS 000000007df40000 00040
[    0.000000] ACPI: APIC 000000007df30390 00068 (v01 INTEL  D915GAG
20060222 MSFT 00000097)
[    0.000000] ACPI: MCFG 000000007df30400 0003C (v01 INTEL  D915GAG
20060222 MSFT 00000097)
[    0.000000] ACPI: ASF! 000000007df36050 00099 (v16 LEGEND I865PASF
00000001 INTL 02002026)
[    0.000000] ACPI: TCPA 000000007df360f0 00032 (v01 INTEL  TBLOEMID
00000001 MSFT 00000097)
[    0.000000] ACPI: WDDT 000000007df36122 00040 (v01 INTEL  OEMWDDT
00000001 INTL 02002026)
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-000000007df2f000
[    0.000000] Bootmem setup node 0 0000000000000000-000000007df2f000
[    0.000000]   NODE_DATA [000000000000a000 - 000000000000efff]
[    0.000000]   bootmap [000000000000f000 -  000000000001ebe7] pages 10
[    0.000000] (7 early reservations) ==> bootmem [0000000000 - 007df2f000]
[    0.000000]   #0 [0000000000 - 0000001000]   BIOS data page ==>
[0000000000 - 0000001000]
[    0.000000]   #1 [0000006000 - 0000008000]       TRAMPOLINE ==>
[0000006000 - 0000008000]
[    0.000000]   #2 [0001000000 - 0001a29e64]    TEXT DATA BSS ==>
[0001000000 - 0001a29e64]
[    0.000000]   #3 [007d59e000 - 007deffcaf]          RAMDISK ==>
[007d59e000 - 007deffcaf]
[    0.000000]   #4 [000009fc00 - 0000100000]    BIOS reserved ==>
[000009fc00 - 0000100000]
[    0.000000]   #5 [0001a2a000 - 0001a2a1b4]              BRK ==>
[0001a2a000 - 0001a2a1b4]
[    0.000000]   #6 [0000008000 - 000000a000]          PGTABLE ==>
[0000008000 - 000000a000]
[    0.000000] found SMP MP-table at [ffff8800000ff780] ff780
[    0.000000]  [ffffea0000000000-ffffea0001bfffff] PMD ->
[ffff880002000000-ffff880003bfffff] on node 0
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000000 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   0x00100000 -> 0x00100000
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[3] active PFN ranges
[    0.000000]     0: 0x00000000 -> 0x00000001
[    0.000000]     0: 0x00000006 -> 0x0000009f
[    0.000000]     0: 0x00000100 -> 0x0007df2f
[    0.000000] On node 0 totalpages: 515785
[    0.000000]   DMA zone: 56 pages used for memmap
[    0.000000]   DMA zone: 101 pages reserved
[    0.000000]   DMA zone: 3837 pages, LIFO batch:0
[    0.000000]   DMA32 zone: 6998 pages used for memmap
[    0.000000]   DMA32 zone: 504793 pages, LIFO batch:31
[    0.000000] ACPI: PM-Timer IO Port: 0x408
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0x01] dfl dfl lint[0x1])
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0x02] dfl dfl lint[0x1])
[    0.000000] ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] ACPI: IRQ0 used by override.
[    0.000000] ACPI: IRQ2 used by override.
[    0.000000] ACPI: IRQ9 used by override.
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] SMP: Allowing 2 CPUs, 0 hotplug CPUs
[    0.000000] nr_irqs_gsi: 24
[    0.000000] PM: Registered nosave memory: 0000000000001000 - 0000000000006000
[    0.000000] PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000e6000
[    0.000000] PM: Registered nosave memory: 00000000000e6000 - 0000000000100000
[    0.000000] Allocating PCI resources starting at 7e000000 (gap:
7e000000:62000000)
[    0.000000] Booting paravirtualized kernel on bare hardware
[    0.000000] NR_CPUS:64 nr_cpumask_bits:64 nr_cpu_ids:2 nr_node_ids:1
[    0.000000] PERCPU: Embedded 30 pages/cpu @ffff880001c00000 s91544
r8192 d23144 u1048576
[    0.000000] pcpu-alloc: s91544 r8192 d23144 u1048576 alloc=1*2097152
[    0.000000] pcpu-alloc: [0] 0 1
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.
Total pages: 508630
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line: BOOT_IMAGE=/casper/vmlinuz
file=/cdrom/preseed/ubuntu.seed boot=casper initrd=/casper/initrd.lz
quiet splash --
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Initializing CPU#0
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Calgary: detecting Calgary via BIOS EBDA area
[    0.000000] Calgary: Unable to locate Rio Grande table in EBDA - bailing!
[    0.000000] Memory: 2014032k/2063548k available (5409k kernel code,
408k absent, 49108k reserved, 2976k data, 876k init)
[    0.000000] SLUB: Genslabs=14, HWalign=64, Order=0-3, MinObjects=0,
CPUs=2, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS:4352 nr_irqs:424
[    0.000000] Console: colour VGA+ 80x25
[    0.000000] console [tty0] enabled
[    0.000000] allocated 20971520 bytes of page_cgroup
[    0.000000] please try 'cgroup_disable=memory' option if you don't
want memory cgroups
[    0.000000] Fast TSC calibration using PIT
[    0.000000] Detected 3000.196 MHz processor.
[    0.010010] Calibrating delay loop (skipped), value calculated
using timer frequency.. 6000.39 BogoMIPS (lpj=30001960)
[    0.010047] Security Framework initialized
[    0.010074] AppArmor: AppArmor initialized
[    0.010421] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.011776] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.012432] Mount-cache hash table entries: 256
[    0.012648] Initializing cgroup subsys ns
[    0.012655] Initializing cgroup subsys cpuacct
[    0.012660] Initializing cgroup subsys memory
[    0.012672] Initializing cgroup subsys devices
[    0.012676] Initializing cgroup subsys freezer
[    0.012679] Initializing cgroup subsys net_cls
[    0.012713] CPU: Trace cache: 12K uops, L1 D cache: 16K
[    0.012718] CPU: L2 cache: 1024K
[    0.012722] CPU 0/0x0 -> Node 0
[    0.012725] CPU: Physical Processor ID: 0
[    0.012727] CPU: Processor Core ID: 0
[    0.012731] mce: CPU supports 4 MCE banks
[    0.012746] CPU0: Thermal monitoring enabled (TM1)
[    0.012753] using mwait in idle threads.
[    0.012756] Performance Events: no PMU driver, software events only.
[    0.021475] ACPI: Core revision 20090903
[    0.033331] ftrace: converting mcount calls to 0f 1f 44 00 00
[    0.033338] ftrace: allocating 22518 entries in 89 pages
[    0.040090] Setting APIC routing to flat
[    0.040411] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.148236] CPU0: Intel(R) Pentium(R) 4 CPU 3.00GHz stepping 01
[    0.150000] Booting processor 1 APIC 0x1 ip 0x6000
[    0.020000] Initializing CPU#1
[    0.020000] CPU: Trace cache: 12K uops, L1 D cache: 16K
[    0.020000] CPU: L2 cache: 1024K
[    0.020000] CPU 1/0x1 -> Node 0
[    0.020000] CPU: Physical Processor ID: 0
[    0.020000] CPU: Processor Core ID: 0
[    0.020000] CPU1: Thermal monitoring enabled (TM1)
[    0.300148] CPU1: Intel(R) Pentium(R) 4 CPU 3.00GHz stepping 01
[    0.300160] checking TSC synchronization [CPU#0 -> CPU#1]: passed.
[    0.310028] Brought up 2 CPUs
[    0.310033] Total of 2 processors activated (12000.71 BogoMIPS).
[    0.310374] CPU0 attaching sched-domain:
[    0.310380]  domain 0: span 0-1 level SIBLING
[    0.310384]   groups: 0 (cpu_power = 589) 1 (cpu_power = 589)
[    0.310392]   domain 1: span 0-1 level MC
[    0.310395]    groups: 0-1 (cpu_power = 1178)
[    0.310405] CPU1 attaching sched-domain:
[    0.310408]  domain 0: span 0-1 level SIBLING
[    0.310411]   groups: 1 (cpu_power = 589) 0 (cpu_power = 589)
[    0.310418]   domain 1: span 0-1 level MC
[    0.310420]    groups: 0-1 (cpu_power = 1178)
[    0.310580] devtmpfs: initialized
[    0.310580] regulator: core version 0.5
[    0.310580] Time: 10:18:35  Date: 07/27/10
[    0.310600] NET: Registered protocol family 16
[    0.310641] Trying to unpack rootfs image as initramfs...
[    0.310875] ACPI: bus type pci registered
[    0.310994] PCI: MCFG configuration 0: base e0000000 segment 0 buses 0 - 255
[    0.311000] PCI: MCFG area at e0000000 reserved in E820
[    0.321206] PCI: Using MMCONFIG at e0000000 - efffffff
[    0.321213] PCI: Using configuration type 1 for base access
[    0.323076] bio: create slab <bio-0> at 0
[    0.324571] ACPI: EC: Look up EC in DSDT
[    0.329130] ACPI: Executed 4 blocks of module-level executable AML code
[    0.339004] ACPI: Interpreter enabled
[    0.339012] ACPI: (supports S0 S1 S3 S4 S5)
[    0.339061] ACPI: Using IOAPIC for interrupt routing
[    0.352264] ACPI: Power Resource [URP1] (off)
[    0.352331] ACPI: Power Resource [FDDP] (off)
[    0.352400] ACPI: Power Resource [LPTP] (off)
[    0.352466] ACPI: Power Resource [URP2] (off)
[    0.352837] ACPI: No dock devices found.
[    0.355090] ACPI Warning for \_SB_.PCI0._OSC: Parameter count
mismatch - ASL declared 5, ACPI requires 4 (20090903/nspredef-336)
[    0.355106] ACPI: PCI Root Bridge [PCI0] (0000:00)
[    0.355295] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
[    0.355302] pci 0000:00:01.0: PME# disabled
[    0.355338] pci 0000:00:02.0: reg 10 32bit mmio: [0xff480000-0xff4fffff]
[    0.355347] pci 0000:00:02.0: reg 14 io port: [0xec00-0xec07]
[    0.355355] pci 0000:00:02.0: reg 18 32bit mmio pref: [0xd0000000-0xdfffffff]
[    0.355363] pci 0000:00:02.0: reg 1c 32bit mmio: [0xff440000-0xff47ffff]
[    0.355466] pci 0000:00:1b.0: reg 10 64bit mmio: [0xff43c000-0xff43ffff]
[    0.355521] pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold
[    0.355527] pci 0000:00:1b.0: PME# disabled
[    0.355612] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
[    0.355619] pci 0000:00:1c.0: PME# disabled
[    0.355704] pci 0000:00:1c.1: PME# supported from D0 D3hot D3cold
[    0.355711] pci 0000:00:1c.1: PME# disabled
[    0.355796] pci 0000:00:1c.2: PME# supported from D0 D3hot D3cold
[    0.355803] pci 0000:00:1c.2: PME# disabled
[    0.355887] pci 0000:00:1c.3: PME# supported from D0 D3hot D3cold
[    0.355894] pci 0000:00:1c.3: PME# disabled
[    0.355958] pci 0000:00:1d.0: reg 20 io port: [0xc800-0xc81f]
[    0.356026] pci 0000:00:1d.1: reg 20 io port: [0xcc00-0xcc1f]
[    0.356090] pci 0000:00:1d.2: reg 20 io port: [0xd000-0xd01f]
[    0.356154] pci 0000:00:1d.3: reg 20 io port: [0xd400-0xd41f]
[    0.356225] pci 0000:00:1d.7: reg 10 32bit mmio: [0xff43bc00-0xff43bfff]
[    0.356289] pci 0000:00:1d.7: PME# supported from D0 D3hot D3cold
[    0.356296] pci 0000:00:1d.7: PME# disabled
[    0.356443] pci 0000:00:1f.0: Force enabled HPET at 0xfed00000
[    0.356455] pci 0000:00:1f.0: quirk: region 0400-047f claimed by
ICH6 ACPI/GPIO/TCO
[    0.356461] pci 0000:00:1f.0: quirk: region 0500-053f claimed by ICH6 GPIO
[    0.356468] pci 0000:00:1f.0: LPC Generic IO decode 1 PIO at 0680-06ff
[    0.356474] pci 0000:00:1f.0: LPC Generic IO decode 2 PIO at 4700-470f
[    0.356507] pci 0000:00:1f.1: reg 10 io port: [0x00-0x07]
[    0.356517] pci 0000:00:1f.1: reg 14 io port: [0x00-0x03]
[    0.356526] pci 0000:00:1f.1: reg 18 io port: [0x00-0x07]
[    0.356535] pci 0000:00:1f.1: reg 1c io port: [0x00-0x03]
[    0.356545] pci 0000:00:1f.1: reg 20 io port: [0xffa0-0xffaf]
[    0.356600] pci 0000:00:1f.2: reg 10 io port: [0xe800-0xe807]
[    0.356610] pci 0000:00:1f.2: reg 14 io port: [0xe400-0xe403]
[    0.356618] pci 0000:00:1f.2: reg 18 io port: [0xe000-0xe007]
[    0.356627] pci 0000:00:1f.2: reg 1c io port: [0xdc00-0xdc03]
[    0.356636] pci 0000:00:1f.2: reg 20 io port: [0xd800-0xd80f]
[    0.356668] pci 0000:00:1f.2: PME# supported from D3hot
[    0.356674] pci 0000:00:1f.2: PME# disabled
[    0.356736] pci 0000:00:1f.3: reg 20 io port: [0xc400-0xc41f]
[    0.356828] pci 0000:00:01.0: bridge 32bit mmio: [0xffa00000-0xffafffff]
[    0.356835] pci 0000:00:01.0: bridge 32bit mmio pref: [0xcff00000-0xcfffffff]
[    0.356898] pci 0000:00:1c.0: bridge 32bit mmio: [0xff600000-0xff6fffff]
[    0.356908] pci 0000:00:1c.0: bridge 64bit mmio pref: [0xcfb00000-0xcfbfffff]
[    0.356970] pci 0000:00:1c.1: bridge 32bit mmio: [0xff700000-0xff7fffff]
[    0.356980] pci 0000:00:1c.1: bridge 64bit mmio pref: [0xcfc00000-0xcfcfffff]
[    0.357042] pci 0000:00:1c.2: bridge 32bit mmio: [0xff800000-0xff8fffff]
[    0.357052] pci 0000:00:1c.2: bridge 64bit mmio pref: [0xcfd00000-0xcfdfffff]
[    0.357112] pci 0000:00:1c.3: bridge 32bit mmio: [0xff900000-0xff9fffff]
[    0.357123] pci 0000:00:1c.3: bridge 64bit mmio pref: [0xcfe00000-0xcfefffff]
[    0.357158] pci 0000:06:00.0: reg 10 io port: [0xbc00-0xbc3f]
[    0.357196] pci 0000:06:00.0: reg 30 32bit mmio pref: [0xff500000-0xff50ffff]
[    0.357248] pci 0000:06:01.0: reg 10 32bit mmio: [0xff511000-0xff5110ff]
[    0.357259] pci 0000:06:01.0: reg 14 io port: [0xb800-0xb807]
[    0.357269] pci 0000:06:01.0: reg 18 io port: [0xb400-0xb4ff]
[    0.357318] pci 0000:06:01.0: supports D2
[    0.357323] pci 0000:06:01.0: PME# supported from D2 D3hot
[    0.357330] pci 0000:06:01.0: PME# disabled
[    0.357386] pci 0000:06:08.0: reg 10 32bit mmio: [0xff510000-0xff510fff]
[    0.357396] pci 0000:06:08.0: reg 14 io port: [0xb000-0xb03f]
[    0.357448] pci 0000:06:08.0: supports D1 D2
[    0.357452] pci 0000:06:08.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.357459] pci 0000:06:08.0: PME# disabled
[    0.357509] pci 0000:00:1e.0: transparent bridge
[    0.357516] pci 0000:00:1e.0: bridge io port: [0xb000-0xbfff]
[    0.357522] pci 0000:00:1e.0: bridge 32bit mmio: [0xff500000-0xff5fffff]
[    0.357532] pci 0000:00:1e.0: bridge 64bit mmio pref: [0xcfa00000-0xcfafffff]
[    0.357569] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
[    0.357767] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEGP._PRT]
[    0.357871] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P2._PRT]
[    0.358149] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX1._PRT]
[    0.358253] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX2._PRT]
[    0.358356] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX3._PRT]
[    0.358459] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX4._PRT]
[    0.360173] ACPI Warning for \_SB_.PCI0._OSC: Parameter count
mismatch - ASL declared 5, ACPI requires 4 (20090903/nspredef-336)
[    0.368564] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 9 10
*11 12 14 15)
[    0.368848] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 10 11
12 14 15) *0, disabled.
[    0.369126] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 *5 6 7 9 10
11 12 14 15)
[    0.369402] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 9 *10
11 12 14 15)
[    0.369677] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 9 *10
11 12 14 15)
[    0.369954] ACPI: PCI Interrupt Link [LNKF] (IRQs *3 4 5 6 7 9 10
11 12 14 15)
[    0.370254] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 *9 10
11 12 14 15)
[    0.370532] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 7 *9 10
11 12 14 15)
[    0.370792] vgaarb: device added:
PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none
[    0.370822] vgaarb: loaded
[    0.371032] SCSI subsystem initialized
[    0.371163] libata version 3.00 loaded.
[    0.371307] usbcore: registered new interface driver usbfs
[    0.371332] usbcore: registered new interface driver hub
[    0.371381] usbcore: registered new device driver usb
[    0.371627] ACPI: WMI: Mapper loaded
[    0.371630] PCI: Using ACPI for IRQ routing
[    0.371914] NetLabel: Initializing
[    0.371918] NetLabel:  domain hash size = 128
[    0.371922] NetLabel:  protocols = UNLABELED CIPSOv4
[    0.371947] NetLabel:  unlabeled traffic allowed by default
[    0.372095] hpet clockevent registered
[    0.372105] HPET: 3 timers in total, 0 timers will be used for per-cpu timer
[    0.372112] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[    0.372123] hpet0: 3 comparators, 64-bit 14.318180 MHz counter
[    2.962643] Switching to clocksource tsc
[    2.963674] AppArmor: AppArmor Filesystem Enabled
[    2.963722] pnp: PnP ACPI init
[    2.963772] ACPI: bus type pnp registered
[    2.971992] pnp: PnP ACPI: found 13 devices
[    2.972000] ACPI: ACPI bus type pnp unregistered
[    2.972046] system 00:08: ioport range 0x4d0-0x4d1 has been reserved
[    2.972053] system 00:08: ioport range 0x400-0x47f has been reserved
[    2.972059] system 00:08: ioport range 0x500-0x53f has been reserved
[    2.972080] system 00:0a: iomem range 0xffc00000-0xfff7ffff has been reserved
[    2.972093] system 00:0b: ioport range 0x400-0x47f has been reserved
[    2.972099] system 00:0b: ioport range 0x680-0x6ff has been reserved
[    2.972105] system 00:0b: ioport range 0x500-0x53f has been reserved
[    2.972111] system 00:0b: iomem range 0xeec00000-0xeec03fff has been reserved
[    2.972117] system 00:0b: iomem range 0xfec00000-0xfec00fff could
not be reserved
[    2.972123] system 00:0b: iomem range 0xfee00000-0xfee00fff has been reserved
[    2.972128] system 00:0b: iomem range 0xe0000000-0xefffffff could
not be reserved
[    2.972134] system 00:0b: iomem range 0xfed13000-0xfed13fff has been reserved
[    2.972140] system 00:0b: iomem range 0xfed14000-0xfed17fff has been reserved
[    2.972146] system 00:0b: iomem range 0xfed18000-0xfed18fff has been reserved
[    2.972151] system 00:0b: iomem range 0xfed19000-0xfed19fff has been reserved
[    2.972157] system 00:0b: iomem range 0xfed1c000-0xfed1ffff has been reserved
[    2.972163] system 00:0b: iomem range 0xfed20000-0xfed9ffff has been reserved
[    2.972176] system 00:0c: iomem range 0x0-0x9ffff could not be reserved
[    2.972181] system 00:0c: iomem range 0xc0000-0xdffff has been reserved
[    2.972187] system 00:0c: iomem range 0xe0000-0xfffff could not be reserved
[    2.972193] system 00:0c: iomem range 0x100000-0x7dffffff could not
be reserved
[    2.977269] pci 0000:06:00.0: BAR 6: address space collision on of
device [0xff500000-0xff50ffff]
[    2.977364] pci 0000:00:01.0: PCI bridge, secondary bus 0000:01
[    2.977368] pci 0000:00:01.0:   IO window: disabled
[    2.977376] pci 0000:00:01.0:   MEM window: 0xffa00000-0xffafffff
[    2.977382] pci 0000:00:01.0:   PREFETCH window: 0xcff00000-0xcfffffff
[    2.977391] pci 0000:00:1c.0: PCI bridge, secondary bus 0000:05
[    2.977397] pci 0000:00:1c.0:   IO window: 0x1000-0x1fff
[    2.977406] pci 0000:00:1c.0:   MEM window: 0xff600000-0xff6fffff
[    2.977412] pci 0000:00:1c.0:   PREFETCH window:
0x000000cfb00000-0x000000cfbfffff
[    2.977423] pci 0000:00:1c.1: PCI bridge, secondary bus 0000:04
[    2.977428] pci 0000:00:1c.1:   IO window: 0x2000-0x2fff
[    2.977436] pci 0000:00:1c.1:   MEM window: 0xff700000-0xff7fffff
[    2.977444] pci 0000:00:1c.1:   PREFETCH window:
0x000000cfc00000-0x000000cfcfffff
[    2.977454] pci 0000:00:1c.2: PCI bridge, secondary bus 0000:03
[    2.977459] pci 0000:00:1c.2:   IO window: 0x3000-0x3fff
[    2.977467] pci 0000:00:1c.2:   MEM window: 0xff800000-0xff8fffff
[    2.977474] pci 0000:00:1c.2:   PREFETCH window:
0x000000cfd00000-0x000000cfdfffff
[    2.977484] pci 0000:00:1c.3: PCI bridge, secondary bus 0000:02
[    2.977490] pci 0000:00:1c.3:   IO window: 0x4000-0x4fff
[    2.977499] pci 0000:00:1c.3:   MEM window: 0xff900000-0xff9fffff
[    2.977506] pci 0000:00:1c.3:   PREFETCH window:
0x000000cfe00000-0x000000cfefffff
[    2.977524] pci 0000:00:1e.0: PCI bridge, secondary bus 0000:06
[    2.977531] pci 0000:00:1e.0:   IO window: 0xb000-0xbfff
[    2.977538] pci 0000:00:1e.0:   MEM window: 0xff500000-0xff5fffff
[    2.977546] pci 0000:00:1e.0:   PREFETCH window:
0x000000cfa00000-0x000000cfafffff
[    2.977574]   alloc irq_desc for 16 on node -1
[    2.977580]   alloc kstat_irqs on node -1
[    2.977594] pci 0000:00:01.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    2.977602] pci 0000:00:01.0: setting latency timer to 64
[    2.977618] pci 0000:00:1c.0: enabling device (0106 -> 0107)
[    2.977625]   alloc irq_desc for 17 on node -1
[    2.977629]   alloc kstat_irqs on node -1
[    2.977637] pci 0000:00:1c.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    2.977645] pci 0000:00:1c.0: setting latency timer to 64
[    2.977659] pci 0000:00:1c.1: enabling device (0106 -> 0107)
[    2.977667] pci 0000:00:1c.1: PCI INT B -> GSI 16 (level, low) -> IRQ 16
[    2.977674] pci 0000:00:1c.1: setting latency timer to 64
[    2.977687] pci 0000:00:1c.2: enabling device (0106 -> 0107)
[    2.977694]   alloc irq_desc for 18 on node -1
[    2.977697]   alloc kstat_irqs on node -1
[    2.977704] pci 0000:00:1c.2: PCI INT C -> GSI 18 (level, low) -> IRQ 18
[    2.977712] pci 0000:00:1c.2: setting latency timer to 64
[    2.977725] pci 0000:00:1c.3: enabling device (0106 -> 0107)
[    2.977731]   alloc irq_desc for 19 on node -1
[    2.977735]   alloc kstat_irqs on node -1
[    2.977741] pci 0000:00:1c.3: PCI INT D -> GSI 19 (level, low) -> IRQ 19
[    2.977748] pci 0000:00:1c.3: setting latency timer to 64
[    2.977758] pci 0000:00:1e.0: setting latency timer to 64
[    2.977766] pci_bus 0000:00: resource 0 io:  [0x00-0xffff]
[    2.977771] pci_bus 0000:00: resource 1 mem: [0x000000-0xffffffffffffffff]
[    2.977775] pci_bus 0000:01: resource 1 mem: [0xffa00000-0xffafffff]
[    2.977780] pci_bus 0000:01: resource 2 pref mem [0xcff00000-0xcfffffff]
[    2.977784] pci_bus 0000:05: resource 0 io:  [0x1000-0x1fff]
[    2.977789] pci_bus 0000:05: resource 1 mem: [0xff600000-0xff6fffff]
[    2.977793] pci_bus 0000:05: resource 2 pref mem [0xcfb00000-0xcfbfffff]
[    2.977798] pci_bus 0000:04: resource 0 io:  [0x2000-0x2fff]
[    2.977802] pci_bus 0000:04: resource 1 mem: [0xff700000-0xff7fffff]
[    2.977806] pci_bus 0000:04: resource 2 pref mem [0xcfc00000-0xcfcfffff]
[    2.977811] pci_bus 0000:03: resource 0 io:  [0x3000-0x3fff]
[    2.977815] pci_bus 0000:03: resource 1 mem: [0xff800000-0xff8fffff]
[    2.977819] pci_bus 0000:03: resource 2 pref mem [0xcfd00000-0xcfdfffff]
[    2.977824] pci_bus 0000:02: resource 0 io:  [0x4000-0x4fff]
[    2.977828] pci_bus 0000:02: resource 1 mem: [0xff900000-0xff9fffff]
[    2.977832] pci_bus 0000:02: resource 2 pref mem [0xcfe00000-0xcfefffff]
[    2.977839] pci_bus 0000:06: resource 0 io:  [0xb000-0xbfff]
[    2.977843] pci_bus 0000:06: resource 1 mem: [0xff500000-0xff5fffff]
[    2.977847] pci_bus 0000:06: resource 2 pref mem [0xcfa00000-0xcfafffff]
[    2.977851] pci_bus 0000:06: resource 3 io:  [0x00-0xffff]
[    2.977855] pci_bus 0000:06: resource 4 mem: [0x000000-0xffffffffffffffff]
[    2.977937] NET: Registered protocol family 2
[    2.978247] IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
[    2.980101] TCP established hash table entries: 262144 (order: 10,
4194304 bytes)
[    2.983649] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    2.984430] TCP: Hash tables configured (established 262144 bind 65536)
[    2.984439] TCP reno registered
[    2.984674] NET: Registered protocol family 1
[    2.984726] pci 0000:00:02.0: Boot video device
[    2.984912] pci 0000:06:08.0: Firmware left e100 interrupts
enabled; disabling
[    2.985413] Scanning for low memory corruption every 60 seconds
[    2.985738] audit: initializing netlink socket (disabled)
[    2.985764] type=2000 audit(1280225917.980:1): initialized
[    2.997279] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    3.000425] VFS: Disk quotas dquot_6.5.2
[    3.000576] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    3.002018] fuse init (API version 7.13)
[    3.002225] msgmni has been set to 3933
[    3.002836] alg: No test for stdrng (krng)
[    3.003017] Block layer SCSI generic (bsg) driver version 0.4
loaded (major 253)
[    3.003024] io scheduler noop registered
[    3.003028] io scheduler anticipatory registered
[    3.003032] io scheduler deadline registered
[    3.003111] io scheduler cfq registered (default)
[    3.003383]   alloc irq_desc for 24 on node -1
[    3.003388]   alloc kstat_irqs on node -1
[    3.003405] pcieport 0000:00:01.0: irq 24 for MSI/MSI-X
[    3.003416] pcieport 0000:00:01.0: setting latency timer to 64
[    3.003579]   alloc irq_desc for 25 on node -1
[    3.003583]   alloc kstat_irqs on node -1
[    3.003595] pcieport 0000:00:1c.0: irq 25 for MSI/MSI-X
[    3.003607] pcieport 0000:00:1c.0: setting latency timer to 64
[    3.003784]   alloc irq_desc for 26 on node -1
[    3.003788]   alloc kstat_irqs on node -1
[    3.003800] pcieport 0000:00:1c.1: irq 26 for MSI/MSI-X
[    3.003812] pcieport 0000:00:1c.1: setting latency timer to 64
[    3.004000]   alloc irq_desc for 27 on node -1
[    3.004004]   alloc kstat_irqs on node -1
[    3.004015] pcieport 0000:00:1c.2: irq 27 for MSI/MSI-X
[    3.004027] pcieport 0000:00:1c.2: setting latency timer to 64
[    3.004210]   alloc irq_desc for 28 on node -1
[    3.004215]   alloc kstat_irqs on node -1
[    3.004228] pcieport 0000:00:1c.3: irq 28 for MSI/MSI-X
[    3.004240] pcieport 0000:00:1c.3: setting latency timer to 64
[    3.004409] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    3.004445] Firmware did not grant requested _OSC control
[    3.004482] Firmware did not grant requested _OSC control
[    3.004512] Firmware did not grant requested _OSC control
[    3.004541] Firmware did not grant requested _OSC control
[    3.004602] Firmware did not grant requested _OSC control
[    3.004629] Firmware did not grant requested _OSC control
[    3.004655] Firmware did not grant requested _OSC control
[    3.004681] Firmware did not grant requested _OSC control
[    3.004710] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
[    3.004934] input: Power Button as
/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0
[    3.004953] ACPI: Power Button [PWRB]
[    3.005046] input: Power Button as
/devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
[    3.005052] ACPI: Power Button [PWRF]
[    3.005777] processor LNXCPU:00: registered as cooling_device0
[    3.005882] processor LNXCPU:01: registered as cooling_device1
[    3.013381] Linux agpgart interface v0.103
[    3.013471] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    3.013641] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[    3.014204] 00:06: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[    3.016522] brd: module loaded
[    3.017565] loop: module loaded
[    3.017784] input: Macintosh mouse button emulation as
/devices/virtual/input/input2
[    3.018001] ata_piix 0000:00:1f.1: version 2.13
[    3.018031] ata_piix 0000:00:1f.1: PCI INT A -> GSI 18 (level, low) -> IRQ 18
[    3.018110] ata_piix 0000:00:1f.1: setting latency timer to 64
[    3.018307] scsi0 : ata_piix
[    3.018517] scsi1 : ata_piix
[    3.020727] ata1: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0xffa0 irq 14
[    3.020737] ata2: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0xffa8 irq 15
[    3.020887] ata_piix 0000:00:1f.2: PCI INT B -> GSI 19 (level, low) -> IRQ 19
[    3.020898] ata_piix 0000:00:1f.2: MAP [ P0 P2 P1 P3 ]
[    3.020989] ata_piix 0000:00:1f.2: setting latency timer to 64
[    3.021109] scsi2 : ata_piix
[    3.021544] scsi3 : ata_piix
[    3.024468] ata3: SATA max UDMA/133 cmd 0xe800 ctl 0xe400 bmdma 0xd800 irq 19
[    3.024476] ata4: SATA max UDMA/133 cmd 0xe000 ctl 0xdc00 bmdma 0xd808 irq 19
[    3.025398] Fixed MDIO Bus: probed
[    3.025482] PPP generic driver version 2.4.2
[    3.025599] tun: Universal TUN/TAP device driver, 1.6
[    3.025605] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[    3.025821] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    3.025882]   alloc irq_desc for 23 on node -1
[    3.025886]   alloc kstat_irqs on node -1
[    3.025898] ehci_hcd 0000:00:1d.7: PCI INT A -> GSI 23 (level, low) -> IRQ 23
[    3.025936] ehci_hcd 0000:00:1d.7: setting latency timer to 64
[    3.025943] ehci_hcd 0000:00:1d.7: EHCI Host Controller
[    3.026018] ehci_hcd 0000:00:1d.7: new USB bus registered, assigned
bus number 1
[    3.026065] ehci_hcd 0000:00:1d.7: debug port 1
[    3.029957] ehci_hcd 0000:00:1d.7: cache line size of 128 is not supported
[    3.030007] ehci_hcd 0000:00:1d.7: irq 23, io mem 0xff43bc00
[    3.052534] ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00
[    3.052755] usb usb1: configuration #1 chosen from 1 choice
[    3.052809] hub 1-0:1.0: USB hub found
[    3.052822] hub 1-0:1.0: 8 ports detected
[    3.052950] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    3.052984] uhci_hcd: USB Universal Host Controller Interface driver
[    3.053045] uhci_hcd 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23
[    3.053058] uhci_hcd 0000:00:1d.0: setting latency timer to 64
[    3.053064] uhci_hcd 0000:00:1d.0: UHCI Host Controller
[    3.053138] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned
bus number 2
[    3.053172] uhci_hcd 0000:00:1d.0: irq 23, io base 0x0000c800
[    3.053334] usb usb2: configuration #1 chosen from 1 choice
[    3.053384] hub 2-0:1.0: USB hub found
[    3.053395] hub 2-0:1.0: 2 ports detected
[    3.053479] uhci_hcd 0000:00:1d.1: PCI INT B -> GSI 19 (level, low) -> IRQ 19
[    3.053489] uhci_hcd 0000:00:1d.1: setting latency timer to 64
[    3.053494] uhci_hcd 0000:00:1d.1: UHCI Host Controller
[    3.053556] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned
bus number 3
[    3.053587] uhci_hcd 0000:00:1d.1: irq 19, io base 0x0000cc00
[    3.053745] usb usb3: configuration #1 chosen from 1 choice
[    3.053793] hub 3-0:1.0: USB hub found
[    3.053804] hub 3-0:1.0: 2 ports detected
[    3.053881] uhci_hcd 0000:00:1d.2: PCI INT C -> GSI 18 (level, low) -> IRQ 18
[    3.053895] uhci_hcd 0000:00:1d.2: setting latency timer to 64
[    3.053900] uhci_hcd 0000:00:1d.2: UHCI Host Controller
[    3.053961] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned
bus number 4
[    3.054004] uhci_hcd 0000:00:1d.2: irq 18, io base 0x0000d000
[    3.054157] usb usb4: configuration #1 chosen from 1 choice
[    3.054209] hub 4-0:1.0: USB hub found
[    3.054221] hub 4-0:1.0: 2 ports detected
[    3.054297] uhci_hcd 0000:00:1d.3: PCI INT D -> GSI 16 (level, low) -> IRQ 16
[    3.054306] uhci_hcd 0000:00:1d.3: setting latency timer to 64
[    3.054312] uhci_hcd 0000:00:1d.3: UHCI Host Controller
[    3.054369] uhci_hcd 0000:00:1d.3: new USB bus registered, assigned
bus number 5
[    3.054409] uhci_hcd 0000:00:1d.3: irq 16, io base 0x0000d400
[    3.054563] usb usb5: configuration #1 chosen from 1 choice
[    3.054614] hub 5-0:1.0: USB hub found
[    3.054625] hub 5-0:1.0: 2 ports detected
[    3.054790] PNP: PS/2 Controller [PNP0303:PS2K] at 0x60,0x64 irq 1
[    3.054795] PNP: PS/2 appears to have AUX port disabled, if this is
incorrect please boot with i8042.nopnp
[    3.055521] serio: i8042 KBD port at 0x60,0x64 irq 1
[    3.055722] mice: PS/2 mouse device common for all mice
[    3.055936] rtc_cmos 00:02: RTC can wake from S4
[    3.056011] rtc_cmos 00:02: rtc core: registered rtc_cmos as rtc0
[    3.056045] rtc0: alarms up to one month, 114 bytes nvram, hpet irqs
[    3.056311] device-mapper: uevent: version 1.0.3
[    3.056498] device-mapper: ioctl: 4.15.0-ioctl (2009-04-01)
initialised: dm-devel@redhat.com
[    3.056615] device-mapper: multipath: version 1.1.0 loaded
[    3.056620] device-mapper: multipath round-robin: version 1.0.0 loaded
[    3.056890] cpuidle: using governor ladder
[    3.056894] cpuidle: using governor menu
[    3.057659] TCP cubic registered
[    3.057921] NET: Registered protocol family 10
[    3.058727] lo: Disabled Privacy Extensions
[    3.059292] NET: Registered protocol family 17
[    3.059529] PM: Resume from disk failed.
[    3.059551] registered taskstats version 1
[    3.059995]   Magic number: 2:366:326
[    3.060115] rtc_cmos 00:02: setting system clock to 2010-07-27
10:18:38 UTC (1280225918)
[    3.060122] BIOS EDD facility v0.16 2004-Jun-25, 0 devices found
[    3.060125] EDD information not available.
[    3.077271] input: AT Translated Set 2 keyboard as
/devices/platform/i8042/serio0/input/input3
[    3.204995] ata1.00: ATAPI: HL-DT-ST DVDRAM XXXXX, A103, max UDMA/33
[    3.223305] ata4.01: ATA-8: XXX WD1001XXXXX, XXXXX, max UDMA/133
[    3.223311] ata4.01: XXXXXXXXXXX sectors, multi 16: LBA48 NCQ (depth 0/32)
[    3.223905] ata3.00: ATA-8: WDC WD1002XXXXX, XXXXX, max UDMA/133
[    3.223911] ata3.00: XXXXXXXXXXX sectors, multi 16: LBA48 NCQ (depth 0/32)
[    3.242817] ata1.00: configured for UDMA/33
[    3.263110] ata3.00: configured for UDMA/133
[    3.263174] ata4.01: configured for UDMA/133
[    3.267721] scsi 0:0:0:0: CD-ROM            HL-DT-ST DVDRAM XXXXX
A103 PQ: 0 ANSI: 5
[    3.273265] sr0: scsi3-mmc drive: 40x/40x writer dvd-ram cd/rw
xa/form2 cdda tray
[    3.273272] Uniform CD-ROM driver Revision: 3.20
[    3.273455] sr 0:0:0:0: Attached scsi CD-ROM sr0
[    3.273559] sr 0:0:0:0: Attached scsi generic sg0 type 5
[    3.273792] scsi 2:0:0:0: Direct-Access     ATA      WDC WD1002XXX0
05.0 PQ: 0 ANSI: 5
[    3.273989] sd 2:0:0:0: Attached scsi generic sg1 type 0
[    3.274170] scsi 3:0:1:0: Direct-Access     ATA      WDC WD1001XXX0
05.0 PQ: 0 ANSI: 5
[    3.274360] sd 3:0:1:0: Attached scsi generic sg2 type 0
[    3.274466] sd 3:0:1:0: [sdb] XXXXXXXXXXX 512-byte logical blocks:
(X.XX XX/XXX XXX)
[    3.274515] sd 2:0:0:0: [sda] XXXXXXXXXXX 512-byte logical blocks:
(X.XX XX/XXX XXX)
[    3.274637] sd 3:0:1:0: [sdb] Write Protect is off
[    3.274642] sd 3:0:1:0: [sdb] Mode Sense: 00 3a 00 00
[    3.274685] sd 2:0:0:0: [sda] Write Protect is off
[    3.274690] sd 2:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    3.274734] sd 3:0:1:0: [sdb] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[    3.274850] sd 2:0:0:0: [sda] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[    3.275202]  sda:
[    3.275309]  sdb: sdb1 sdb2
[    3.286257] sd 3:0:1:0: [sdb] Attached SCSI disk
[    3.287038]  sda1 sda2
[    3.287417] sd 2:0:0:0: [sda] Attached SCSI disk
[    3.486486] Freeing initrd memory: 9607k freed
[    3.492690] Freeing unused kernel memory: 876k freed
[    3.493264] Write protecting the kernel read-only data: 7680k
[    3.524986] udev: starting version 151
[    3.592650] usb 2-1: new low speed USB device using uhci_hcd and address 2
[    3.611879] agpgart-intel 0000:00:00.0: Intel 915G Chipset
[    3.612325] agpgart-intel 0000:00:00.0: detected 32508K stolen memory
[    3.729003] agpgart-intel 0000:00:00.0: AGP aperture is 256M @ 0xd0000000
[    3.749672] e100: Intel(R) PRO/100 Network Driver, 3.5.24-k2-NAPI
[    3.749678] e100: Copyright(c) 1999-2006 Intel Corporation
[    3.749747]   alloc irq_desc for 20 on node -1
[    3.749754]   alloc kstat_irqs on node -1
[    3.749768] e100 0000:06:08.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20
[    3.774035] e100 0000:06:08.0: PME# disabled
[    3.774219] usb 2-1: configuration #1 chosen from 1 choice
[    3.782942] e100: eth0: e100_probe: addr 0xff510000, irq 20, MAC
addr 00:XX:XX:XX:XX:XX
[    3.782990]   alloc irq_desc for 21 on node -1
[    3.782994]   alloc kstat_irqs on node -1
[    3.783006] 3c59x 0000:06:00.0: PCI INT A -> GSI 21 (level, low) -> IRQ 21
[    3.783012] 3c59x: Donald Becker and others.
[    3.783020] 0000:06:00.0: 3Com PCI 3c905 Boomerang 100baseTx at
000000000001bc00.
[    3.789356] [drm] Initialized drm 1.1.0 20060810
[    3.830889] i915 0000:00:02.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    3.830898] i915 0000:00:02.0: setting latency timer to 64
[    3.842987] [drm] set up 31M of stolen space
[    3.844011] [drm] initialized overlay support
[    3.850924] usbcore: registered new interface driver hiddev
[    3.863525] input: HID 062a:0001 as
/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1:1.0/input/input4
[    3.863777] generic-usb 0003:062A:0001.0001: input,hidraw0: USB HID
v1.10 Mouse [HID 062a:0001] on usb-0000:00:1d.0-1/input0
[    3.863821] usbcore: registered new interface driver usbhid
[    3.863880] usbhid: v2.6:USB HID core driver
[    4.024392] fb0: inteldrmfb frame buffer device
[    4.024396] registered panic notifier
[    4.024406] [drm] Initialized i915 1.6.0 20080730 for 0000:00:02.0 on minor 0
[    4.027240] vga16fb: initializing
[    4.027248] vga16fb: mapped to 0xffff8800000a0000
[    4.027257] vga16fb: not registering due to another framebuffer present
[    4.159860] Console: switching to colour frame buffer device 240x90
[    5.211492] xor: automatically using best checksumming function: generic_sse
[    5.260009]    generic_sse:  4654.000 MB/sec
[    5.260012] xor: using function: generic_sse (4654.000 MB/sec)
[    5.265763] device-mapper: dm-raid45: initialized v0.2594b
[    7.298727] ISO 9660 Extensions: Microsoft Joliet Level 3
[    7.359461] ISO 9660 Extensions: RRIP_1991A
[    7.690903] aufs 2-standalone.tree-20091207
[    7.894415] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[   66.277270] udev: starting version 151
[   70.278187] intel_rng: Firmware space is locked read-only. If you can't or
[   70.278191] intel_rng: don't want to disable this in firmware setup, and if
[   70.278194] intel_rng: you are certain that your system has a functional
[   70.278196] intel_rng: RNG, try using the 'no_fwh_detect' option.
[   71.220160] parport_pc 00:07: reported by Plug and Play ACPI
[   71.220202] parport0: PC-style at 0x378, irq 7 [PCSPP,TRISTATE,EPP]
[   72.292606] ppdev: user-space parallel port driver
[   74.597802] eth1:  setting half-duplex.
[   74.598240] ADDRCONF(NETDEV_UP): eth1: link is not ready
[   74.641281] ADDRCONF(NETDEV_UP): eth0: link is not ready
[   74.650159] e100: eth0 NIC Link is Up 100 Mbps Full Duplex
[   74.650636] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   76.063072] HDA Intel 0000:00:1b.0: PCI INT A -> GSI 16 (level,
low) -> IRQ 16
[   76.063127] HDA Intel 0000:00:1b.0: setting latency timer to 64
[   76.391707] input: HDA Digital PCBeep as
/devices/pci0000:00/0000:00:1b.0/input/input5
[   84.730015] eth0: no IPv6 routers present
[   88.239423] lp0: using parport0 (interrupt-driven).

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[KOSAKI Motohiro]

> On 27 July 2010 18:09, dave b <db.pub.mail@gmail.com> wrote:
> > On 27 July 2010 16:09, KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
> >>> > Do you mean the issue will be gone if disabling intel graphics?
> >>> It may be a general issue or it could just be specific :)
> >
> > I will try with the latest ubuntu and report how that goes (that will
> > be using fairly new xorg etc.) it is likely to be hidden issue just
> > with the intel graphics driver. However, my concern is that it isn't -
> > and it is about how shared graphics memory is handled :)
> 
> 
> Ok my desktop still stalled and no oom killer was invoked when I added
> swap to a live-cd of 10.04 amd64.
> 
> *Without* *swap* *on* - the oom killer was invoked - here is a copy of it.

This stack seems similar following bug. can you please try to disable intel graphics
driver?

https://bugzilla.kernel.org/show_bug.cgi?id=14933


> [  298.180542] Xorg invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0
> [  298.180553] Xorg cpuset=/ mems_allowed=0
> [  298.180560] Pid: 3808, comm: Xorg Not tainted 2.6.32-21-generic #32-Ubuntu
> [  298.180564] Call Trace:
> [  298.180583]  [<ffffffff810b37cd>] ? cpuset_print_task_mems_allowed+0x9d/0xb0
> [  298.180595]  [<ffffffff810f64f4>] oom_kill_process+0xd4/0x2f0
> [  298.180603]  [<ffffffff810f6ab0>] ? select_bad_process+0xd0/0x110
> [  298.180609]  [<ffffffff810f6b48>] __out_of_memory+0x58/0xc0
> [  298.180616]  [<ffffffff810f6cde>] out_of_memory+0x12e/0x1a0
> [  298.180626]  [<ffffffff81540c9e>] ? _spin_lock+0xe/0x20
> [  298.180633]  [<ffffffff810f9d21>] __alloc_pages_slowpath+0x511/0x580
> [  298.180641]  [<ffffffff810f9eee>] __alloc_pages_nodemask+0x15e/0x1a0
> [  298.180650]  [<ffffffff8112ca57>] alloc_pages_current+0x87/0xd0
> [  298.180657]  [<ffffffff810f8e0e>] __get_free_pages+0xe/0x50
> [  298.180666]  [<ffffffff81154994>] __pollwait+0xb4/0xf0
> [  298.180673]  [<ffffffff814e09a5>] unix_poll+0x25/0xc0
> [  298.180682]  [<ffffffff81449bea>] sock_poll+0x1a/0x20
> [  298.180688]  [<ffffffff811545b2>] do_select+0x3a2/0x6d0
> [  298.180696]  [<ffffffff811548e0>] ? __pollwait+0x0/0xf0
> [  298.180702]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
> [  298.180708]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
> [  298.180714]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
> [  298.180721]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
> [  298.180727]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
> [  298.180732]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
> [  298.180737]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
> [  298.180741]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
> [  298.180745]  [<ffffffff811549d0>] ? pollwake+0x0/0x60
> [  298.180749]  [<ffffffff811550ba>] core_sys_select+0x18a/0x2c0
> [  298.180777]  [<ffffffffa001eced>] ? drm_ioctl+0x13d/0x480 [drm]
> [  298.180784]  [<ffffffff81085320>] ? autoremove_wake_function+0x0/0x40
> [  298.180790]  [<ffffffff810397a9>] ? default_spin_lock_flags+0x9/0x10
> [  298.180795]  [<ffffffff81540bbf>] ? _spin_lock_irqsave+0x2f/0x40
> [  298.180800]  [<ffffffff81019e89>] ? read_tsc+0x9/0x20
> [  298.180805]  [<ffffffff8108f9c9>] ? ktime_get_ts+0xa9/0xe0
> [  298.180810]  [<ffffffff81155447>] sys_select+0x47/0x110
> [  298.180816]  [<ffffffff810131b2>] system_call_fastpath+0x16/0x1b
> [  298.180819] Mem-Info:
> [  298.180822] Node 0 DMA per-cpu:
> [  298.180827] CPU    0: hi:    0, btch:   1 usd:   0
> [  298.180830] CPU    1: hi:    0, btch:   1 usd:   0
> [  298.180832] Node 0 DMA32 per-cpu:
> [  298.180837] CPU    0: hi:  186, btch:  31 usd:  60
> [  298.180839] CPU    1: hi:  186, btch:  31 usd: 137
> [  298.180845] active_anon:374344 inactive_anon:81753 isolated_anon:0
> [  298.180847]  active_file:7038 inactive_file:7089 isolated_file:0
> [  298.180848]  unevictable:0 dirty:0 writeback:0 unstable:0
> [  298.180849]  free:3399 slab_reclaimable:4226 slab_unreclaimable:4383
> [  298.180851]  mapped:13010 shmem:45284 pagetables:5496 bounce:0
> [  298.180854] Node 0 DMA free:7920kB min:40kB low:48kB high:60kB
> active_anon:3880kB inactive_anon:4096kB active_file:0kB
> inactive_file:0kB unevictable:0kB isolated(anon):0kB
> isolated(file):0kB present:15348kB mlocked:0kB dirty:0kB writeback:0kB
> mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:8kB
> kernel_stack:0kB pagetables:16kB unstable:0kB bounce:0kB
> writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
> [  298.180866] lowmem_reserve[]: 0 1971 1971 1971
> [  298.180871] Node 0 DMA32 free:5676kB min:5660kB low:7072kB
> high:8488kB active_anon:1493496kB inactive_anon:322916kB
> active_file:28152kB inactive_file:28356kB unevictable:0kB
> isolated(anon):0kB isolated(file):0kB present:2019172kB mlocked:0kB
> dirty:0kB writeback:0kB mapped:52040kB shmem:181136kB
> slab_reclaimable:16904kB slab_unreclaimable:17524kB
> kernel_stack:2096kB pagetables:21968kB unstable:0kB bounce:0kB
> writeback_tmp:0kB pages_scanned:41088 all_unreclaimable? no
> [  298.180884] lowmem_reserve[]: 0 0 0 0
> [  298.180889] Node 0 DMA: 4*4kB 2*8kB 1*16kB 2*32kB 2*64kB 2*128kB
> 1*256kB 2*512kB 2*1024kB 2*2048kB 0*4096kB = 7920kB
> [  298.180904] Node 0 DMA32: 397*4kB 1*8kB 1*16kB 1*32kB 1*64kB
> 1*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 5676kB
> [  298.180918] 59413 total pagecache pages
> [  298.180920] 0 pages in swap cache
> [  298.180923] Swap cache stats: add 0, delete 0, find 0/0
> [  298.180925] Free swap  = 0kB
> [  298.180927] Total swap = 0kB
> [  298.188124] 515887 pages RAM
> [  298.188127] 9764 pages reserved
> [  298.188129] 108553 pages shared
> [  298.188131] 467319 pages non-shared
> [  298.188136] Out of memory: kill process 3821 (gnome-session) score
> 503983 or a child
> [  298.188141] Killed process 3855 (ssh-agent)

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[dave b]

On 27 July 2010 21:14, KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
>> On 27 July 2010 18:09, dave b <db.pub.mail@gmail.com> wrote:
>> > On 27 July 2010 16:09, KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
>> >>> > Do you mean the issue will be gone if disabling intel graphics?
>> >>> It may be a general issue or it could just be specific :)
>> >
>> > I will try with the latest ubuntu and report how that goes (that will
>> > be using fairly new xorg etc.) it is likely to be hidden issue just
>> > with the intel graphics driver. However, my concern is that it isn't -
>> > and it is about how shared graphics memory is handled :)
>>
>>
>> Ok my desktop still stalled and no oom killer was invoked when I added
>> swap to a live-cd of 10.04 amd64.
>>
>> *Without* *swap* *on* - the oom killer was invoked - here is a copy of it.
>
> This stack seems similar following bug. can you please try to disable intel graphics
> driver?
>
> https://bugzilla.kernel.org/show_bug.cgi?id=14933

Ok I am not sure how to do that :)
I could revert the patch and see if it 'fixes' this :)

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[KOSAKI Motohiro]

> On 27 July 2010 21:14, KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
> >> On 27 July 2010 18:09, dave b <db.pub.mail@gmail.com> wrote:
> >> > On 27 July 2010 16:09, KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
> >> >>> > Do you mean the issue will be gone if disabling intel graphics?
> >> >>> It may be a general issue or it could just be specific :)
> >> >
> >> > I will try with the latest ubuntu and report how that goes (that will
> >> > be using fairly new xorg etc.) it is likely to be hidden issue just
> >> > with the intel graphics driver. However, my concern is that it isn't -
> >> > and it is about how shared graphics memory is handled :)
> >>
> >>
> >> Ok my desktop still stalled and no oom killer was invoked when I added
> >> swap to a live-cd of 10.04 amd64.
> >>
> >> *Without* *swap* *on* - the oom killer was invoked - here is a copy of it.
> >
> > This stack seems similar following bug. can you please try to disable intel graphics
> > driver?
> >
> > https://bugzilla.kernel.org/show_bug.cgi?id=14933
> 
> Ok I am not sure how to do that :)
> I could revert the patch and see if it 'fixes' this :)

Oops, no, revert is not good action. the patch is correct. 
probably my explanation was not clear. sorry.

I did hope to disable 'driver' (i.e. using vga), not disable the patch.

Thanks.

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[dave b]

On 28 July 2010 15:06, KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
>> On 27 July 2010 21:14, KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
>> >> On 27 July 2010 18:09, dave b <db.pub.mail@gmail.com> wrote:
>> >> > On 27 July 2010 16:09, KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
>> >> >>> > Do you mean the issue will be gone if disabling intel graphics?
>> >> >>> It may be a general issue or it could just be specific :)
>> >> >
>> >> > I will try with the latest ubuntu and report how that goes (that will
>> >> > be using fairly new xorg etc.) it is likely to be hidden issue just
>> >> > with the intel graphics driver. However, my concern is that it isn't -
>> >> > and it is about how shared graphics memory is handled :)
>> >>
>> >>
>> >> Ok my desktop still stalled and no oom killer was invoked when I added
>> >> swap to a live-cd of 10.04 amd64.
>> >>
>> >> *Without* *swap* *on* - the oom killer was invoked - here is a copy of it.
>> >
>> > This stack seems similar following bug. can you please try to disable intel graphics
>> > driver?
>> >
>> > https://bugzilla.kernel.org/show_bug.cgi?id=14933
>>
>> Ok I am not sure how to do that :)
>> I could revert the patch and see if it 'fixes' this :)
>
> Oops, no, revert is not good action. the patch is correct.
> probably my explanation was not clear. sorry.
>
> I did hope to disable 'driver' (i.e. using vga), not disable the patch.

Oh you mean in xorg, I will also blacklist the module. Sure that patch
might not it but in 2.6.26 the problem isn't there :)

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[dave b]

On 28 July 2010 17:14, dave b <db.pub.mail@gmail.com> wrote:
> On 28 July 2010 15:06, KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
>>> On 27 July 2010 21:14, KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
>>> >> On 27 July 2010 18:09, dave b <db.pub.mail@gmail.com> wrote:
>>> >> > On 27 July 2010 16:09, KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
>>> >> >>> > Do you mean the issue will be gone if disabling intel graphics?
>>> >> >>> It may be a general issue or it could just be specific :)
>>> >> >
>>> >> > I will try with the latest ubuntu and report how that goes (that will
>>> >> > be using fairly new xorg etc.) it is likely to be hidden issue just
>>> >> > with the intel graphics driver. However, my concern is that it isn't -
>>> >> > and it is about how shared graphics memory is handled :)
>>> >>
>>> >>
>>> >> Ok my desktop still stalled and no oom killer was invoked when I added
>>> >> swap to a live-cd of 10.04 amd64.
>>> >>
>>> >> *Without* *swap* *on* - the oom killer was invoked - here is a copy of it.
>>> >
>>> > This stack seems similar following bug. can you please try to disable intel graphics
>>> > driver?
>>> >
>>> > https://bugzilla.kernel.org/show_bug.cgi?id=14933
>>>
>>> Ok I am not sure how to do that :)
>>> I could revert the patch and see if it 'fixes' this :)
>>
>> Oops, no, revert is not good action. the patch is correct.
>> probably my explanation was not clear. sorry.
>>
>> I did hope to disable 'driver' (i.e. using vga), not disable the patch.
>
> Oh you mean in xorg, I will also blacklist the module. Sure that patch
> might not it but in 2.6.26 the problem isn't there :)

Ok I re-tested with 2.6.26 and 2.6.34.1
So I will describe what happens below:

2.6.26 - with xorg running
"Given I have a test file called a.out
 And I can see Xorg
 And I am using 2.6.26
 And I have swap on
 When I run it I run a.out
 Then I see the system freeze up slightly
 And the hard drive churns( and the cpu is doing something as the
large fan kicks)
 And after a while the system unfreezes"

2.6.26 - from single mode - before xorg starts and i915 is *not* loaded.
"Given I have a test file called a.out
 And I cannot see Xorg
 And I am using 2.6.26
 And I have swap on
 When I run it I run a.out
 Then I see the system freeze up
 And the system fan doesn't spin any faster
 And the system just sits idle"

2.6.34.1
With and without xorg - WITH spam on the same behaviour as in the
2.6.26 kernel appears (when xorg is not loaded).

OOM attached from the 2.6.26 kernel when I used magic keys to invoke
the oom killer :) (this was on the 2.6.26 kernel - before i915 had
loaded and in single mode).

[  280.323899] SysRq : Manual OOM execution
[  280.324009] events/0 invoked oom-killer: gfp_mask=0xd0, order=0, oomkilladj=0
[  280.324056] Pid: 9, comm: events/0 Not tainted 2.6.26-2-amd64 #1
[  280.324098]
[  280.324099] Call Trace:
[  280.324200]  [<ffffffff8027388c>] oom_kill_process+0x57/0x1dc
[  280.324247]  [<ffffffff8023b49d>] __capable+0x9/0x1c
[  280.324290]  [<ffffffff80273bb7>] badness+0x188/0x1c7
[  280.324341]  [<ffffffff80273deb>] out_of_memory+0x1f5/0x28e
[  280.324396]  [<ffffffff8037824c>] moom_callback+0x0/0x1a
[  280.324449]  [<ffffffff80243070>] run_workqueue+0x82/0x111
[  280.324497]  [<ffffffff8024393d>] worker_thread+0xd5/0xe0
[  280.324543]  [<ffffffff80246171>] autoremove_wake_function+0x0/0x2e
[  280.324596]  [<ffffffff80243868>] worker_thread+0x0/0xe0
[  280.324637]  [<ffffffff8024604b>] kthread+0x47/0x74
[  280.324678]  [<ffffffff802300ed>] schedule_tail+0x27/0x5c
[  280.326721]  [<ffffffff8020cf38>] child_rip+0xa/0x12
[  280.326788]  [<ffffffff80246004>] kthread+0x0/0x74
[  280.326831]  [<ffffffff8020cf2e>] child_rip+0x0/0x12
[  280.326877]
[  280.326911] Mem-info:
[  280.326948] Node 0 DMA per-cpu:
[  280.327010] CPU    0: hi:    0, btch:   1 usd:   0
[  280.327050] CPU    1: hi:    0, btch:   1 usd:   0
[  280.327090] Node 0 DMA32 per-cpu:
[  280.327151] CPU    0: hi:  186, btch:  31 usd: 139
[  280.327192] CPU    1: hi:  186, btch:  31 usd: 175
[  280.327234] Active:31 inactive:491089 dirty:0 writeback:23480 unstable:0
[  280.327235]  free:3005 slab:7514 mapped:1 pagetables:1063 bounce:0
[  280.327318] Node 0 DMA free:7880kB min:28kB low:32kB high:40kB
active:0kB inactive:3620kB present:10788kB pages_scanned:16863
all_unreclaimable? yes
[  280.327404] lowmem_reserve[]: 0 1971 1971 1971
[  280.327553] Node 0 DMA32 free:4140kB min:5664kB low:7080kB
high:8496kB active:124kB inactive:1960736kB present:2019172kB
pages_scanned:782860 all_unreclaimable? no
[  280.327651] lowmem_reserve[]: 0 0 0 0
[  280.327686] Node 0 DMA: 4*4kB 3*8kB 2*16kB 4*32kB 2*64kB 1*128kB
3*256kB 3*512kB 1*1024kB 0*2048kB 1*4096kB = 7880kB
[  280.327686] Node 0 DMA32: 477*4kB 7*8kB 0*16kB 2*32kB 3*64kB
1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 4140kB
[  280.327686] 491299 total pagecache pages
[  280.327686] Swap cache: add 495985, delete 4762, find 0/1
[  280.327686] Free swap  = 3963444kB
[  280.327686] Total swap = 5947384kB
[  280.335694] 515887 pages of RAM
[  280.335694] 8379 reserved pages
[  280.335694] 83 pages shared
[  280.335694] 491223 pages swap cached
[  280.335694] Out of memory: kill process 2842 (a.out) score 494723 or a child
[  280.335694] Killed process 2842 (a.out)
[  283.864479] SysRq : Manual OOM execution



And this is from 2.6.26 when xorg is running:

[  325.828622] [drm] Initialized i915 1.6.0 20060119 on minor 0
[  575.537309] hald invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0
[  575.537319] Pid: 3183, comm: hald Not tainted 2.6.26-2-amd64 #1
[  575.537323]
[  575.537324] Call Trace:
[  575.537362]  [<ffffffff8027388c>] oom_kill_process+0x57/0x1dc
[  575.537374]  [<ffffffff8023b49d>] __capable+0x9/0x1c
[  575.537383]  [<ffffffff80273bb7>] badness+0x188/0x1c7
[  575.537406]  [<ffffffff80273deb>] out_of_memory+0x1f5/0x28e
[  575.537444]  [<ffffffff80276b3c>] __alloc_pages_internal+0x31d/0x3bf
[  575.537494]  [<ffffffff802787f2>] __do_page_cache_readahead+0x79/0x183
[  575.537531]  [<ffffffff802730a1>] filemap_fault+0x15d/0x33c
[  575.537570]  [<ffffffff8027e620>] __do_fault+0x50/0x3e6
[  575.537623]  [<ffffffff80281983>] handle_mm_fault+0x3f4/0x867
[  575.537654]  [<ffffffff80248b56>] hrtimer_start+0x112/0x134
[  575.537664]  [<ffffffff80229020>] hrtick_start_fair+0xfb/0x144
[  575.537700]  [<ffffffff80221fbc>] do_page_fault+0x5d8/0x9c8
[  575.537715]  [<ffffffff8042916f>] thread_return+0x6b/0xac
[  575.537755]  [<ffffffff8042a869>] error_exit+0x0/0x60
[  575.537815]
[  575.537818] Mem-info:
[  575.537821] Node 0 DMA per-cpu:
[  575.537826] CPU    0: hi:    0, btch:   1 usd:   0
[  575.537830] CPU    1: hi:    0, btch:   1 usd:   0
[  575.537833] Node 0 DMA32 per-cpu:
[  575.537837] CPU    0: hi:  186, btch:  31 usd: 171
[  575.537841] CPU    1: hi:  186, btch:  31 usd: 134
[  575.537847] Active:227811 inactive:228679 dirty:0 writeback:0 unstable:0
[  575.537849]  free:3352 slab:4828 mapped:620 pagetables:7858 bounce:0
[  575.537853] Node 0 DMA free:7900kB min:28kB low:32kB high:40kB
active:1884kB inactive:1732kB present:10788kB pages_scanned:17985
all_unreclaimable? no
[  575.537861] lowmem_reserve[]: 0 1971 1971 1971
[  575.537868] Node 0 DMA32 free:5508kB min:5664kB low:7080kB
high:8496kB active:909360kB inactive:912984kB present:2019172kB
pages_scanned:4362542 all_unreclaimable? yes
[  575.537876] lowmem_reserve[]: 0 0 0 0
[  575.537883] Node 0 DMA: 15*4kB 14*8kB 9*16kB 3*32kB 5*64kB 2*128kB
3*256kB 2*512kB 1*1024kB 0*2048kB 1*4096kB = 7900kB
[  575.537902] Node 0 DMA32: 113*4kB 34*8kB 25*16kB 11*32kB 1*64kB
1*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 5508kB
[  575.537921] 697 total pagecache pages
[  575.537925] Swap cache: add 1990647, delete 1990647, find 16681/17672
[  575.537928] Free swap  = 0kB
[  575.537931] Total swap = 5947384kB
[  575.551393] 515887 pages of RAM
[  575.551400] 8380 reserved pages
[  575.551402] 40837 pages shared
[  575.551405] 0 pages swap cached
[  575.551410] Out of memory: kill process 3956 (a.out) score 1888511 or a child
[  575.551439] Killed process 3956 (a.out)
[  575.623722] hald invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0
[  575.623731] Pid: 3183, comm: hald Not tainted 2.6.26-2-amd64 #1
[  575.623735]
[  575.623736] Call Trace:
[  575.623772]  [<ffffffff8027388c>] oom_kill_process+0x57/0x1dc
[  575.623783]  [<ffffffff8023b49d>] __capable+0x9/0x1c
[  575.623791]  [<ffffffff80273bb7>] badness+0x188/0x1c7
[  575.623809]  [<ffffffff80273deb>] out_of_memory+0x1f5/0x28e
[  575.623846]  [<ffffffff80276b3c>] __alloc_pages_internal+0x31d/0x3bf
[  575.623894]  [<ffffffff802787f2>] __do_page_cache_readahead+0x79/0x183
[  575.623930]  [<ffffffff802730a1>] filemap_fault+0x15d/0x33c
[  575.623966]  [<ffffffff8027e620>] __do_fault+0x50/0x3e6
[  575.624017]  [<ffffffff80281983>] handle_mm_fault+0x3f4/0x867
[  575.624046]  [<ffffffff80248b56>] hrtimer_start+0x112/0x134
[  575.624055]  [<ffffffff80229020>] hrtick_start_fair+0xfb/0x144
[  575.624088]  [<ffffffff80221fbc>] do_page_fault+0x5d8/0x9c8
[  575.624101]  [<ffffffff8042916f>] thread_return+0x6b/0xac
[  575.624139]  [<ffffffff8042a869>] error_exit+0x0/0x60
[  575.624198]
[  575.624201] Mem-info:
[  575.624204] Node 0 DMA per-cpu:
[  575.624208] CPU    0: hi:    0, btch:   1 usd:   0
[  575.624212] CPU    1: hi:    0, btch:   1 usd:   0
[  575.624215] Node 0 DMA32 per-cpu:
[  575.624219] CPU    0: hi:  186, btch:  31 usd: 153
[  575.624222] CPU    1: hi:  186, btch:  31 usd: 134
[  575.624228] Active:228889 inactive:227627 dirty:0 writeback:310 unstable:0
[  575.624230]  free:3352 slab:4828 mapped:620 pagetables:7858 bounce:0
[  575.624234] Node 0 DMA free:7900kB min:28kB low:32kB high:40kB
active:1880kB inactive:1720kB present:10788kB pages_scanned:18498
all_unreclaimable? yes
[  575.624241] lowmem_reserve[]: 0 1971 1971 1971
[  575.624248] Node 0 DMA32 free:5508kB min:5664kB low:7080kB
high:8496kB active:913676kB inactive:908788kB present:2019172kB
pages_scanned:4479719 all_unreclaimable? yes
[  575.624256] lowmem_reserve[]: 0 0 0 0
[  575.624262] Node 0 DMA: 15*4kB 14*8kB 9*16kB 3*32kB 5*64kB 2*128kB
3*256kB 2*512kB 1*1024kB 0*2048kB 1*4096kB = 7900kB
[  575.624280] Node 0 DMA32: 113*4kB 34*8kB 25*16kB 11*32kB 1*64kB
1*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 5508kB
[  575.624297] 1007 total pagecache pages
[  575.624301] Swap cache: add 1990950, delete 1990647, find 16681/17673
[  575.624304] Free swap  = 6660kB
[  575.624306] Total swap = 5947384kB
[  575.636736] 515887 pages of RAM
[  575.636736] 8380 reserved pages
[  575.636736] 40742 pages shared
[  575.636736] 303 pages swap cached
[  575.636736] Out of memory: kill process 3500 (x-session-manag)
score 410763 or a child
[  575.636736] Killed process 3568 (seahorse-agent)

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[dave b]

On 29 July 2010 19:47, dave b <db.pub.mail@gmail.com> wrote:
> On 28 July 2010 17:14, dave b <db.pub.mail@gmail.com> wrote:
>> On 28 July 2010 15:06, KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
>>>> On 27 July 2010 21:14, KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
>>>> >> On 27 July 2010 18:09, dave b <db.pub.mail@gmail.com> wrote:
>>>> >> > On 27 July 2010 16:09, KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
>>>> >> >>> > Do you mean the issue will be gone if disabling intel graphics?
>>>> >> >>> It may be a general issue or it could just be specific :)
>>>> >> >
>>>> >> > I will try with the latest ubuntu and report how that goes (that will
>>>> >> > be using fairly new xorg etc.) it is likely to be hidden issue just
>>>> >> > with the intel graphics driver. However, my concern is that it isn't -
>>>> >> > and it is about how shared graphics memory is handled :)
>>>> >>
>>>> >>
>>>> >> Ok my desktop still stalled and no oom killer was invoked when I added
>>>> >> swap to a live-cd of 10.04 amd64.
>>>> >>
>>>> >> *Without* *swap* *on* - the oom killer was invoked - here is a copy of it.
>>>> >
>>>> > This stack seems similar following bug. can you please try to disable intel graphics
>>>> > driver?
>>>> >
>>>> > https://bugzilla.kernel.org/show_bug.cgi?id=14933
>>>>
>>>> Ok I am not sure how to do that :)
>>>> I could revert the patch and see if it 'fixes' this :)
>>>
>>> Oops, no, revert is not good action. the patch is correct.
>>> probably my explanation was not clear. sorry.
>>>
>>> I did hope to disable 'driver' (i.e. using vga), not disable the patch.
>>
>> Oh you mean in xorg, I will also blacklist the module. Sure that patch
>> might not it but in 2.6.26 the problem isn't there :)
>
> Ok I re-tested with 2.6.26 and 2.6.34.1
> So I will describe what happens below:
>
> 2.6.26 - with xorg running
> "Given I have a test file called a.out
>  And I can see Xorg
>  And I am using 2.6.26
>  And I have swap on
>  When I run it I run a.out
>  Then I see the system freeze up slightly
>  And the hard drive churns( and the cpu is doing something as the
> large fan kicks)
>  And after a while the system unfreezes"
>
> 2.6.26 - from single mode - before xorg starts and i915 is *not* loaded.
> "Given I have a test file called a.out
>  And I cannot see Xorg
>  And I am using 2.6.26
>  And I have swap on
>  When I run it I run a.out
>  Then I see the system freeze up
>  And the system fan doesn't spin any faster
>  And the system just sits idle"
>
> 2.6.34.1
> With and without xorg - WITH spam on the same behaviour as in the
> 2.6.26 kernel appears (when xorg is not loaded).
>
> OOM attached from the 2.6.26 kernel when I used magic keys to invoke
> the oom killer :) (this was on the 2.6.26 kernel - before i915 had
> loaded and in single mode).

s/spam/same/

Re: PROBLEM: oom killer and swap weirdness on 2.6.3* kernels

[dave b]

Ok this issue is still around and still *really* annoying.
So I had a 5mb text file, I put %s/\n/, in vim, my desktop stalls as
vim uses memory it sits there for ~10 minutes before finally the oom
killer wakes up and does something....
This is on totally different hardware now(amd phenom ddr3 ram, SATA 3
disk) and Here is some dmesg output :)


ep 21 22:41:44 RANDOMBOXEN kernel: [329160.956367] kjournald     D
ffff88011be59a00     0   982      2 0x00000000
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956370]  ffff88011bf9fbf0
0000000000000046 ffff88011bf9fbc0 ffffffffa00f0775
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956373]  ffff88011bf9ffd8
0000000000013900 ffff88011bf9ffd8 ffff88011be59680
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956375]  0000000000013900
0000000000013900 0000000000013900 0000000000013900
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956377] Call Trace:
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956399]
[<ffffffffa00f0775>] ? dm_table_unplug_all+0x54/0xc6 [dm_mod]
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956405]
[<ffffffff812e4f80>] io_schedule+0x7b/0xc1
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956408]
[<ffffffff8110d0ea>] sync_buffer+0x3b/0x3f
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956409]
[<ffffffff812e5488>] __wait_on_bit+0x47/0x79
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956411]
[<ffffffff8110d0af>] ? sync_buffer+0x0/0x3f
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956413]
[<ffffffff8110d0af>] ? sync_buffer+0x0/0x3f
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956415]
[<ffffffff812e5524>] out_of_line_wait_on_bit+0x6a/0x77
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956418]
[<ffffffff8105b678>] ? wake_bit_function+0x0/0x2a
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956419]
[<ffffffff8110d06f>] __wait_on_buffer+0x1f/0x21
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956425]
[<ffffffffa0165824>] journal_commit_transaction+0xa42/0xfba [jbd]
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956427]
[<ffffffff812e4e36>] ? schedule+0x64d/0x71c
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956430]
[<ffffffff8104fe0a>] ? lock_timer_base+0x26/0x4a
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956432]
[<ffffffff8104fee4>] ? try_to_del_timer_sync+0xb6/0xc3
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956437]
[<ffffffffa0168942>] kjournald+0xef/0x23b [jbd]
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956439]
[<ffffffff8105b640>] ? autoremove_wake_function+0x0/0x38
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956444]
[<ffffffffa0168853>] ? kjournald+0x0/0x23b [jbd]
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956446]
[<ffffffff8105b1ae>] kthread+0x7d/0x85
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956449]
[<ffffffff81003964>] kernel_thread_helper+0x4/0x10
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956451]
[<ffffffff8105b131>] ? kthread+0x0/0x85
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956453]
[<ffffffff81003960>] ? kernel_thread_helper+0x0/0x10
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956461] openvpn       D
ffff88011beb3080     0  2196      1 0x00000000
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956463]  ffff88011bc51cb8
0000000000000086 0000000000000282 00000000f9897afa
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956469]  ffff88011bc51fd8
0000000000013900 ffff88011bc51fd8 ffff88011beb2d00
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956471]  0000000000013900
0000000000013900 0000000000013900 0000000000013900
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956473] Call Trace:
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956475]
[<ffffffff812e4f80>] io_schedule+0x7b/0xc1
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956477]
[<ffffffff810b0e18>] sync_page+0x41/0x45
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956478]
[<ffffffff812e5383>] __wait_on_bit_lock+0x45/0x8c
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956480]
[<ffffffff810b0dd7>] ? sync_page+0x0/0x45
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956481]
[<ffffffff810b0dc3>] __lock_page+0x63/0x6a
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956483]
[<ffffffff8105b678>] ? wake_bit_function+0x0/0x2a
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956486]
[<ffffffff810c857d>] handle_mm_fault+0x61f/0x93e
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956491]
[<ffffffffa017b06e>] ? ext3_dirty_inode+0x7b/0x83 [ext3]
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956494]
[<ffffffff81028571>] do_page_fault+0x2f6/0x31a
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956496]
[<ffffffff810ff51a>] ? notify_change+0x298/0x2aa
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956498]
[<ffffffff810ead20>] ? do_truncate+0x7b/0x86
Sep 21 22:41:44 RANDOMBOXEN kernel: [329160.956500]
[<ffffffff812e6fc5>] page_fault+0x25/0x30
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956504] postgres      D
ffff88011ced9a00     0  2241   2237 0x00000000
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956506]  ffff88011bc4fb78
0000000000000086 ffff88011bc4fc38 000000003b0edb74
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956508]  ffff88011bc4ffd8
0000000000013900 ffff88011bc4ffd8 ffff88011ced9680
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956510]  0000000000013900
0000000000013900 0000000000013900 0000000000013900
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956511] Call Trace:
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956513]
[<ffffffff812e4f80>] io_schedule+0x7b/0xc1
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956515]
[<ffffffff810b0e18>] sync_page+0x41/0x45
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956516]
[<ffffffff812e5383>] __wait_on_bit_lock+0x45/0x8c
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956518]
[<ffffffff810b0dd7>] ? sync_page+0x0/0x45
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956519]
[<ffffffff810b0dc3>] __lock_page+0x63/0x6a
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956521]
[<ffffffff8105b678>] ? wake_bit_function+0x0/0x2a
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956523]
[<ffffffff810b0ef2>] find_lock_page+0x39/0x5d
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956524]
[<ffffffff810b159e>] filemap_fault+0x1cc/0x31f
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956526]
[<ffffffff810c5e40>] __do_fault+0x50/0x413
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956528]
[<ffffffff810c8433>] handle_mm_fault+0x4d5/0x93e
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956530]
[<ffffffff81028571>] do_page_fault+0x2f6/0x31a
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956531]
[<ffffffff810cc37c>] ? unmap_region+0x125/0x13b
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956533]
[<ffffffff810cc107>] ? remove_vma+0x64/0x6c
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956534]
[<ffffffff810cd398>] ? do_munmap+0x31a/0x33e
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956536]
[<ffffffff812e6fc5>] page_fault+0x25/0x30
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956541] Xorg          D
ffff88011d6c4700     0  2472   2470 0x00400004
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956543]  ffff88011cc75b78
0000000000003086 ffff88011cc75c38 00000000047655ff
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956545]  ffff88011cc75fd8
0000000000013900 ffff88011cc75fd8 ffff88011d6c4380
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956546]  0000000000013900
0000000000013900 0000000000013900 0000000000013900
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956548] Call Trace:
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956550]
[<ffffffff812e4f80>] io_schedule+0x7b/0xc1
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956552]
[<ffffffff810b0e18>] sync_page+0x41/0x45
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956553]
[<ffffffff812e5383>] __wait_on_bit_lock+0x45/0x8c
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956554]
[<ffffffff810b0dd7>] ? sync_page+0x0/0x45
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956556]
[<ffffffff810b0dc3>] __lock_page+0x63/0x6a
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956558]
[<ffffffff8105b678>] ? wake_bit_function+0x0/0x2a
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956559]
[<ffffffff810b0ef2>] find_lock_page+0x39/0x5d
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956561]
[<ffffffff810b159e>] filemap_fault+0x1cc/0x31f
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956563]
[<ffffffff810c5e40>] __do_fault+0x50/0x413
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956565]
[<ffffffff810c8433>] handle_mm_fault+0x4d5/0x93e
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956567]
[<ffffffff8105478c>] ? get_signal_to_deliver+0x123/0x3ab
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956569]
[<ffffffff81028571>] do_page_fault+0x2f6/0x31a
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956571]
[<ffffffff8100ba65>] ? restore_i387_xstate+0x6e/0x168
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956573]
[<ffffffff81154dbe>] ? security_file_permission+0x11/0x13
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956575]
[<ffffffff810028ec>] ? sys_rt_sigreturn+0x1e8/0x23c
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956577]
[<ffffffff812e6fc5>] page_fault+0x25/0x30
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956581] apache2       D
ffff88011bc04700     0  2547      1 0x00000000
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956583]  ffff88011d7cdb78
0000000000000082 ffff88011d7cdc38 00000000e2f0079f
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956584]  ffff88011d7cdfd8
0000000000013900 ffff88011d7cdfd8 ffff88011bc04380
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956586]  0000000000013900
0000000000013900 0000000000013900 0000000000013900
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956588] Call Trace:
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956590]
[<ffffffff812e4f80>] io_schedule+0x7b/0xc1
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956591]
[<ffffffff810b0e18>] sync_page+0x41/0x45
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956593]
[<ffffffff812e5383>] __wait_on_bit_lock+0x45/0x8c
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956594]
[<ffffffff810b0dd7>] ? sync_page+0x0/0x45
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956596]
[<ffffffff810b0dc3>] __lock_page+0x63/0x6a
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956597]
[<ffffffff8105b678>] ? wake_bit_function+0x0/0x2a
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956599]
[<ffffffff810b0ef2>] find_lock_page+0x39/0x5d
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956601]
[<ffffffff810b159e>] filemap_fault+0x1cc/0x31f
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956603]
[<ffffffff810c5e40>] __do_fault+0x50/0x413
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956604]
[<ffffffff812e53bc>] ? __wait_on_bit_lock+0x7e/0x8c
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956606]
[<ffffffff810c8433>] handle_mm_fault+0x4d5/0x93e
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956608]
[<ffffffff8100350e>] ? apic_timer_interrupt+0xe/0x20
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956609]
[<ffffffff81028571>] do_page_fault+0x2f6/0x31a
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956611]
[<ffffffff810f9504>] ? poll_select_copy_remaining+0xc5/0xe9
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956613]
[<ffffffff810fa820>] ? sys_select+0xa7/0xbc
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956615]
[<ffffffff812e6fc5>] page_fault+0x25/0x30
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956619] gconfd-2      D
ffff8801150c4700     0  2622      1 0x00000000
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956621]  ffff8801151b1cb8
0000000000000086 0000000000000282 00000000f8220c09
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956623]  ffff8801151b1fd8
0000000000013900 ffff8801151b1fd8 ffff8801150c4380
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956625]  0000000000013900
0000000000013900 0000000000013900 0000000000013900
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956626] Call Trace:
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956628]
[<ffffffff812e4f80>] io_schedule+0x7b/0xc1
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956630]
[<ffffffff810b0e18>] sync_page+0x41/0x45
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956631]
[<ffffffff812e5383>] __wait_on_bit_lock+0x45/0x8c
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956633]
[<ffffffff810b0dd7>] ? sync_page+0x0/0x45
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956634]
[<ffffffff810b0dc3>] __lock_page+0x63/0x6a
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956636]
[<ffffffff8105b678>] ? wake_bit_function+0x0/0x2a
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956638]
[<ffffffff810c857d>] handle_mm_fault+0x61f/0x93e
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956639]
[<ffffffff81028571>] do_page_fault+0x2f6/0x31a
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956641]
[<ffffffff812e6fc5>] page_fault+0x25/0x30
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956646] gnome-panel   D
ffff88011beb1a00     0  2741   2625 0x00000000
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956648]  ffff880111fa1cb8
0000000000000086 0000000000000282 0000000038a28654
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956650]  ffff880111fa1fd8
0000000000013900 ffff880111fa1fd8 ffff88011beb1680
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956652]  0000000000013900
0000000000013900 0000000000013900 0000000000013900
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956653] Call Trace:
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956655]
[<ffffffff812e4f80>] io_schedule+0x7b/0xc1
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956657]
[<ffffffff810b0e18>] sync_page+0x41/0x45
Sep 21 22:41:45 RANDOMBOXEN kernel: [329160.956658]
[<ffffffff812e5383>] __wait_on_bit_lock+0x45/0x8c
Sep 21 22:41:46 RANDOMBOXEN kernel: [329160.956660]
[<ffffffff810b0dd7>] ? sync_page+0x0/0x45
Sep 21 22:41:46 RANDOMBOXEN kernel: [329160.956661]
[<ffffffff810b0dc3>] __lock_page+0x63/0x6a
Sep 21 22:41:46 RANDOMBOXEN kernel: [329160.956663]
[<ffffffff8105b678>] ? wake_bit_function+0x0/0x2a
Sep 21 22:41:46 RANDOMBOXEN kernel: [329160.956665]
[<ffffffff810c857d>] handle_mm_fault+0x61f/0x93e
Sep 21 22:41:46 RANDOMBOXEN kernel: [329160.956666]
[<ffffffff810f578d>] ? putname+0x30/0x39
Sep 21 22:41:46 RANDOMBOXEN kernel: [329160.956668]
[<ffffffff810f7156>] ? user_path_at+0x5d/0x8c
Sep 21 22:41:46 RANDOMBOXEN kernel: [329160.956669]
[<ffffffff81028571>] do_page_fault+0x2f6/0x31a
Sep 21 22:41:46 RANDOMBOXEN kernel: [329160.956671]
[<ffffffff810ef0b0>] ? sys_newstat+0x2c/0x3b
Sep 21 22:41:46 RANDOMBOXEN kernel: [329160.956673]
[<ffffffff812e6fc5>] page_fault+0x25/0x30
Sep 21 22:41:47 RANDOMBOXEN kernel: [329160.956677] nautilus      D
ffff880111f31a00     0  2743   2625 0x00000000
Sep 21 22:41:47 RANDOMBOXEN kernel: [329160.956679]  ffff880111e69cb8
0000000000000086 0000000000000282 00000000d6f5b63a
Sep 21 22:41:47 RANDOMBOXEN kernel: [329160.956680]  ffff880111e69fd8
0000000000013900 ffff880111e69fd8 ffff880111f31680
Sep 21 22:41:47 RANDOMBOXEN kernel: [329160.956682]  0000000000013900
0000000000013900 0000000000013900 0000000000013900
Sep 21 22:41:47 RANDOMBOXEN kernel: [329160.956684] Call Trace:
Sep 21 22:41:47 RANDOMBOXEN kernel: [329160.956686]
[<ffffffff812e4f80>] io_schedule+0x7b/0xc1
Sep 21 22:41:47 RANDOMBOXEN kernel: [329160.956687]
[<ffffffff810b0e18>] sync_page+0x41/0x45
Sep 21 22:41:47 RANDOMBOXEN kernel: [329160.956689]
[<ffffffff812e5383>] __wait_on_bit_lock+0x45/0x8c
Sep 21 22:41:47 RANDOMBOXEN kernel: [329160.956690]
[<ffffffff810b0dd7>] ? sync_page+0x0/0x45
Sep 21 22:41:47 RANDOMBOXEN kernel: [329160.956691]
[<ffffffff810b0dc3>] __lock_page+0x63/0x6a
Sep 21 22:41:47 RANDOMBOXEN kernel: [329160.956693]
[<ffffffff8105b678>] ? wake_bit_function+0x0/0x2a
Sep 21 22:41:47 RANDOMBOXEN kernel: [329160.956695]
[<ffffffff810c857d>] handle_mm_fault+0x61f/0x93e
Sep 21 22:41:47 RANDOMBOXEN kernel: [329160.956697]
[<ffffffff81028571>] do_page_fault+0x2f6/0x31a
Sep 21 22:41:47 RANDOMBOXEN kernel: [329160.956699]
[<ffffffff81009861>] ? read_tsc+0xe/0x25
Sep 21 22:41:47 RANDOMBOXEN kernel: [329160.956701]
[<ffffffff8106342d>] ? ktime_get_ts+0xb1/0xbe
Sep 21 22:41:54 RANDOMBOXEN kernel: [329160.956703]
[<ffffffff810f9662>] ? poll_select_set_timeout+0x5c/0x77
Sep 21 22:41:54 RANDOMBOXEN kernel: [329160.956704]
[<ffffffff812e6fc5>] page_fault+0x25/0x30
Sep 21 22:41:54 RANDOMBOXEN kernel: [329160.956708] update-notifi D
ffff88011089b080     0  2752   2625 0x00000000
Sep 21 22:41:59 RANDOMBOXEN kernel: [329160.956710]  ffff88011090dcb8
0000000000000086 0000000000000282 00000000c088e457
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956712]  ffff88011090dfd8
0000000000013900 ffff88011090dfd8 ffff88011089ad00
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956714]  0000000000013900
0000000000013900 0000000000013900 0000000000013900
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956715] Call Trace:
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956717]
[<ffffffff812e4f80>] io_schedule+0x7b/0xc1
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956719]
[<ffffffff810b0e18>] sync_page+0x41/0x45
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956720]
[<ffffffff812e5383>] __wait_on_bit_lock+0x45/0x8c
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956722]
[<ffffffff810b0dd7>] ? sync_page+0x0/0x45
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956723]
[<ffffffff810b0dc3>] __lock_page+0x63/0x6a
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956725]
[<ffffffff8105b678>] ? wake_bit_function+0x0/0x2a
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956727]
[<ffffffff810c857d>] handle_mm_fault+0x61f/0x93e
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956729]
[<ffffffff81028571>] do_page_fault+0x2f6/0x31a
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956730]
[<ffffffff81009861>] ? read_tsc+0xe/0x25
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956732]
[<ffffffff8106342d>] ? ktime_get_ts+0xb1/0xbe
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956734]
[<ffffffff810f9662>] ? poll_select_set_timeout+0x5c/0x77
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956735]
[<ffffffff812e6fc5>] page_fault+0x25/0x30
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956739] nm-applet     D
ffff88011089dd80     0  2762   2625 0x00000000
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956741]  ffff880110a27cb8
0000000000000082 0000000000000282 0000000008ebc327
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956743]  ffff880110a27fd8
0000000000013900 ffff880110a27fd8 ffff88011089da00
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956745]  0000000000013900
0000000000013900 0000000000013900 0000000000013900
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956747] Call Trace:
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956749]
[<ffffffff812e4f80>] io_schedule+0x7b/0xc1
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956750]
[<ffffffff810b0e18>] sync_page+0x41/0x45
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956751]
[<ffffffff812e5383>] __wait_on_bit_lock+0x45/0x8c
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956753]
[<ffffffff810b0dd7>] ? sync_page+0x0/0x45
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956754]
[<ffffffff810b0dc3>] __lock_page+0x63/0x6a
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956756]
[<ffffffff8105b678>] ? wake_bit_function+0x0/0x2a
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956758]
[<ffffffff810c857d>] handle_mm_fault+0x61f/0x93e
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956760]
[<ffffffff81028571>] do_page_fault+0x2f6/0x31a
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956761]
[<ffffffff81009861>] ? read_tsc+0xe/0x25
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956763]
[<ffffffff8100350e>] ? apic_timer_interrupt+0xe/0x20
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956765]
[<ffffffff8106342d>] ? ktime_get_ts+0xb1/0xbe
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956767]
[<ffffffff810f9662>] ? poll_select_set_timeout+0x5c/0x77
Sep 21 22:42:16 RANDOMBOXEN kernel: [329160.956768]
[<ffffffff812e6fc5>] page_fault+0x25/0x30
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481246] gnome-volume-ma
invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481250] gnome-volume-ma
cpuset=/ mems_allowed=0
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481253] Pid: 2769, comm:
gnome-volume-ma Not tainted 2.6.35.4 #1
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481255] Call Trace:
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481262]
[<ffffffff810824f1>] ? cpuset_print_task_mems_allowed+0x8d/0x98
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481265]
[<ffffffff810b2e89>] dump_header+0x65/0x182
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481269]
[<ffffffff8119031b>] ? ___ratelimit+0xc7/0xe4
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481271]
[<ffffffff810b2feb>] oom_kill_process+0x45/0x110
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481273]
[<ffffffff810b3439>] __out_of_memory+0x143/0x15a
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481275]
[<ffffffff810b35a1>] out_of_memory+0x151/0x183
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481278]
[<ffffffff810b726b>] __alloc_pages_nodemask+0x550/0x6a1
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481281]
[<ffffffff810de22b>] alloc_pages_current+0xa8/0xd1
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481283]
[<ffffffff810b108a>] __page_cache_alloc+0x77/0x82
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481286]
[<ffffffff810b88ce>] __do_page_cache_readahead+0x96/0x1a2
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481288]
[<ffffffff810b89f6>] ra_submit+0x1c/0x20
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481290]
[<ffffffff810b1582>] filemap_fault+0x1b0/0x31f
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481293]
[<ffffffff810c5e40>] __do_fault+0x50/0x413
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481296]
[<ffffffff812e53bc>] ? __wait_on_bit_lock+0x7e/0x8c
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481298]
[<ffffffff810b0dd7>] ? sync_page+0x0/0x45
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481301]
[<ffffffff810c8433>] handle_mm_fault+0x4d5/0x93e
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481304]
[<ffffffff81028571>] do_page_fault+0x2f6/0x31a
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481307]
[<ffffffff81009861>] ? read_tsc+0xe/0x25
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481310]
[<ffffffff8106342d>] ? ktime_get_ts+0xb1/0xbe
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481313]
[<ffffffff810f9662>] ? poll_select_set_timeout+0x5c/0x77
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481316]
[<ffffffff812e6fc5>] page_fault+0x25/0x30
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481318] Mem-Info:
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481319] Node 0 DMA per-cpu:
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481321] CPU    0: hi:
0, btch:   1 usd:   0
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481323] CPU    1: hi:
0, btch:   1 usd:   0
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481324] CPU    2: hi:
0, btch:   1 usd:   0
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481325] CPU    3: hi:
0, btch:   1 usd:   0
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481327] CPU    4: hi:
0, btch:   1 usd:   0
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481328] CPU    5: hi:
0, btch:   1 usd:   0
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481329] Node 0 DMA32 per-cpu:
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481331] CPU    0: hi:
186, btch:  31 usd: 179
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481332] CPU    1: hi:
186, btch:  31 usd: 183
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481333] CPU    2: hi:
186, btch:  31 usd: 196
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481335] CPU    3: hi:
186, btch:  31 usd: 202
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481336] CPU    4: hi:
186, btch:  31 usd: 176
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481337] CPU    5: hi:
186, btch:  31 usd: 166
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481338] Node 0 Normal per-cpu:
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481340] CPU    0: hi:
186, btch:  31 usd: 160
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481341] CPU    1: hi:
186, btch:  31 usd: 175
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481342] CPU    2: hi:
186, btch:  31 usd: 156
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481343] CPU    3: hi:
186, btch:  31 usd: 158
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481345] CPU    4: hi:
186, btch:  31 usd: 146
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481346] CPU    5: hi:
186, btch:  31 usd: 186
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481349] active_anon:668154
inactive_anon:166010 isolated_anon:544
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481350]  active_file:172
inactive_file:213 isolated_file:0
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481351]  unevictable:1349
dirty:0 writeback:166443 unstable:0
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481352]  free:5834
slab_reclaimable:5297 slab_unreclaimable:44051
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481352]  mapped:762
shmem:22 pagetables:13167 bounce:0
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481354] Node 0 DMA
free:15004kB min:28kB low:32kB high:40kB active_anon:48kB
inactive_anon:256kB active_file:0kB inactive_file:0kB unevictable:0kB
isolated(anon):0kB isolated(file):0kB present:15688kB mlocked:0kB
dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB
slab_unreclaimable:560kB kernel_stack:0kB pagetables:0kB unstable:0kB
bounce:0kB writeback_tmp:0kB pages_scanned:576 all_unreclaimable? yes
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481361] lowmem_reserve[]:
0 3254 3759 3759
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481363] Node 0 DMA32
free:7412kB min:6780kB low:8472kB high:10168kB active_anon:2519256kB
inactive_anon:507512kB active_file:0kB inactive_file:52kB
unevictable:2912kB isolated(anon):2176kB isolated(file):0kB
present:3332768kB mlocked:2912kB dirty:0kB writeback:509772kB
mapped:96kB shmem:68kB slab_reclaimable:13044kB
slab_unreclaimable:99424kB kernel_stack:1392kB pagetables:32212kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4732464
all_unreclaimable? yes
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481371] lowmem_reserve[]:
0 0 505 505
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481372] Node 0 Normal
free:920kB min:1052kB low:1312kB high:1576kB active_anon:153312kB
inactive_anon:156272kB active_file:688kB inactive_file:800kB
unevictable:2484kB isolated(anon):0kB isolated(file):0kB
present:517120kB mlocked:2484kB dirty:0kB writeback:156000kB
mapped:2952kB shmem:20kB slab_reclaimable:8144kB
slab_unreclaimable:76220kB kernel_stack:1576kB pagetables:20456kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:555904
all_unreclaimable? yes
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481379] lowmem_reserve[]: 0 0 0 0
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481381] Node 0 DMA: 15*4kB
4*8kB 2*16kB 5*32kB 2*64kB 2*128kB 2*256kB 1*512kB 1*1024kB 2*2048kB
2*4096kB = 15004kB
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481386] Node 0 DMA32:
1041*4kB 2*8kB 8*16kB 11*32kB 5*64kB 5*128kB 3*256kB 0*512kB 1*1024kB
0*2048kB 0*4096kB = 7412kB
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481391] Node 0 Normal:
230*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB
0*2048kB 0*4096kB = 920kB
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481396] 168181 total pagecache pages
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481397] 167149 pages in swap cache
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481398] Swap cache stats:
add 2171614, delete 2004465, find 789342/867065
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481400] Free swap  = 456344kB
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.481400] Total swap = 5947388kB
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.495910] 983024 pages RAM
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.495911] 32468 pages reserved
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.495912] 29764 pages shared
Sep 21 22:42:16 RANDOMBOXEN kernel: [329348.495912] 939943 pages non-shared
Sep 21 22:42:39 RANDOMBOXEN kernel: [329431.629246] vim invoked
oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629246] vim cpuset=/ mems_allowed=0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629246] Pid: 27080, comm:
vim Not tainted 2.6.35.4 #1
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629246] Call Trace:
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629246]
[<ffffffff810824f1>] ? cpuset_print_task_mems_allowed+0x8d/0x98
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629246]
[<ffffffff810b2e89>] dump_header+0x65/0x182
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629246]
[<ffffffff8119031b>] ? ___ratelimit+0xc7/0xe4
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629246]
[<ffffffff810b2feb>] oom_kill_process+0x45/0x110
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629247]
[<ffffffff810b3439>] __out_of_memory+0x143/0x15a
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629249]
[<ffffffff810b35a1>] out_of_memory+0x151/0x183
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629251]
[<ffffffff810b726b>] __alloc_pages_nodemask+0x550/0x6a1
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629254]
[<ffffffff810de369>] alloc_page_vma+0x115/0x134
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629257]
[<ffffffff810c82cb>] handle_mm_fault+0x36d/0x93e
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629259]
[<ffffffff81028571>] do_page_fault+0x2f6/0x31a
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629261]
[<ffffffff810cdbb0>] ? do_brk+0x22d/0x320
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629264]
[<ffffffff812e6fc5>] page_fault+0x25/0x30
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629265] Mem-Info:
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629266] Node 0 DMA per-cpu:
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629268] CPU    0: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629269] CPU    1: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629270] CPU    2: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629272] CPU    3: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629273] CPU    4: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629274] CPU    5: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629275] Node 0 DMA32 per-cpu:
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629276] CPU    0: hi:
186, btch:  31 usd:  34
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629277] CPU    1: hi:
186, btch:  31 usd: 149
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629278] CPU    2: hi:
186, btch:  31 usd:  72
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629279] CPU    3: hi:
186, btch:  31 usd:  49
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629280] CPU    4: hi:
186, btch:  31 usd:  38
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629281] CPU    5: hi:
186, btch:  31 usd: 158
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629282] Node 0 Normal per-cpu:
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629283] CPU    0: hi:
186, btch:  31 usd:  82
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629284] CPU    1: hi:
186, btch:  31 usd: 179
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629285] CPU    2: hi:
186, btch:  31 usd:  98
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629287] CPU    3: hi:
186, btch:  31 usd:  33
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629288] CPU    4: hi:
186, btch:  31 usd: 104
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629289] CPU    5: hi:
186, btch:  31 usd: 138
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629292] active_anon:694864
inactive_anon:174958 isolated_anon:0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629292]  active_file:317
inactive_file:391 isolated_file:0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629293]  unevictable:1350
dirty:10 writeback:386 unstable:0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629293]  free:6107
slab_reclaimable:4729 slab_unreclaimable:9023
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629294]  mapped:921
shmem:22 pagetables:13597 bounce:0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629295] Node 0 DMA
free:15040kB min:28kB low:32kB high:40kB active_anon:336kB
inactive_anon:416kB active_file:0kB inactive_file:0kB unevictable:0kB
isolated(anon):0kB isolated(file):0kB present:15688kB mlocked:0kB
dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB
slab_unreclaimable:72kB kernel_stack:0kB pagetables:4kB unstable:0kB
bounce:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? yes
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629301] lowmem_reserve[]:
0 3254 3759 3759
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629303] Node 0 DMA32
free:8652kB min:6780kB low:8472kB high:10168kB active_anon:2594112kB
inactive_anon:514336kB active_file:4kB inactive_file:124kB
unevictable:2916kB isolated(anon):0kB isolated(file):0kB
present:3332768kB mlocked:2916kB dirty:8kB writeback:1540kB
mapped:16kB shmem:64kB slab_reclaimable:11076kB
slab_unreclaimable:20900kB kernel_stack:1160kB pagetables:33188kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:192
all_unreclaimable? yes
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629309] lowmem_reserve[]:
0 0 505 505
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629311] Node 0 Normal
free:736kB min:1052kB low:1312kB high:1576kB active_anon:185008kB
inactive_anon:185080kB active_file:1264kB inactive_file:1440kB
unevictable:2484kB isolated(anon):0kB isolated(file):0kB
present:517120kB mlocked:2484kB dirty:32kB writeback:4kB mapped:3668kB
shmem:24kB slab_reclaimable:7840kB slab_unreclaimable:15120kB
kernel_stack:1592kB pagetables:21196kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:4320 all_unreclaimable? yes
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629317] lowmem_reserve[]: 0 0 0 0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629318] Node 0 DMA: 2*4kB
3*8kB 2*16kB 2*32kB 3*64kB 1*128kB 1*256kB 0*512kB 2*1024kB 2*2048kB
2*4096kB = 15040kB
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629323] Node 0 DMA32:
1328*4kB 2*8kB 10*16kB 10*32kB 6*64kB 5*128kB 3*256kB 0*512kB 1*1024kB
0*2048kB 0*4096kB = 8624kB
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629327] Node 0 Normal:
186*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB
0*2048kB 0*4096kB = 744kB
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629331] 34872 total pagecache pages
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629332] 33392 pages in swap cache
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629333] Swap cache stats:
add 2310505, delete 2277113, find 790781/869652
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629334] Free swap  = 0kB
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.629334] Total swap = 5947388kB
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.643262] 983024 pages RAM
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.643263] 32468 pages reserved
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.643264] 28528 pages shared
Sep 21 22:42:40 RANDOMBOXEN kernel: [329431.643265] 941544 pages non-shared
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440409] vim invoked
oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440413] vim cpuset=/ mems_allowed=0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440415] Pid: 27080, comm:
vim Not tainted 2.6.35.4 #1
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440416] Call Trace:
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440422]
[<ffffffff810824f1>] ? cpuset_print_task_mems_allowed+0x8d/0x98
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440425]
[<ffffffff810b2e89>] dump_header+0x65/0x182
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440429]
[<ffffffff8119031b>] ? ___ratelimit+0xc7/0xe4
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440431]
[<ffffffff810b2feb>] oom_kill_process+0x45/0x110
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440433]
[<ffffffff810b3439>] __out_of_memory+0x143/0x15a
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440435]
[<ffffffff810b35a1>] out_of_memory+0x151/0x183
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440438]
[<ffffffff810b726b>] __alloc_pages_nodemask+0x550/0x6a1
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440440]
[<ffffffff810de369>] alloc_page_vma+0x115/0x134
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440443]
[<ffffffff810c82cb>] handle_mm_fault+0x36d/0x93e
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440447]
[<ffffffff8100358e>] ? invalidate_interrupt2+0xe/0x20
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440449]
[<ffffffff81028571>] do_page_fault+0x2f6/0x31a
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440451]
[<ffffffff810cdbb0>] ? do_brk+0x22d/0x320
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440455]
[<ffffffff812e6fc5>] page_fault+0x25/0x30
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440456] Mem-Info:
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440457] Node 0 DMA per-cpu:
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440459] CPU    0: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440460] CPU    1: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440462] CPU    2: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440463] CPU    3: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440464] CPU    4: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440465] CPU    5: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440466] Node 0 DMA32 per-cpu:
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440468] CPU    0: hi:
186, btch:  31 usd:  83
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440469] CPU    1: hi:
186, btch:  31 usd: 174
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440471] CPU    2: hi:
186, btch:  31 usd: 163
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440472] CPU    3: hi:
186, btch:  31 usd:  39
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440473] CPU    4: hi:
186, btch:  31 usd:  53
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440474] CPU    5: hi:
186, btch:  31 usd: 178
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440475] Node 0 Normal per-cpu:
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440477] CPU    0: hi:
186, btch:  31 usd:  47
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440478] CPU    1: hi:
186, btch:  31 usd:  58
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440479] CPU    2: hi:
186, btch:  31 usd: 137
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440480] CPU    3: hi:
186, btch:  31 usd:  42
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440482] CPU    4: hi:
186, btch:  31 usd: 122
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440483] CPU    5: hi:
186, btch:  31 usd: 160
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440486] active_anon:693654
inactive_anon:175786 isolated_anon:0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440487]  active_file:199
inactive_file:716 isolated_file:0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440488]  unevictable:1350
dirty:0 writeback:439 unstable:0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440488]  free:6144
slab_reclaimable:4669 slab_unreclaimable:9109
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440489]  mapped:795
shmem:24 pagetables:13463 bounce:0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440491] Node 0 DMA
free:15040kB min:28kB low:32kB high:40kB active_anon:336kB
inactive_anon:416kB active_file:0kB inactive_file:0kB unevictable:0kB
isolated(anon):0kB isolated(file):0kB present:15688kB mlocked:0kB
dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB
slab_unreclaimable:72kB kernel_stack:0kB pagetables:4kB unstable:0kB
bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440497] lowmem_reserve[]:
0 3254 3759 3759
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440500] Node 0 DMA32
free:8580kB min:6780kB low:8472kB high:10168kB active_anon:2589352kB
inactive_anon:517648kB active_file:100kB inactive_file:1592kB
unevictable:2916kB isolated(anon):0kB isolated(file):0kB
present:3332768kB mlocked:2916kB dirty:0kB writeback:1756kB
mapped:120kB shmem:68kB slab_reclaimable:10980kB
slab_unreclaimable:21156kB kernel_stack:1160kB pagetables:32592kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2784
all_unreclaimable? yes
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440507] lowmem_reserve[]:
0 0 505 505
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440509] Node 0 Normal
free:956kB min:1052kB low:1312kB high:1576kB active_anon:184928kB
inactive_anon:185080kB active_file:696kB inactive_file:1272kB
unevictable:2484kB isolated(anon):0kB isolated(file):0kB
present:517120kB mlocked:2484kB dirty:0kB writeback:0kB mapped:3060kB
shmem:28kB slab_reclaimable:7696kB slab_unreclaimable:15208kB
kernel_stack:1592kB pagetables:21256kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:3040 all_unreclaimable? yes
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440516] lowmem_reserve[]: 0 0 0 0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440517] Node 0 DMA: 2*4kB
3*8kB 2*16kB 2*32kB 3*64kB 1*128kB 1*256kB 0*512kB 2*1024kB 2*2048kB
2*4096kB = 15040kB
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440522] Node 0 DMA32:
1208*4kB 68*8kB 14*16kB 10*32kB 6*64kB 5*128kB 3*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 8736kB
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440527] Node 0 Normal:
247*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB
0*2048kB 0*4096kB = 988kB
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440532] 43587 total pagecache pages
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440533] 41932 pages in swap cache
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440534] Swap cache stats:
add 2321150, delete 2279218, find 790795/869679
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440535] Free swap  = 0kB
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.440536] Total swap = 5947388kB
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.454068] 983024 pages RAM
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.454069] 32468 pages reserved
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.454070] 27996 pages shared
Sep 21 22:42:40 RANDOMBOXEN kernel: [329443.454071] 941542 pages non-shared
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692683] vim invoked
oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692693] vim cpuset=/ mems_allowed=0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692700] Pid: 27080, comm:
vim Not tainted 2.6.35.4 #1
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692705] Call Trace:
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692718]
[<ffffffff810824f1>] ? cpuset_print_task_mems_allowed+0x8d/0x98
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692729]
[<ffffffff810b2e89>] dump_header+0x65/0x182
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692738]
[<ffffffff8119031b>] ? ___ratelimit+0xc7/0xe4
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692745]
[<ffffffff810b2feb>] oom_kill_process+0x45/0x110
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692753]
[<ffffffff810b3439>] __out_of_memory+0x143/0x15a
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692760]
[<ffffffff810b35a1>] out_of_memory+0x151/0x183
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692768]
[<ffffffff810b726b>] __alloc_pages_nodemask+0x550/0x6a1
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692777]
[<ffffffff810de369>] alloc_page_vma+0x115/0x134
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692785]
[<ffffffff810c82cb>] handle_mm_fault+0x36d/0x93e
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692794]
[<ffffffff8100354e>] ? invalidate_interrupt0+0xe/0x20
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692802]
[<ffffffff8105ed39>] ? down_read_trylock+0x15/0x20
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692810]
[<ffffffff81028571>] do_page_fault+0x2f6/0x31a
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692817]
[<ffffffff810cdbb0>] ? do_brk+0x22d/0x320
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692825]
[<ffffffff812e6fc5>] page_fault+0x25/0x30
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692830] Mem-Info:
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692833] Node 0 DMA per-cpu:
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692839] CPU    0: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692844] CPU    1: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692848] CPU    2: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692853] CPU    3: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692857] CPU    4: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692861] CPU    5: hi:
0, btch:   1 usd:   0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692865] Node 0 DMA32 per-cpu:
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692870] CPU    0: hi:
186, btch:  31 usd: 165
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692875] CPU    1: hi:
186, btch:  31 usd: 155
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692879] CPU    2: hi:
186, btch:  31 usd:  76
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692883] CPU    3: hi:
186, btch:  31 usd: 144
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692888] CPU    4: hi:
186, btch:  31 usd: 127
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692892] CPU    5: hi:
186, btch:  31 usd: 181
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692895] Node 0 Normal per-cpu:
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692900] CPU    0: hi:
186, btch:  31 usd:  30
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692905] CPU    1: hi:
186, btch:  31 usd:  60
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692909] CPU    2: hi:
186, btch:  31 usd:  37
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692913] CPU    3: hi:
186, btch:  31 usd: 154
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692918] CPU    4: hi:
186, btch:  31 usd:  94
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692922] CPU    5: hi:
186, btch:  31 usd:  81
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692933] active_anon:693215
inactive_anon:175789 isolated_anon:0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692936]  active_file:250
inactive_file:867 isolated_file:0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692938]  unevictable:1350
dirty:1 writeback:1426 unstable:0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692941]  free:6096
slab_reclaimable:4636 slab_unreclaimable:9159
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692943]  mapped:795
shmem:24 pagetables:13357 bounce:0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692948] Node 0 DMA
free:15040kB min:28kB low:32kB high:40kB active_anon:336kB
inactive_anon:416kB active_file:0kB inactive_file:0kB unevictable:0kB
isolated(anon):0kB isolated(file):0kB present:15688kB mlocked:0kB
dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB
slab_unreclaimable:72kB kernel_stack:0kB pagetables:4kB unstable:0kB
bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692972] lowmem_reserve[]:
0 3254 3759 3759
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.692979] Node 0 DMA32
free:8356kB min:6780kB low:8472kB high:10168kB active_anon:2588216kB
inactive_anon:518176kB active_file:180kB inactive_file:1772kB
unevictable:2916kB isolated(anon):0kB isolated(file):0kB
present:3332768kB mlocked:2916kB dirty:0kB writeback:4936kB
mapped:116kB shmem:68kB slab_reclaimable:10880kB
slab_unreclaimable:20960kB kernel_stack:1144kB pagetables:32148kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2976
all_unreclaimable? yes
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.693004] lowmem_reserve[]:
0 0 505 505
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.693011] Node 0 Normal
free:988kB min:1052kB low:1312kB high:1576kB active_anon:184308kB
inactive_anon:184564kB active_file:820kB inactive_file:1696kB
unevictable:2484kB isolated(anon):0kB isolated(file):0kB
present:517120kB mlocked:2484kB dirty:4kB writeback:768kB
mapped:3064kB shmem:28kB slab_reclaimable:7664kB
slab_unreclaimable:15604kB kernel_stack:1592kB pagetables:21276kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4501
all_unreclaimable? yes
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.693035] lowmem_reserve[]: 0 0 0 0
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.693041] Node 0 DMA: 2*4kB
3*8kB 2*16kB 2*32kB 3*64kB 1*128kB 1*256kB 0*512kB 2*1024kB 2*2048kB
2*4096kB = 15040kB
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.693058] Node 0 DMA32:
1155*4kB 46*8kB 15*16kB 10*32kB 6*64kB 5*128kB 3*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 8364kB
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.693074] Node 0 Normal:
181*4kB 33*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB
0*2048kB 0*4096kB = 988kB
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.693091] 36599 total pagecache pages
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.693094] 34748 pages in swap cache
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.693099] Swap cache stats:
add 2323833, delete 2289085, find 790812/869719
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.693103] Free swap  = 0kB
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.693106] Total swap = 5947388kB
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.725269] 983024 pages RAM
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.725272] 32468 pages reserved
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.725275] 27862 pages shared
Sep 21 22:42:40 RANDOMBOXEN kernel: [329446.725278] 941530 pages non-shared
Sep 21 22:42:45 RANDOMBOXEN kernel: [329456.848434] ntpd invoked
oom-killer: gfp_mask=0xd0, order=0, oom_adj=0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848444] ntpd cpuset=/ mems_allowed=0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848450] Pid: 2414, comm:
ntpd Not tainted 2.6.35.4 #1
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848455] Call Trace:
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848468]
[<ffffffff810824f1>] ? cpuset_print_task_mems_allowed+0x8d/0x98
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848478]
[<ffffffff810b2e89>] dump_header+0x65/0x182
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848488]
[<ffffffff8119031b>] ? ___ratelimit+0xc7/0xe4
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848495]
[<ffffffff810b2feb>] oom_kill_process+0x45/0x110
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848502]
[<ffffffff810b3439>] __out_of_memory+0x143/0x15a
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848510]
[<ffffffff810b35a1>] out_of_memory+0x151/0x183
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848518]
[<ffffffff810b726b>] __alloc_pages_nodemask+0x550/0x6a1
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848527]
[<ffffffff810de22b>] alloc_pages_current+0xa8/0xd1
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848534]
[<ffffffff810b6284>] __get_free_pages+0x9/0x46
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848541]
[<ffffffff810fa893>] __pollwait+0x5e/0xcb
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848549]
[<ffffffff81258b37>] datagram_poll+0x23/0xcf
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848556]
[<ffffffff812a8457>] udp_poll+0x18/0x4d
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848562]
[<ffffffff8124d7ab>] sock_poll+0x18/0x1a
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848568]
[<ffffffff810fa17f>] do_select+0x3b2/0x5b9
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848576]
[<ffffffff810fa835>] ? __pollwait+0x0/0xcb
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848583]
[<ffffffff810fa900>] ? pollwake+0x0/0x5c
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848589]
[<ffffffff810fa900>] ? pollwake+0x0/0x5c
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848595]
[<ffffffff810fa900>] ? pollwake+0x0/0x5c
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848602]
[<ffffffff810fa900>] ? pollwake+0x0/0x5c
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848608]
[<ffffffff810fa900>] ? pollwake+0x0/0x5c
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848615]
[<ffffffff810fa900>] ? pollwake+0x0/0x5c
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848621]
[<ffffffff810fa900>] ? pollwake+0x0/0x5c
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848627]
[<ffffffff810fa900>] ? pollwake+0x0/0x5c
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848634]
[<ffffffff810fa900>] ? pollwake+0x0/0x5c
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848641]
[<ffffffff810fa529>] core_sys_select+0x1a3/0x251
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848649]
[<ffffffff81028556>] ? do_page_fault+0x2db/0x31a
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848657]
[<ffffffff8100ba65>] ? restore_i387_xstate+0x6e/0x168
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848665]
[<ffffffff810fa80d>] sys_select+0x94/0xbc
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848673]
[<ffffffff81002b42>] system_call_fastpath+0x16/0x1b
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848678] Mem-Info:
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848681] Node 0 DMA per-cpu:
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848687] CPU    0: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848692] CPU    1: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848696] CPU    2: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848700] CPU    3: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848705] CPU    4: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848709] CPU    5: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848713] Node 0 DMA32 per-cpu:
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848718] CPU    0: hi:
186, btch:  31 usd: 155
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848723] CPU    1: hi:
186, btch:  31 usd:  93
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848727] CPU    2: hi:
186, btch:  31 usd: 163
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848731] CPU    3: hi:
186, btch:  31 usd: 139
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848736] CPU    4: hi:
186, btch:  31 usd: 173
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848740] CPU    5: hi:
186, btch:  31 usd: 158
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848743] Node 0 Normal per-cpu:
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848748] CPU    0: hi:
186, btch:  31 usd: 168
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848753] CPU    1: hi:
186, btch:  31 usd: 124
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848757] CPU    2: hi:
186, btch:  31 usd:  75
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848761] CPU    3: hi:
186, btch:  31 usd:  47
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848765] CPU    4: hi:
186, btch:  31 usd:  67
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848770] CPU    5: hi:
186, btch:  31 usd: 136
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848780] active_anon:694172
inactive_anon:175990 isolated_anon:0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848783]  active_file:408
inactive_file:439 isolated_file:0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848786]  unevictable:1350
dirty:0 writeback:6 unstable:0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848788]  free:6263
slab_reclaimable:4511 slab_unreclaimable:8999
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848790]  mapped:942
shmem:24 pagetables:13273 bounce:0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848796] Node 0 DMA
free:15040kB min:28kB low:32kB high:40kB active_anon:336kB
inactive_anon:416kB active_file:0kB inactive_file:0kB unevictable:0kB
isolated(anon):0kB isolated(file):0kB present:15688kB mlocked:0kB
dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB
slab_unreclaimable:72kB kernel_stack:0kB pagetables:4kB unstable:0kB
bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848819] lowmem_reserve[]:
0 3254 3759 3759
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848826] Node 0 DMA32
free:8756kB min:6780kB low:8472kB high:10168kB active_anon:2591524kB
inactive_anon:518468kB active_file:252kB inactive_file:236kB
unevictable:2912kB isolated(anon):0kB isolated(file):0kB
present:3332768kB mlocked:2912kB dirty:0kB writeback:24kB mapped:204kB
shmem:68kB slab_reclaimable:10524kB slab_unreclaimable:20868kB
kernel_stack:1136kB pagetables:31772kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:768 all_unreclaimable? yes
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848851] lowmem_reserve[]:
0 0 505 505
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848857] Node 0 Normal
free:1256kB min:1052kB low:1312kB high:1576kB active_anon:184828kB
inactive_anon:185076kB active_file:1380kB inactive_file:1520kB
unevictable:2488kB isolated(anon):0kB isolated(file):0kB
present:517120kB mlocked:2488kB dirty:0kB writeback:0kB mapped:3564kB
shmem:28kB slab_reclaimable:7520kB slab_unreclaimable:15056kB
kernel_stack:1592kB pagetables:21316kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:4384 all_unreclaimable? yes
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848881] lowmem_reserve[]: 0 0 0 0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848887] Node 0 DMA: 2*4kB
3*8kB 2*16kB 2*32kB 3*64kB 1*128kB 1*256kB 0*512kB 2*1024kB 2*2048kB
2*4096kB = 15040kB
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848904] Node 0 DMA32:
1311*4kB 23*8kB 10*16kB 11*32kB 6*64kB 5*128kB 3*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 8756kB
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848924] Node 0 Normal:
302*4kB 6*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB
0*2048kB 0*4096kB = 1256kB
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848941] 27025 total pagecache pages
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848944] 25453 pages in swap cache
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848949] Swap cache stats:
add 2328770, delete 2303317, find 790880/869989
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848953] Free swap  = 0kB
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.848956] Total swap = 5947388kB
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.881421] 983024 pages RAM
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.881425] 32468 pages reserved
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.881428] 28009 pages shared
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.881430] 941193 pages non-shared
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928261] gdm invoked
oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928271] gdm cpuset=/ mems_allowed=0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928279] Pid: 2470, comm:
gdm Not tainted 2.6.35.4 #1
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928283] Call Trace:
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928297]
[<ffffffff810824f1>] ? cpuset_print_task_mems_allowed+0x8d/0x98
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928307]
[<ffffffff810b2e89>] dump_header+0x65/0x182
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928314]
[<ffffffff810b3237>] ? badness+0x157/0x216
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928323]
[<ffffffff8119031b>] ? ___ratelimit+0xc7/0xe4
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928330]
[<ffffffff810b2feb>] oom_kill_process+0x45/0x110
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928338]
[<ffffffff810b3439>] __out_of_memory+0x143/0x15a
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928345]
[<ffffffff810b35a1>] out_of_memory+0x151/0x183
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928353]
[<ffffffff810b726b>] __alloc_pages_nodemask+0x550/0x6a1
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928362]
[<ffffffff810de22b>] alloc_pages_current+0xa8/0xd1
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928369]
[<ffffffff810b108a>] __page_cache_alloc+0x77/0x82
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928377]
[<ffffffff810b88ce>] __do_page_cache_readahead+0x96/0x1a2
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928385]
[<ffffffff810b89f6>] ra_submit+0x1c/0x20
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928391]
[<ffffffff810b1582>] filemap_fault+0x1b0/0x31f
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928404]
[<ffffffff810c5e40>] __do_fault+0x50/0x413
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928412]
[<ffffffff810c8433>] handle_mm_fault+0x4d5/0x93e
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928419]
[<ffffffff810eb565>] ? do_sync_read+0xc7/0x10d
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928427]
[<ffffffff81028571>] do_page_fault+0x2f6/0x31a
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928435]
[<ffffffff810917fc>] ? call_rcu_sched+0x10/0x12
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928442]
[<ffffffff81091807>] ? call_rcu+0x9/0xb
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928449]
[<ffffffff8106045e>] ? __put_cred+0x43/0x45
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928456]
[<ffffffff810606d3>] ? commit_creds+0x10f/0x119
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928465]
[<ffffffff812e6fc5>] page_fault+0x25/0x30
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928469] Mem-Info:
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928473] Node 0 DMA per-cpu:
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928479] CPU    0: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928483] CPU    1: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928488] CPU    2: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928492] CPU    3: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928496] CPU    4: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928501] CPU    5: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928504] Node 0 DMA32 per-cpu:
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928510] CPU    0: hi:
186, btch:  31 usd: 155
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928514] CPU    1: hi:
186, btch:  31 usd: 177
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928519] CPU    2: hi:
186, btch:  31 usd: 163
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928523] CPU    3: hi:
186, btch:  31 usd: 139
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928527] CPU    4: hi:
186, btch:  31 usd: 173
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928531] CPU    5: hi:
186, btch:  31 usd: 160
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928535] Node 0 Normal per-cpu:
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928540] CPU    0: hi:
186, btch:  31 usd:  38
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928544] CPU    1: hi:
186, btch:  31 usd:  83
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928548] CPU    2: hi:
186, btch:  31 usd:  24
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928552] CPU    3: hi:
186, btch:  31 usd:  39
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928556] CPU    4: hi:
186, btch:  31 usd:  37
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928561] CPU    5: hi:
186, btch:  31 usd: 137
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928572] active_anon:694170
inactive_anon:175944 isolated_anon:64
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928574]  active_file:365
inactive_file:630 isolated_file:0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928577]  unevictable:1350
dirty:0 writeback:6 unstable:0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928579]  free:6170
slab_reclaimable:4511 slab_unreclaimable:8999
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928582]  mapped:948
shmem:24 pagetables:13273 bounce:0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928587] Node 0 DMA
free:15040kB min:28kB low:32kB high:40kB active_anon:336kB
inactive_anon:416kB active_file:0kB inactive_file:0kB unevictable:0kB
isolated(anon):0kB isolated(file):0kB present:15688kB mlocked:0kB
dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB
slab_unreclaimable:72kB kernel_stack:0kB pagetables:4kB unstable:0kB
bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928610] lowmem_reserve[]:
0 3254 3759 3759
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928618] Node 0 DMA32
free:8632kB min:6780kB low:8472kB high:10168kB active_anon:2591516kB
inactive_anon:518220kB active_file:80kB inactive_file:0kB
unevictable:2912kB isolated(anon):256kB isolated(file):0kB
present:3332768kB mlocked:2912kB dirty:0kB writeback:24kB mapped:212kB
shmem:68kB slab_reclaimable:10524kB slab_unreclaimable:20868kB
kernel_stack:1136kB pagetables:31772kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:16642 all_unreclaimable? yes
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928642] lowmem_reserve[]:
0 0 505 505
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928649] Node 0 Normal
free:1008kB min:1052kB low:1312kB high:1576kB active_anon:184828kB
inactive_anon:185140kB active_file:1380kB inactive_file:2556kB
unevictable:2488kB isolated(anon):0kB isolated(file):0kB
present:517120kB mlocked:2488kB dirty:0kB writeback:0kB mapped:3580kB
shmem:28kB slab_reclaimable:7520kB slab_unreclaimable:15056kB
kernel_stack:1592kB pagetables:21316kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:4384 all_unreclaimable? yes
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928673] lowmem_reserve[]: 0 0 0 0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928679] Node 0 DMA: 2*4kB
3*8kB 2*16kB 2*32kB 3*64kB 1*128kB 1*256kB 0*512kB 2*1024kB 2*2048kB
2*4096kB = 15040kB
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928696] Node 0 DMA32:
1311*4kB 23*8kB 11*16kB 11*32kB 6*64kB 5*128kB 3*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 8772kB
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928712] Node 0 Normal:
252*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB
0*2048kB 0*4096kB = 1008kB
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928729] 27271 total pagecache pages
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928732] 25477 pages in swap cache
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928737] Swap cache stats:
add 2328796, delete 2303319, find 790882/869993
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928741] Free swap  = 0kB
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.928744] Total swap = 5947388kB
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.961130] 983024 pages RAM
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.961133] 32468 pages reserved
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.961136] 27963 pages shared
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.961139] 941277 pages non-shared
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964179] gdm invoked
oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964183] gdm cpuset=/ mems_allowed=0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964185] Pid: 2470, comm:
gdm Not tainted 2.6.35.4 #1
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964186] Call Trace:
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964192]
[<ffffffff810824f1>] ? cpuset_print_task_mems_allowed+0x8d/0x98
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964195]
[<ffffffff810b2e89>] dump_header+0x65/0x182
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964199]
[<ffffffff8119031b>] ? ___ratelimit+0xc7/0xe4
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964201]
[<ffffffff810b2feb>] oom_kill_process+0x45/0x110
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964203]
[<ffffffff810b3439>] __out_of_memory+0x143/0x15a
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964205]
[<ffffffff810b35a1>] out_of_memory+0x151/0x183
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964207]
[<ffffffff810b726b>] __alloc_pages_nodemask+0x550/0x6a1
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964210]
[<ffffffff810de22b>] alloc_pages_current+0xa8/0xd1
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964213]
[<ffffffff810b108a>] __page_cache_alloc+0x77/0x82
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964215]
[<ffffffff810b88ce>] __do_page_cache_readahead+0x96/0x1a2
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964217]
[<ffffffff810b89f6>] ra_submit+0x1c/0x20
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964219]
[<ffffffff810b1582>] filemap_fault+0x1b0/0x31f
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964222]
[<ffffffff810c5e40>] __do_fault+0x50/0x413
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964224]
[<ffffffff810c8433>] handle_mm_fault+0x4d5/0x93e
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964227]
[<ffffffff810eb565>] ? do_sync_read+0xc7/0x10d
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964229]
[<ffffffff81028571>] do_page_fault+0x2f6/0x31a
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964232]
[<ffffffff810917fc>] ? call_rcu_sched+0x10/0x12
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964234]
[<ffffffff81091807>] ? call_rcu+0x9/0xb
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964236]
[<ffffffff8106045e>] ? __put_cred+0x43/0x45
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964238]
[<ffffffff810606d3>] ? commit_creds+0x10f/0x119
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964241]
[<ffffffff812e6fc5>] page_fault+0x25/0x30
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964242] Mem-Info:
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964243] Node 0 DMA per-cpu:
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964245] CPU    0: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964247] CPU    1: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964248] CPU    2: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964249] CPU    3: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964250] CPU    4: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964252] CPU    5: hi:
0, btch:   1 usd:   0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964253] Node 0 DMA32 per-cpu:
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964254] CPU    0: hi:
186, btch:  31 usd: 155
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964256] CPU    1: hi:
186, btch:  31 usd: 177
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964257] CPU    2: hi:
186, btch:  31 usd: 163
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964258] CPU    3: hi:
186, btch:  31 usd: 139
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964259] CPU    4: hi:
186, btch:  31 usd: 173
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964261] CPU    5: hi:
186, btch:  31 usd: 160
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964262] Node 0 Normal per-cpu:
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964263] CPU    0: hi:
186, btch:  31 usd:  38
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964264] CPU    1: hi:
186, btch:  31 usd:  83
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964266] CPU    2: hi:
186, btch:  31 usd:  23
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964267] CPU    3: hi:
186, btch:  31 usd:  39
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964268] CPU    4: hi:
186, btch:  31 usd:  37
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964269] CPU    5: hi:
186, btch:  31 usd: 142
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964273] active_anon:694170
inactive_anon:175944 isolated_anon:64
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964273]  active_file:365
inactive_file:630 isolated_file:0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964274]  unevictable:1350
dirty:0 writeback:6 unstable:0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964275]  free:6170
slab_reclaimable:4511 slab_unreclaimable:8999
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964275]  mapped:948
shmem:24 pagetables:13273 bounce:0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964277] Node 0 DMA
free:15040kB min:28kB low:32kB high:40kB active_anon:336kB
inactive_anon:416kB active_file:0kB inactive_file:0kB unevictable:0kB
isolated(anon):0kB isolated(file):0kB present:15688kB mlocked:0kB
dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB
slab_unreclaimable:72kB kernel_stack:0kB pagetables:4kB unstable:0kB
bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964284] lowmem_reserve[]:
0 3254 3759 3759
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964286] Node 0 DMA32
free:8632kB min:6780kB low:8472kB high:10168kB active_anon:2591516kB
inactive_anon:518220kB active_file:80kB inactive_file:0kB
unevictable:2912kB isolated(anon):256kB isolated(file):0kB
present:3332768kB mlocked:2912kB dirty:0kB writeback:24kB mapped:212kB
shmem:68kB slab_reclaimable:10524kB slab_unreclaimable:20868kB
kernel_stack:1136kB pagetables:31772kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:16642 all_unreclaimable? yes
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964293] lowmem_reserve[]:
0 0 505 505
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964295] Node 0 Normal
free:1008kB min:1052kB low:1312kB high:1576kB active_anon:184828kB
inactive_anon:185140kB active_file:1380kB inactive_file:2556kB
unevictable:2488kB isolated(anon):0kB isolated(file):0kB
present:517120kB mlocked:2488kB dirty:0kB writeback:0kB mapped:3580kB
shmem:28kB slab_reclaimable:7520kB slab_unreclaimable:15056kB
kernel_stack:1592kB pagetables:21316kB unstable:0kB bounce:0kB
writeback_tmp:0kB pages_scanned:4384 all_unreclaimable? yes
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964302] lowmem_reserve[]: 0 0 0 0
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964303] Node 0 DMA: 2*4kB
3*8kB 2*16kB 2*32kB 3*64kB 1*128kB 1*256kB 0*512kB 2*1024kB 2*2048kB
2*4096kB = 15040kB
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964308] Node 0 DMA32:
1311*4kB 23*8kB 11*16kB 11*32kB 6*64kB 5*128kB 3*256kB 0*512kB
1*1024kB 0*2048kB 0*4096kB = 8772kB
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964313] Node 0 Normal:
252*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB
0*2048kB 0*4096kB = 1008kB
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964318] 27271 total pagecache pages
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964319] 25477 pages in swap cache
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964320] Swap cache stats:
add 2328796, delete 2303319, find 790882/869994
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964322] Free swap  = 0kB
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.964322] Total swap = 5947388kB
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.976647] 983024 pages RAM
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.976648] 32468 pages reserved
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.976649] 27964 pages shared
Sep 21 22:42:46 RANDOMBOXEN kernel: [329456.976649] 941277 pages non-shared