From: Wu Fengguang <fengguang.wu@intel.com>
To: Jan Kara <jack@suse.cz>
Cc: "linux-mm@kvack.org" <linux-mm@kvack.org>,
"user-mode-linux-devel@lists.sourceforge.net"
<user-mode-linux-devel@lists.sourceforge.net>
Subject: Re: Problems with IO-less throttling?
Date: Tue, 8 Nov 2011 15:38:57 +0800 [thread overview]
Message-ID: <20111108073857.GA1363@localhost> (raw)
In-Reply-To: <20111107233657.GL15796@quack.suse.cz>
Hi Jan,
On Tue, Nov 08, 2011 at 07:36:57AM +0800, Jan Kara wrote:
> Hello,
>
> today I was testing some patch with Linus' kernel from today (commit
> 31555213f03bca37d2c02e10946296052f4ecfcd) and I rather easily tripped
> out-of-memory messages and processes got killed.
>
> The unusual thing I'm doing is that I'm using user-mode-linux and
> the started virtual machine has relatively low amount of memory (I was
> tripping OOM immediately with mem=64M or mem=128M, I could still hit it
> after a while with mem=256M). The load I was running was just "dd
> if=/dev/zero of=/tmp/file bs=128k". The backing filesystem was ext3. Now
> with older kernel (e.g. 3.1) running even with mem=64M was fine.
>
> Example of an OOM message:
>
> dd invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0,
> oom_score_adj=0
> Call Trace:
> 6ff5b868: [<602f550d>] _raw_spin_unlock+0x9/0xb
> 6ff5b878: [<6005ad53>] dump_header.clone.2+0xba/0x1b6
> 6ff5b8b8: [<6005af77>] oom_kill_process.clone.0+0x45/0x236
> 6ff5b928: [<6005b506>] out_of_memory+0x25c/0x305
> 6ff5b9a8: [<6005e563>] __alloc_pages_nodemask+0x558/0x5fd
> 6ff5ba08: [<60270a2b>] radix_tree_lookup_slot+0xe/0x10
> 6ff5ba88: [<60065933>] shmem_getpage_gfp+0x269/0x475
> 6ff5bb18: [<60065b72>] shmem_write_begin+0x33/0x35
> 6ff5bb28: [<60059fa4>] generic_file_buffered_write+0x132/0x28f
> 6ff5bbf8: [<6005a47a>] __generic_file_aio_write+0x379/0x3b5
> 6ff5bcb8: [<6005a53a>] generic_file_aio_write+0x84/0xdd
> 6ff5bd28: [<6007fb09>] do_sync_write+0xd1/0x10e
> 6ff5bdc8: [<600174c1>] segv+0x8f/0x24a
> 6ff5be58: [<6007fc0b>] vfs_write+0xc5/0x16d
> 6ff5be98: [<6007fd64>] sys_write+0x45/0x6c
> 6ff5bed8: [<60017e48>] handle_syscall+0x50/0x70
> 6ff5bef8: [<60024ae6>] userspace+0x320/0x3d6
> 6ff5bfc8: [<600151ac>] fork_handler+0x7d/0x84
>
> Mem-Info:
> Normal per-cpu:
> CPU 0: hi: 90, btch: 15 usd: 0
> active_anon:850 inactive_anon:59583 isolated_anon:0
> active_file:0 inactive_file:16 isolated_file:0
> unevictable:0 dirty:0 writeback:0 unstable:0
> free:510 slab_reclaimable:540 slab_unreclaimable:461
> mapped:0 shmem:59685 pagetables:111 bounce:0
> Normal free:2040kB min:2032kB low:2540kB high:3048kB active_anon:3400kB
> inactive_anon:238332kB active_file:0kB inactive_file:64kB unevictable:0kB
> isolated(anon):0kB isolated(file):0kB present:258560kB mlocked:0kB
> dirty:0kB writeback:0kB mapped:0kB shmem:238740kB slab_reclaimable:2160kB
It looks most present pages are consumed by shmem/inactive_anon. Do
you have idea what's the shmem used for?
Thanks,
Fengguang
> slab_unreclaimable:1844kB kernel_stack:272kB pagetables:444kB unstable:0kB
> bounce:0kB writeback_tmp:0kB pages_scanned:46434 all_unreclaimable? yes
> lowmem_reserve[]: 0 0
> Normal: 498*4kB 4*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB
> 0*2048kB 0*4096kB = 2040kB
> 59701 total pagecache pages
> 0 pages in swap cache
> Swap cache stats: add 0, delete 0, find 0/0
> Free swap = 0kB
> Total swap = 0kB
> 65536 pages RAM
> 3222 pages reserved
> 16 pages shared
> 61703 pages non-shared
> [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
> [ 670] 0 670 1489 37 0 0 0 syslogd
> [ 676] 0 676 682 21 0 0 0 klogd
> [ 720] 0 720 5597 94 0 0 0 exim4
> [ 725] 0 725 680 18 0 0 0 inetd
> [ 739] 0 739 2282 36 0 0 0 atd
> [ 745] 0 745 2890 47 0 0 0 cron
> [ 762] 0 762 6338 75 0 0 0 login
> [ 763] 0 763 6338 74 0 0 0 login
> [ 764] 0 764 2593 131 0 0 0 bash
> [ 772] 0 772 2590 129 0 0 0 bash
> [ 805] 0 805 1262 58 0 0 0 dd
> Out of memory: Kill process 670 (syslogd) score 1 or sacrifice child
> Killed process 670 (syslogd) total-vm:5956kB, anon-rss:148kB, file-rss:0kB
>
> I'd be suspecting new IO-less throttling code but from the OOM report it
> seems there are no dirty or writeback pages so that seems to speak against
> that theory.
>
> Honza
> --
> Jan Kara <jack@suse.cz>
> SUSE Labs, CR
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2011-11-08 7:39 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-11-07 23:36 Problems with IO-less throttling? Jan Kara
2011-11-08 7:38 ` Wu Fengguang [this message]
2011-11-08 13:19 ` Jan Kara
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20111108073857.GA1363@localhost \
--to=fengguang.wu@intel.com \
--cc=jack@suse.cz \
--cc=linux-mm@kvack.org \
--cc=user-mode-linux-devel@lists.sourceforge.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).