From: Jan Stancek <jstancek@redhat.com>
To: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>, linux-mm@kvack.org
Cc: ltp@lists.linux.it
Subject: Re: [BUG] oom hangs the system, NMI backtrace shows most CPUs in shrink_slab
Date: Fri, 22 Jan 2016 16:14:40 +0100 [thread overview]
Message-ID: <56A24760.5020503@redhat.com> (raw)
In-Reply-To: <569E1010.2070806@I-love.SAKURA.ne.jp>
On 01/19/2016 11:29 AM, Tetsuo Handa wrote:
> although I
> couldn't find evidence that mlock() and madvice() are related with this hangup,
I simplified reproducer by having only single thread allocating
memory when OOM triggers:
http://jan.stancek.eu/tmp/oom_hangs/console.log.3-v4.4-8606-with-memalloc.txt
In this instance it was mmap + mlock, as you can see from oom call trace.
It made it to do_exit(), but couldn't complete it:
[19419.667714] oom01 R running task 11568 32704 32676 0x00000084
[19419.675606] 0000000000000000 0000000000000000 ffffffff81c8dfc0 0000000000000007
[19419.683910] ffff88045fbd7ad8 ffff880451c88000 ffff88045aac8000 ffff880456f18000
[19419.692207] ffff88044fb239c0 0000000000000000 ffffffff81205113 ffff880456f17428
[19419.700507] Call Trace:
[19419.703235] [<ffffffff8177e8af>] ? _raw_spin_unlock+0x1f/0x40
[19419.709746] [<ffffffff81779207>] ? preempt_schedule_common+0x1f/0x38
[19419.716936] [<ffffffff8177923c>] ? _cond_resched+0x1c/0x30
[19419.723146] [<ffffffff811e7369>] ? shrink_slab.part.42+0x119/0x540
[19419.730141] [<ffffffff81027c59>] ? sched_clock+0x9/0x10
[19419.736070] [<ffffffff811ecc2a>] ? shrink_zone+0x30a/0x330
[19419.742290] [<ffffffff811ecff4>] ? do_try_to_free_pages+0x174/0x440
[19419.749381] [<ffffffff811ed3c0>] ? try_to_free_pages+0x100/0x2c0
[19419.756182] [<ffffffff81269102>] ? __alloc_pages_slowpath+0x278/0x78c
[19419.763468] [<ffffffff811dcdb1>] ? __alloc_pages_nodemask+0x4a1/0x4d0
[19419.770755] [<ffffffff810d592c>] ? local_clock+0x1c/0x30
[19419.776782] [<ffffffff8123482e>] ? alloc_pages_vma+0xbe/0x2d0
[19419.783294] [<ffffffff812224d8>] ? __read_swap_cache_async+0x118/0x200
[19419.790666] [<ffffffff812225e6>] ? read_swap_cache_async+0x26/0x60
[19419.797665] [<ffffffff81222785>] ? swapin_readahead+0x165/0x1f0
[19419.804367] [<ffffffff811d0e15>] ? find_get_entry+0x5/0x220
[19419.810686] [<ffffffff811d1c4c>] ? pagecache_get_page+0x2c/0x270
[19419.817487] [<ffffffff810f930a>] ? __lock_acquire+0x2aa/0x1130
[19419.824094] [<ffffffff8120aeac>] ? do_swap_page.isra.61+0x48c/0x830
[19419.831185] [<ffffffff810f51f4>] ? __lock_is_held+0x54/0x70
[19419.837502] [<ffffffff8120d3b9>] ? handle_mm_fault+0xa99/0x1720
[19419.844207] [<ffffffff8120c979>] ? handle_mm_fault+0x59/0x1720
[19419.850814] [<ffffffff810f51f4>] ? __lock_is_held+0x54/0x70
[19419.857130] [<ffffffff8107480a>] ? __do_page_fault+0x1ca/0x470
[19419.863737] [<ffffffff81074ae0>] ? do_page_fault+0x30/0x80
[19419.869955] [<ffffffff8177ff87>] ? native_iret+0x7/0x7
[19419.875788] [<ffffffff81781838>] ? page_fault+0x28/0x30
[19419.881708] [<ffffffff813c77ef>] ? __get_user_8+0x1f/0x29
[19419.887831] [<ffffffff8113b072>] ? exit_robust_list+0x52/0x1a0
[19419.894437] [<ffffffff810f8cfd>] ? trace_hardirqs_on+0xd/0x10
[19419.900952] [<ffffffff81095913>] ? mm_release+0x143/0x160
[19419.907075] [<ffffffff8109bf36>] ? do_exit+0x166/0xce0
[19419.912899] [<ffffffff8109cb3c>] ? do_group_exit+0x4c/0xc0
[19419.919118] [<ffffffff810aaaf1>] ? get_signal+0x331/0x8f0
[19419.925243] [<ffffffff8101d3c7>] ? do_signal+0x37/0x680
[19419.931172] [<ffffffff810d592c>] ? local_clock+0x1c/0x30
[19419.937197] [<ffffffff81117653>] ? rcu_read_lock_sched_held+0x93/0xa0
[19419.944485] [<ffffffff8123f11d>] ? kfree+0x1bd/0x290
[19419.950124] [<ffffffff810926a4>] ? exit_to_usermode_loop+0x33/0xac
[19419.957110] [<ffffffff810926cf>] ? exit_to_usermode_loop+0x5e/0xac
[19419.964107] [<ffffffff81003d0b>] ? syscall_return_slowpath+0xbb/0x130
[19419.971383] [<ffffffff8177f55a>] ? int_ret_from_sys_call+0x25/0x9f
[19430.309232] MemAlloc-Info: 12 stalling task, 0 dying task, 0 victim task.
[19430.316821] MemAlloc: auditd(783) seq=615 gfp=0x24201ca order=0 delay=40786
[19430.324593] MemAlloc: irqbalance(806) seq=8107 gfp=0x24201ca order=0 delay=44125
[19430.332847] MemAlloc: chronyd(808) seq=3155 gfp=0x24200ca order=0 delay=10259
[19430.340812] MemAlloc: systemd-logind(818) seq=2191 gfp=0x24200ca order=0 delay=21762
[19430.349456] MemAlloc: NetworkManager(820) seq=10854 gfp=0x24200ca order=0 delay=45212
[19430.358190] MemAlloc: gssproxy(826) seq=586 gfp=0x24201ca order=0 delay=44172
[19430.366157] MemAlloc: tuned(1337) seq=40098 gfp=0x24201ca order=0 delay=45585
[19430.374121] MemAlloc: crond(2242) seq=5612 gfp=0x24201ca order=0 delay=41014
[19430.381989] MemAlloc: systemd-journal(22961) seq=151917 gfp=0x24201ca order=0 delay=46264
[19430.391118] MemAlloc: sendmail(31908) seq=7256 gfp=0x24200ca order=0 delay=43318
[19430.399365] MemAlloc: kworker/2:2(32161) seq=9 gfp=0x2400000 order=0 delay=45574
[19430.407619] MemAlloc: oom01(32704) seq=6391 gfp=0x24200ca order=0 delay=44849 exiting
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2016-01-22 15:14 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-01-18 15:38 [BUG] oom hangs the system, NMI backtrace shows most CPUs in shrink_slab Jan Stancek
2016-01-19 10:29 ` Tetsuo Handa
2016-01-19 15:13 ` Jan Stancek
2016-01-20 10:23 ` [BUG] oom hangs the system, NMI backtrace shows most CPUs inshrink_slab Tetsuo Handa
2016-01-20 13:17 ` [BUG] oom hangs the system, NMI backtrace shows most CPUs in shrink_slab Tetsuo Handa
2016-01-20 15:10 ` Tejun Heo
2016-01-20 15:54 ` Tetsuo Handa
2016-01-22 15:14 ` Jan Stancek [this message]
2016-01-23 6:30 ` Tetsuo Handa
2016-01-26 7:48 ` [LTP] " Jan Stancek
2016-01-26 14:46 ` Tetsuo Handa
2016-01-27 11:02 ` Tetsuo Handa
2016-01-28 15:48 ` Tetsuo Handa
2016-01-29 7:32 ` Jan Stancek
2016-01-29 12:35 ` Tetsuo Handa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=56A24760.5020503@redhat.com \
--to=jstancek@redhat.com \
--cc=linux-mm@kvack.org \
--cc=ltp@lists.linux.it \
--cc=penguin-kernel@I-love.SAKURA.ne.jp \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).