All of lore.kernel.org
 help / color / mirror / Atom feed
From: David Ahern <dsahern@gmail.com>
To: Dave Jones <davej@redhat.com>,
	Dave Hansen <dave.hansen@intel.com>,
	Ingo Molnar <mingo@kernel.org>,
	Markus Trippelsdorf <markus@trippelsdorf.de>,
	Thomas Gleixner <tglx@linutronix.de>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Linux Kernel <linux-kernel@vger.kernel.org>,
	Peter Anvin <hpa@zytor.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Dave Hansen <dave.hansen@linux.intel.com>
Subject: Re: Yet more softlockups.
Date: Fri, 12 Jul 2013 11:12:13 -0600	[thread overview]
Message-ID: <51E038ED.7050600@gmail.com> (raw)
In-Reply-To: <20130712154521.GD1020@redhat.com>

On 7/12/13 9:45 AM, Dave Jones wrote:
> Here's a fun trick:
>
> trinity -c perf_event_open -C4 -q -l off
>
> Within about a minute, that brings any of my boxes to its knees.
> The softlockup detector starts going nuts, and then the box wedges solid.

I tried that in a VM running latest Linus tree. I see trinity children 
getting nuked regularly from oom.

I was dumping Vm elements using:

while [ 1 ]; do echo $(date) $(egrep Vm /proc/$pid/status); sleep 1; done

And right before the process is killed was the line:

Fri Jul 12 11:00:19 MDT 2013 VmPeak: 2867472 kB VmSize: 2867472 kB 
VmLck: 0 kB VmPin: 0 kB VmHWM: 1493092 kB VmRSS: 1493092 kB VmData: 
2857944 kB VmStk: 136 kB VmExe: 100 kB VmLib: 1844 kB VmPTE: 5628 kB 
VmSwap: 0 kB

The VmData is growing fairly steadily and strace shows a lot of brk 
calls. Is that normal for trinity - or this command line?

Looking at the perf_event_open calls I see a lot of E2BIG errors in 
addition to EINVAL. e.g,

...
perf_event_open(0xba9000, 0, 0x4c, 0xcc, 0) = -1 EINVAL (Invalid argument)
alarm(0)                                = 1
getppid()                               = 9031
alarm(1)                                = 0
perf_event_open(0xba9000, 0x2a6e, 0xe, 0xfd, 0) = -1 E2BIG (Argument 
list too long)
alarm(0)                                = 1
getppid()                               = 9031
alarm(1)                                = 0
...

David

  parent reply	other threads:[~2013-07-12 17:12 UTC|newest]

Thread overview: 41+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-07-04  1:55 scheduling while atomic & hang Dave Jones
2013-07-04  2:49 ` Linus Torvalds
2013-07-04  4:43   ` H. Peter Anvin
2013-07-04  7:49   ` Dave Jones
2013-07-04 17:22     ` Linus Torvalds
2013-07-10 20:54     ` J. Bruce Fields
2013-07-04 18:08   ` H. Peter Anvin
2013-07-05  6:51     ` Ingo Molnar
2013-07-05  7:19       ` Frederic Weisbecker
2013-07-05 10:05         ` Ingo Molnar
2013-07-05 19:27       ` Linus Torvalds
2013-07-06  8:02         ` Frederic Weisbecker
2013-07-06  8:07           ` Ingo Molnar
2013-07-05 14:38   ` Yet more softlockups Dave Jones
2013-07-05 15:15     ` Thomas Gleixner
2013-07-05 16:00       ` Dave Jones
2013-07-05 16:02         ` Thomas Gleixner
2013-07-05 16:41           ` H. Peter Anvin
2013-07-05 18:20             ` Seiji Aguchi
2013-07-05 22:21               ` Thomas Gleixner
2013-07-06  7:24         ` Ingo Molnar
2013-07-07  0:24           ` Dave Jones
2013-07-10 15:13           ` Dave Jones
2013-07-10 15:20             ` Markus Trippelsdorf
2013-07-10 15:40               ` Dave Jones
2013-07-12 10:31                 ` Ingo Molnar
2013-07-12 15:38                   ` Dave Hansen
2013-07-12 15:45                     ` Dave Jones
2013-07-12 15:55                       ` Dave Hansen
2013-07-12 17:00                         ` Dave Jones
2013-07-12 17:12                       ` David Ahern [this message]
2013-07-12 17:18                         ` Dave Jones
2013-07-12 17:40                           ` David Ahern
2013-07-12 17:50                             ` Dave Jones
2013-07-12 18:07                               ` David Ahern
2013-07-12 18:22                                 ` Dave Hansen
2013-07-12 20:13                                 ` Dave Hansen
2013-07-13  1:40                       ` Vince Weaver
2013-07-10 15:39             ` Vince Weaver
2013-07-10 15:45               ` Dave Jones
2013-07-10 21:54               ` Dave Jones

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=51E038ED.7050600@gmail.com \
    --to=dsahern@gmail.com \
    --cc=dave.hansen@intel.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=davej@redhat.com \
    --cc=hpa@zytor.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=markus@trippelsdorf.de \
    --cc=mingo@kernel.org \
    --cc=peterz@infradead.org \
    --cc=tglx@linutronix.de \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.