From: Ingo Molnar <mingo@kernel.org>
To: Dmitry Vyukov <dvyukov@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
Wang Nan <wangnan0@huawei.com>, Ingo Molnar <mingo@redhat.com>,
LKML <linux-kernel@vger.kernel.org>,
He Kuang <hekuang@huawei.com>,
Alexei Starovoitov <ast@kernel.org>,
Arnaldo Carvalho de Melo <acme@redhat.com>,
Brendan Gregg <brendan.d.gregg@gmail.com>,
Jiri Olsa <jolsa@kernel.org>,
Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>,
Namhyung Kim <namhyung@kernel.org>, Zefan Li <lizefan@huawei.com>,
pi3orama@163.com
Subject: Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer
Date: Tue, 8 Mar 2016 19:02:39 +0100 [thread overview]
Message-ID: <20160308180238.GA4220@gmail.com> (raw)
In-Reply-To: <20160308175753.GA3015@gmail.com>
* Ingo Molnar <mingo@kernel.org> wrote:
>
> * Ingo Molnar <mingo@kernel.org> wrote:
>
> > Things got a lot more lively after that!
> >
> > But most of the overhead seems to come from systemd trying to dump core or
> > something like that:
> >
> > 85872 mingo 20 0 34712 3016 2656 S 4.6 0.0 0:00.14 systemd-coredum
> > 85440 mingo 20 0 34712 3028 2664 S 4.2 0.0 0:00.13 systemd-coredum
> > 85751 mingo 20 0 34712 3076 2716 S 4.2 0.0 0:00.13 systemd-coredum
> > 85840 mingo 20 0 34712 2988 2624 S 4.2 0.0 0:00.13 systemd-coredum
> > 85861 mingo 20 0 34712 3080 2720 S 4.2 0.0 0:00.13 systemd-coredum
> > 85954 mingo 20 0 34712 3028 2664 S 4.2 0.0 0:00.13 systemd-coredum
> >
> > and I have:
> >
> > fomalhaut:~/go/src/github.com/google/syzkaller> ulimit -c
> > 0
> >
> > weird ... Has any of you seen such behavior?
>
> So the workaround for that is to disable systemd trying to log every core dump to
> the system journal (!), via:
>
> echo > /proc/sys/kernel/core_pattern
So with that fixed, I finally started fuzzing for real.
With nproc set to 120 it seems to be chugging along at about 25% system
utilization:
Tasks: 1271 total, 1 running, 1270 sleeping, 0 stopped, 0 zombie
%Cpu(s): 4.5 us, 35.3 sy, 0.0 ni, 55.2 id, 0.5 wa, 4.5 hi, 0.0 si, 0.0 st
KiB Mem : 26401230+total, 25401017+free, 1624640 used, 8377496 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 26143329+avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
87593 mingo 20 0 7850436 162284 11672 S 2941 0.1 33:01.83 syz-fuzzer
923 root 20 0 84772 44344 43840 S 22.8 0.0 1:23.86 systemd-journal
1369 root 16 -4 114636 3256 2832 S 15.7 0.0 0:29.03 auditd
1379 root 12 -8 80236 1764 1432 S 8.3 0.0 0:15.79 audispd
878 root 20 0 0 0 0 S 6.9 0.0 0:16.55 jbd2/sda1-8
1381 root 16 -4 52216 3232 2892 S 3.8 0.0 0:07.08 sedispatch
Even that one is not ideal - obviously there's way too much systemd-journal
overhead, but I'm unable to turn the darn thing off ...
with nproc=480 it does not seem to be working very well - it quickly generates:
2016/03/08 18:59:38 local-0: lost connection: exit status 2
2016/03/08 18:59:38 local-0: saving crash 'lost connection' to crash-local-0-1457459978295423413
and then after some time starts to ramp up again. It's mostly idling around.
Thanks,
Ingo
next prev parent reply other threads:[~2016-03-08 18:02 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-03-07 3:50 [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Wang Nan
2016-03-07 3:50 ` [RESEND PATCH 1/5] perf core: Introduce new ioctl options to pause and resume " Wang Nan
2016-03-07 3:50 ` [RESEND PATCH 2/5] perf core: Set event's default overflow_handler Wang Nan
2016-03-07 3:50 ` [RESEND PATCH 3/5] perf core: Prepare writing into ring buffer from end Wang Nan
2016-03-07 3:50 ` [RESEND PATCH 4/5] perf core: Add backward attribute to perf event Wang Nan
2016-03-07 3:50 ` [RESEND PATCH 5/5] perf core: Reduce perf event output overhead by new overflow handler Wang Nan
2016-03-08 13:44 ` [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Peter Zijlstra
2016-03-08 13:49 ` Ingo Molnar
2016-03-08 13:57 ` Peter Zijlstra
2016-03-08 15:29 ` Ingo Molnar
2016-03-08 15:35 ` Dmitry Vyukov
2016-03-08 15:54 ` Ingo Molnar
2016-03-08 16:11 ` Dmitry Vyukov
2016-03-08 16:27 ` Ingo Molnar
2016-03-08 16:29 ` Dmitry Vyukov
2016-03-08 16:32 ` Peter Zijlstra
2016-03-08 16:44 ` Ingo Molnar
2016-03-08 16:48 ` Ingo Molnar
2016-03-08 16:59 ` Dmitry Vyukov
2016-03-08 17:24 ` Ingo Molnar
2016-03-08 17:27 ` Dmitry Vyukov
2016-03-08 17:37 ` Ingo Molnar
2016-03-08 17:41 ` Dmitry Vyukov
2016-03-08 17:48 ` Ingo Molnar
2016-03-08 17:56 ` Dmitry Vyukov
2016-03-08 17:56 ` Peter Zijlstra
2016-03-08 17:57 ` Ingo Molnar
2016-03-08 18:02 ` Ingo Molnar [this message]
2016-03-08 18:22 ` Ingo Molnar
2016-03-08 18:31 ` Ingo Molnar
2016-03-08 17:55 ` Peter Zijlstra
2016-03-08 16:30 ` Peter Zijlstra
2016-03-09 10:53 ` Borislav Petkov
2016-03-09 11:19 ` Dmitry Vyukov
2016-03-08 19:56 ` Jiri Olsa
2016-03-08 20:07 ` Peter Zijlstra
2016-03-08 20:44 ` Jiri Olsa
2016-03-08 21:04 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160308180238.GA4220@gmail.com \
--to=mingo@kernel.org \
--cc=acme@redhat.com \
--cc=ast@kernel.org \
--cc=brendan.d.gregg@gmail.com \
--cc=dvyukov@google.com \
--cc=hekuang@huawei.com \
--cc=jolsa@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lizefan@huawei.com \
--cc=masami.hiramatsu.pt@hitachi.com \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=pi3orama@163.com \
--cc=wangnan0@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox