From: Ingo Molnar <mingo@kernel.org>
To: Dmitry Vyukov <dvyukov@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
Wang Nan <wangnan0@huawei.com>, Ingo Molnar <mingo@redhat.com>,
LKML <linux-kernel@vger.kernel.org>,
He Kuang <hekuang@huawei.com>,
Alexei Starovoitov <ast@kernel.org>,
Arnaldo Carvalho de Melo <acme@redhat.com>,
Brendan Gregg <brendan.d.gregg@gmail.com>,
Jiri Olsa <jolsa@kernel.org>,
Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>,
Namhyung Kim <namhyung@kernel.org>, Zefan Li <lizefan@huawei.com>,
pi3orama@163.com
Subject: Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer
Date: Tue, 8 Mar 2016 18:24:25 +0100 [thread overview]
Message-ID: <20160308172425.GA3017@gmail.com> (raw)
In-Reply-To: <CACT4Y+amJpRg9jdz1vEupJO8gsbeh7PuOKTiuvkuu7cWigY50Q@mail.gmail.com>
* Dmitry Vyukov <dvyukov@google.com> wrote:
> On Tue, Mar 8, 2016 at 5:48 PM, Ingo Molnar <mingo@kernel.org> wrote:
> >
> > * Ingo Molnar <mingo@kernel.org> wrote:
> >
> >> It only had a couple of seconds of runtime:
> >>
> >> 49652 mingo 20 0 1434276 52144 11344 S 0.0 0.0 0:00.54 syz-manager
> >> 49661 mingo 20 0 2196672 43948 10448 S 0.0 0.0 0:05.59 syz-fuzzer
> >
> > Ah, so it appears to making some progress:
> >
> > 49652 mingo 20 0 1581740 47600 11344 S 0.0 0.0 0:00.58 syz-manager
> > 49661 mingo 20 0 2204868 43720 10448 S 0.0 0.0 0:07.49 syz-fuzzer
> >
> > 49652 mingo 20 0 1598132 31512 11344 S 0.0 0.0 0:00.61 syz-manager
> > 49661 mingo 20 0 2204868 44252 10448 S 0.0 0.0 0:09.09 syz-fuzzer
> >
> > but only about +1 second runtime added every minute or so. Is that expected?
>
> The main work is done by child syz-executor processes.
Hm, they don't seem to be doing anything:
fomalhaut:~> ps aux | grep syz-executor
mingo 41506 0.0 0.0 0 0 pts/2 Z+ 18:19 0:00 [syz-executor] <defunct>
mingo 41509 0.0 0.0 0 0 pts/2 Z+ 18:19 0:00 [syz-executor] <defunct>
mingo 41513 0.0 0.0 0 0 pts/2 Z+ 18:19 0:00 [syz-executor] <defunct>
mingo 41515 0.0 0.0 0 0 pts/2 Z+ 18:19 0:00 [syz-executor] <defunct>
mingo 41523 0.0 0.0 0 0 pts/2 Z+ 18:19 0:00 [syz-executor] <defunct>
mingo 41601 0.0 0.0 0 0 pts/2 Z+ 18:19 0:00 [syz-executor] <defunct>
mingo 41608 0.0 0.0 0 0 pts/2 Z+ 18:19 0:00 [syz-executor] <defunct>
mingo 41662 0.0 0.0 0 0 pts/2 Z+ 18:19 0:00 [syz-executor] <defunct>
mingo 41764 0.0 0.0 0 0 pts/2 Z+ 18:19 0:00 [syz-executor] <defunct>
mingo 41966 0.0 0.0 0 0 pts/2 Z+ 18:19 0:00 [syz-executor] <defunct>
mingo 42029 0.0 0.0 0 0 pts/2 Z+ 18:19 0:00 [syz-executor] <defunct>
mingo 42084 0.0 0.0 0 0 pts/2 Z+ 18:19 0:00 [syz-executor] <defunct>
mingo 42145 0.0 0.0 0 0 pts/2 Z+ 18:19 0:00 [syz-executor] <defunct>
mingo 42149 0.0 0.0 0 0 pts/2 Z+ 18:19 0:00 [syz-executor] <defunct>
mingo 42166 0.0 0.0 0 0 pts/2 Z+ 18:19 0:00 [syz-executor] <defunct>
mingo 42175 0.0 0.0 0 0 pts/2 Z+ 18:19 0:00 [syz-executor] <defunct>
mingo 57627 2.2 0.0 1860540 44884 pts/2 Sl+ 18:16 0:04 /home/mingo/go/src/github.com/google/syzkaller/workdir/instance-0/syz-fuzzer -executor /home/mingo/go/src/github.com/google/syzkaller/workdir/instance-0/syz-executor -name local-0 -manager 127.0.0.1:33809 -output=none -procs 16 -leak=false -cover=false -nobody=true -v 0
... because they are recycling:
fomalhaut:~> ps aux | grep syz-executor
mingo 57627 1.6 0.0 1942468 44624 pts/2 Sl+ 18:16 0:05 /home/mingo/go/src/github.com/google/syzkaller/workdir/instance-0/syz-fuzzer -executor /home/mingo/go/src/github.com/google/syzkaller/workdir/instance-0/syz-executor -name local-0 -manager 127.0.0.1:33809 -output=none -procs 16 -leak=false -cover=false -nobody=true -v 0
mingo 98448 0.0 0.0 0 0 pts/2 Z+ 18:21 0:00 [syz-executor] <defunct>
mingo 98454 0.0 0.0 0 0 pts/2 Z+ 18:21 0:00 [syz-executor] <defunct>
mingo 98468 0.0 0.0 0 0 pts/2 Z+ 18:21 0:00 [syz-executor] <defunct>
mingo 98472 0.0 0.0 0 0 pts/2 Z+ 18:21 0:00 [syz-executor] <defunct>
mingo 98476 0.0 0.0 0 0 pts/2 Z+ 18:21 0:00 [syz-executor] <defunct>
mingo 98522 0.0 0.0 0 0 pts/2 Z+ 18:21 0:00 [syz-executor] <defunct>
mingo 98525 0.0 0.0 0 0 pts/2 Z+ 18:21 0:00 [syz-executor] <defunct>
mingo 98548 0.0 0.0 0 0 pts/2 Z+ 18:21 0:00 [syz-executor] <defunct>
mingo 98568 0.0 0.0 0 0 pts/2 Z+ 18:21 0:00 [syz-executor] <defunct>
mingo 98596 0.0 0.0 0 0 pts/2 Z+ 18:21 0:00 [syz-executor] <defunct>
mingo 98618 0.0 0.0 0 0 pts/2 Z+ 18:21 0:00 [syz-executor] <defunct>
mingo 98644 0.0 0.0 0 0 pts/2 Z+ 18:21 0:00 [syz-executor] <defunct>
mingo 98695 0.0 0.0 0 0 pts/2 Z+ 18:21 0:00 [syz-executor] <defunct>
mingo 98708 0.0 0.0 0 0 pts/2 Z+ 18:21 0:00 [syz-executor] <defunct>
mingo 98737 0.0 0.0 0 0 pts/2 Z+ 18:21 0:00 [syz-executor] <defunct>
mingo 98756 0.0 0.0 0 0 pts/2 Z+ 18:21 0:00 [syz-executor] <defunct>
I'm not seeing anything happen in 'top' - only a mostly idle system.
> syz-manager/syz-fuzzer only guide the process.
> You can set "procs" param in config to higher value to increase CPU
> utilization. To get more bugs you want to saturate all CPUs to trigger
> more unusual thread interleavings.
So right now it doesn't seem to saturate 16 CPUs - not even close to it.
> If there is a second unfinished thread hanging on a kernel spinlock or
> mutex, then it's definitely bad.
> It also helps to enable CONFIG_RCU_STALL_COMMON=y,
> CONFIG_DEBUG_ATOMIC_SLEEP=y, CONFIG_WQ_WATCHDOG=y and spinlock/mutex
> debugging. These can detect various stalls.
I can just Ctrl-C the manager and it shuts down within a few seconds:
2016/03/08 17:39:25 serving rpc on tcp://127.0.0.1:33809
2016/03/08 17:51:45 local-0: lost connection: exit status 2
2016/03/08 17:51:45 local-0: saving crash 'lost connection' to crash-local-0-1457455905403390570
2016/03/08 18:04:04 local-0: lost connection: exit status 2
2016/03/08 18:04:04 local-0: saving crash 'lost connection' to crash-local-0-1457456644779165131
2016/03/08 18:16:24 local-0: lost connection: exit status 2
2016/03/08 18:16:24 local-0: saving crash 'lost connection' to crash-local-0-1457457384707190124
^C2016/03/08 18:22:53 shutting down...
with nothing hanging around:
fomalhaut:~/go/src/github.com/google/syzkaller> ps aux | grep -i syz
mingo 1374 0.0 0.0 118476 2376 pts/2 S+ 18:23 0:00 grep --color=auto -i syz
and with no kernel messages in dmesg - and with a fully functional system.
I'm running the 16-task load on a 120 CPU system - should I increase it to 120?
Does the code expect to saturate the system?
Thanks,
Ingo
next prev parent reply other threads:[~2016-03-08 17:24 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-03-07 3:50 [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Wang Nan
2016-03-07 3:50 ` [RESEND PATCH 1/5] perf core: Introduce new ioctl options to pause and resume " Wang Nan
2016-03-07 3:50 ` [RESEND PATCH 2/5] perf core: Set event's default overflow_handler Wang Nan
2016-03-07 3:50 ` [RESEND PATCH 3/5] perf core: Prepare writing into ring buffer from end Wang Nan
2016-03-07 3:50 ` [RESEND PATCH 4/5] perf core: Add backward attribute to perf event Wang Nan
2016-03-07 3:50 ` [RESEND PATCH 5/5] perf core: Reduce perf event output overhead by new overflow handler Wang Nan
2016-03-08 13:44 ` [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Peter Zijlstra
2016-03-08 13:49 ` Ingo Molnar
2016-03-08 13:57 ` Peter Zijlstra
2016-03-08 15:29 ` Ingo Molnar
2016-03-08 15:35 ` Dmitry Vyukov
2016-03-08 15:54 ` Ingo Molnar
2016-03-08 16:11 ` Dmitry Vyukov
2016-03-08 16:27 ` Ingo Molnar
2016-03-08 16:29 ` Dmitry Vyukov
2016-03-08 16:32 ` Peter Zijlstra
2016-03-08 16:44 ` Ingo Molnar
2016-03-08 16:48 ` Ingo Molnar
2016-03-08 16:59 ` Dmitry Vyukov
2016-03-08 17:24 ` Ingo Molnar [this message]
2016-03-08 17:27 ` Dmitry Vyukov
2016-03-08 17:37 ` Ingo Molnar
2016-03-08 17:41 ` Dmitry Vyukov
2016-03-08 17:48 ` Ingo Molnar
2016-03-08 17:56 ` Dmitry Vyukov
2016-03-08 17:56 ` Peter Zijlstra
2016-03-08 17:57 ` Ingo Molnar
2016-03-08 18:02 ` Ingo Molnar
2016-03-08 18:22 ` Ingo Molnar
2016-03-08 18:31 ` Ingo Molnar
2016-03-08 17:55 ` Peter Zijlstra
2016-03-08 16:30 ` Peter Zijlstra
2016-03-09 10:53 ` Borislav Petkov
2016-03-09 11:19 ` Dmitry Vyukov
2016-03-08 19:56 ` Jiri Olsa
2016-03-08 20:07 ` Peter Zijlstra
2016-03-08 20:44 ` Jiri Olsa
2016-03-08 21:04 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160308172425.GA3017@gmail.com \
--to=mingo@kernel.org \
--cc=acme@redhat.com \
--cc=ast@kernel.org \
--cc=brendan.d.gregg@gmail.com \
--cc=dvyukov@google.com \
--cc=hekuang@huawei.com \
--cc=jolsa@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lizefan@huawei.com \
--cc=masami.hiramatsu.pt@hitachi.com \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=pi3orama@163.com \
--cc=wangnan0@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.