From: Alexey Budankov <alexey.budankov@linux.intel.com>
To: Jiri Olsa <jolsa@redhat.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>,
Ingo Molnar <mingo@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Namhyung Kim <namhyung@kernel.org>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Andi Kleen <ak@linux.intel.com>,
linux-kernel <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v3 0/9] perf: enable compression of record mode trace to save storage space
Date: Wed, 27 Feb 2019 17:56:52 +0300 [thread overview]
Message-ID: <f3b53a68-80ed-c817-e1fa-c58c3d06ce97@linux.intel.com> (raw)
In-Reply-To: <20190227142754.GA30149@krava>
On 27.02.2019 17:27, Jiri Olsa wrote:
> On Tue, Feb 26, 2019 at 08:31:38AM +0300, Alexey Budankov wrote:
>>
>> The patch set implements runtime trace compression (-z option) in
>> record mode and trace auto decompression in report and inject modes.
>> Streaming Zstandard (Zstd) API (zstd) is used for compression and
>> decompression of data that come from kernel mmaped data buffers.
>>
>> Usage of implemented -z,--compression_level=n option provides ~3-5x
>> avg. trace file size reduction on variety of tested workloads what
>> saves storage space on larger server systems where trace file size
>> can easily reach several tens or even hundreds of GiBs, especially
>> when profiling with dwarf-based stacks and tracing of context switches.
>> Implemented -f,--mmap-flush option can be used to avoid compressing
>> every single byte of data and increase compression ratio at the same
>> time lowering tool runtime overhead. Default option value is 1 what
>> is equal to the current perf record implementation. The option is
>> independent from -z setting and doesn't vary with compression level:
>>
>> $ tools/perf/perf record -z 1 -e cycles -- matrix.gcc
>> $ tools/perf/perf record --aio=1 -z 1 -e cycles -- matrix.gcc
>> $ tools/perf/perf record -z 1 -f 1024 -e cycles -- matrix.gcc
>> $ tools/perf/perf record --aio=1 -z 1 -f 1024 -e cycles -- matrix.gcc
>>
>> Runtime compression overhead has been measured for serial and AIO
>> trace writing modes when profiling matrix multiplication workload
>> with the following results:
>>
>> -------------------------------------------------------------
>> | SERIAL | AIO-1 |
>> -----------------------------------------------------------------
>> |-z | OVH(x) | ratio(x) size(MiB) | OVH(x) | ratio(x) size(MiB) |
>> |----------------------------------------------------------------
>> | 0 | 1,00 | 1,000 179,424 | 1,00 | 1,000 187,527 |
>> | 1 | 1,04 | 8,427 181,148 | 1,01 | 8,474 188,562 |
>> | 2 | 1,07 | 8,055 186,953 | 1,03 | 7,912 191,773 |
>> | 3 | 1,04 | 8,283 181,908 | 1,03 | 8,220 191,078 |
>> | 5 | 1,09 | 8,101 187,705 | 1,05 | 7,780 190,065 |
>> | 8 | 1,05 | 9,217 179,191 | 1,12 | 6,111 193,024 |
>> -----------------------------------------------------------------
>>
>> OVH = (Execution time with -z N) / (Execution time with -z 0)
>>
>> ratio - compression ratio
>> size - number of bytes that was compressed
>>
>> size ~= trace file x ratio
>>
>> See complete description of measurement conditions and details below.
>>
>> Introduced compression functionality can be disabled or configured from
>> the command line using NO_LIBZSTD and LIBZSTD_DIR defines:
>>
>> $ make -C tools/perf NO_LIBZSTD=1 clean all
>> $ make -C tools/perf LIBZSTD_DIR=/path/to/zstd/sources/ clean all
>>
>> If your system has some version of the zstd package preinstalled then
>> the build system finds and uses it during the build. Auto detection
>> feature status is reported just before compilation starts, as usual.
>> If you still prefer to compile with some version of zstd that is not
>> preinstalled you have capability to refer the compilation to that
>> version using LIBZSTD_DIR define.
>>
>> See 'tools/perf/perf test' run results below.
>>
>> ---
>> Alexey Budankov (9):
>> feature: implement libzstd check, LIBZSTD_DIR and NO_LIBZSTD defines
>> perf record: implement -f,--mmap-flush=<threshold> option
>> perf session: define bytes_transferred and bytes_compressed metrics
>> perf record: implement COMPRESSED event record and its attributes
>> perf mmap: implement dedicated memory buffer for data compression
>> perf util: introduce Zstd based streaming compression API
>> perf record: implement -z,--compression_level=n option and compression
>> perf report: implement record trace decompression
>> perf inject: enable COMPRESSED records decompression
>
> what commit id is this post based on? I can't get it applied:
I think it is that one below:
ca9d246ede355946273271c37bf2e9e9fe533212 (HEAD -> zstd.V3) perf inject: enable COMPRESSED records decompression
4a71d4b79f5c1793e4d28851ff0ab11e1b0b2fd9 perf report: implement record trace decompression
5c2ba4e117cfacc415fbeac8e26484cfb80070b7 perf record: implement -z,--compression_level=n option and compression
9634ec18b3090daf5b0137a9aa7dd87552245a83 perf util: introduce Zstd based streaming compression API
bc692fb6a1a00e005592233a4ced1198e9d0a78d perf mmap: implement dedicated memory buffer for data compression
9ef63f64522f51bdba9e311303dd8bd73ce88f95 perf record: implement COMPRESSED event record and its attributes
8b470b5b6668532a679cecceeca2e56c1c71b8e2 perf session: define bytes_transferred and bytes_compressed metrics
1666675103895646f78b5f30c9bab77dfbfb91a1 perf record: implement -f,--mmap-flush=<threshold> option
3347a6774f85bcb06a04c95c188eb7f8e99811e1 feature: implement libzstd check, LIBZSTD_DIR and NO_LIBZSTD defines
306b805b097d42e66b0cdae19f97e1d1fa60909e (perf/core) Merge branch 'perf/core' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core
=> c25b9ecf016e7824096611d4b8b0c6c271205316 (origin/perf/core) perf script python: add Python3 support to syscall-counts-by-pid.py
c6e562de9f34fd4df2833a5ef4cf59948385e816 perf script python: add Python3 support to syscall-counts.py
b3b4ccaa756984bc81ed7436e55c6fca5d02f00d perf script python: add Python3 support to stat-cpi.py
Thanks,
Alexey
>
> Applying: feature: implement libzstd check, LIBZSTD_DIR and NO_LIBZSTD defines
> Applying: perf record: implement -f,--mmap-flush=<threshold> option
> error: corrupt patch at line 276
> Patch failed at 0002 perf record: implement -f,--mmap-flush=<threshold> option
> Use 'git am --show-current-patch' to see the failed patch
> When you have resolved this problem, run "git am --continue".
> If you prefer to skip this patch, run "git am --skip" instead.
> To restore the original branch and stop patching, run "git am --abort".
>
>
> jirka
>
next prev parent reply other threads:[~2019-02-27 14:57 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-26 5:31 [PATCH v3 0/9] perf: enable compression of record mode trace to save storage space Alexey Budankov
2019-02-26 5:55 ` [PATCH v3 2/9] perf record: implement -f,--mmap-flush=<threshold> option Alexey Budankov
2019-02-26 5:57 ` [PATCH v3 3/9] perf session: define bytes_transferred and bytes_compressed metrics Alexey Budankov
2019-02-26 5:59 ` [PATCH v3 4/9] perf record: implement COMPRESSED event record and its attributes Alexey Budankov
2019-02-26 6:03 ` [PATCH v3 5/9] perf mmap: implement dedicated memory buffer for data compression Alexey Budankov
2019-02-26 6:10 ` [PATCH v3 6/9] perf util: introduce Zstd based streaming compression API Alexey Budankov
2019-02-26 6:20 ` [PATCH v3 7/9] perf record: implement -z,--compression_level=n option and compression Alexey Budankov
2019-02-26 6:26 ` [PATCH v3 8/9] perf report: implement record trace decompression Alexey Budankov
2019-02-26 6:28 ` [PATCH v3 9/9] perf inject: enable COMPRESSED records decompression Alexey Budankov
2019-02-26 7:01 ` [PATCH v3 1/9] feature: implement libzstd check, LIBZSTD_DIR and NO_LIBZSTD defines Alexey Budankov
2019-02-27 14:27 ` [PATCH v3 0/9] perf: enable compression of record mode trace to save storage space Jiri Olsa
2019-02-27 14:56 ` Alexey Budankov [this message]
2019-02-27 14:28 ` Jiri Olsa
2019-02-27 15:40 ` Alexey Budankov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f3b53a68-80ed-c817-e1fa-c58c3d06ce97@linux.intel.com \
--to=alexey.budankov@linux.intel.com \
--cc=acme@kernel.org \
--cc=ak@linux.intel.com \
--cc=alexander.shishkin@linux.intel.com \
--cc=jolsa@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox