From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS, UNWANTED_LANGUAGE_BODY,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C030C43381 for ; Fri, 1 Mar 2019 15:52:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3240620850 for ; Fri, 1 Mar 2019 15:52:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388664AbfCAPwP (ORCPT ); Fri, 1 Mar 2019 10:52:15 -0500 Received: from mga01.intel.com ([192.55.52.88]:65431 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728172AbfCAPwO (ORCPT ); Fri, 1 Mar 2019 10:52:14 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Mar 2019 07:52:14 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,428,1544515200"; d="scan'208";a="303605290" Received: from linux.intel.com ([10.54.29.200]) by orsmga005.jf.intel.com with ESMTP; 01 Mar 2019 07:52:14 -0800 Received: from [10.252.14.197] (abudanko-mobl.ccr.corp.intel.com [10.252.14.197]) by linux.intel.com (Postfix) with ESMTP id 949495805B4; Fri, 1 Mar 2019 07:52:11 -0800 (PST) Subject: [PATCH v5 06/10] perf util: introduce Zstd based streaming compression API From: Alexey Budankov To: Arnaldo Carvalho de Melo Cc: Jiri Olsa , Namhyung Kim , Alexander Shishkin , Peter Zijlstra , Ingo Molnar , Andi Kleen , linux-kernel References: <4d1b11a4-77ed-d9af-ed22-875fc17b6050@linux.intel.com> Organization: Intel Corp. Message-ID: Date: Fri, 1 Mar 2019 18:52:10 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 MIME-Version: 1.0 In-Reply-To: <4d1b11a4-77ed-d9af-ed22-875fc17b6050@linux.intel.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Implemented functions are based on Zstd streaming compression API. The functions are used in runtime to compress data that come from mmaped kernel buffer data and then stored into a trace. Signed-off-by: Alexey Budankov --- tools/perf/util/Build | 2 + tools/perf/util/compress.h | 18 ++++++++ tools/perf/util/zstd.c | 95 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 115 insertions(+) create mode 100644 tools/perf/util/zstd.c diff --git a/tools/perf/util/Build b/tools/perf/util/Build index 8dd3102301ea..920ee8bebd83 100644 --- a/tools/perf/util/Build +++ b/tools/perf/util/Build @@ -145,6 +145,8 @@ perf-y += scripting-engines/ perf-$(CONFIG_ZLIB) += zlib.o perf-$(CONFIG_LZMA) += lzma.o +perf-y += zstd.o + perf-y += demangle-java.o perf-y += demangle-rust.o diff --git a/tools/perf/util/compress.h b/tools/perf/util/compress.h index 892e92e7e7fc..e0987616db94 100644 --- a/tools/perf/util/compress.h +++ b/tools/perf/util/compress.h @@ -2,6 +2,11 @@ #ifndef PERF_COMPRESS_H #define PERF_COMPRESS_H +#include +#ifdef HAVE_ZSTD_SUPPORT +#include +#endif + #ifdef HAVE_ZLIB_SUPPORT int gzip_decompress_to_file(const char *input, int output_fd); bool gzip_is_compressed(const char *input); @@ -12,4 +17,17 @@ int lzma_decompress_to_file(const char *input, int output_fd); bool lzma_is_compressed(const char *input); #endif +struct zstd_data { +#ifdef HAVE_ZSTD_SUPPORT + ZSTD_CStream *cstream; +#endif +}; + +int zstd_init(struct zstd_data *data, int level); +int zstd_fini(struct zstd_data *data); + +size_t zstd_compress_stream_to_records(struct zstd_data *data, + void *dst, size_t dst_size, void *src, size_t src_size, size_t max_record_size, + size_t process_header(void *record, size_t increment)); + #endif /* PERF_COMPRESS_H */ diff --git a/tools/perf/util/zstd.c b/tools/perf/util/zstd.c new file mode 100644 index 000000000000..e5d44d0f6b5d --- /dev/null +++ b/tools/perf/util/zstd.c @@ -0,0 +1,95 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include + +#include "util/compress.h" +#include "util/debug.h" + +#ifdef HAVE_ZSTD_SUPPORT + +int zstd_init(struct zstd_data *data, int level) +{ + size_t ret; + + data->cstream = ZSTD_createCStream(); + if (data->cstream == NULL) { + pr_err("Couldn't create compression stream.\n"); + return -1; + } + + ret = ZSTD_initCStream(data->cstream, level); + if (ZSTD_isError(ret)) { + pr_err("Failed to initialize compression stream: %s\n", ZSTD_getErrorName(ret)); + return -1; + } + + return 0; +} + +int zstd_fini(struct zstd_data *data) +{ + if (data->cstream) { + ZSTD_freeCStream(data->cstream); + data->cstream = NULL; + } + + return 0; +} + +size_t zstd_compress_stream_to_records(struct zstd_data *data, + void *dst, size_t dst_size, void *src, size_t src_size, size_t max_record_size, + size_t process_header(void *record, size_t increment)) +{ + size_t ret, size, compressed = 0; + ZSTD_inBuffer input = { src, src_size, 0 }; + ZSTD_outBuffer output; + void *record; + + while (input.pos < input.size) { + record = dst; + size = process_header(record, 0); + compressed += size; + dst += size; + dst_size -= size; + output = (ZSTD_outBuffer){ dst, (dst_size > max_record_size) ? + max_record_size : dst_size, 0 }; + ret = ZSTD_compressStream(data->cstream, &output, &input); + ZSTD_flushStream(data->cstream, &output); + if (ZSTD_isError(ret)) { + pr_err("failed to compress %ld bytes: %s\n", + (long)src_size, ZSTD_getErrorName(ret)); + memcpy(dst, src, src_size); + return src_size; + } + size = output.pos; + size = process_header(record, size); + compressed += size; + dst += size; + dst_size -= size; + } + + return compressed; +} + +#else /* !HAVE_ZSTD_SUPPORT */ + +int zstd_init(struct zstd_data *data __maybe_unused, int level __maybe_unused) +{ + return 0; +} + +int zstd_fini(struct zstd_data *data __maybe_unused) +{ + return 0; +} + +size_t zstd_compress_stream_to_records(struct zstd_data *data __maybe_unused, + void *dst __maybe_unused, size_t dst_size __maybe_unused, + void *src __maybe_unused, size_t src_size __maybe_unused, + size_t max_record_size __maybe_unused, + size_t process_header(void *record, size_t increment) __maybe_unused) +{ + return 0; +} + +#endif -- 2.20.1