From: "Tzvetomir Stoyanov (VMware)" <tz.stoyanov@gmail.com>
To: rostedt@goodmis.org
Cc: linux-trace-devel@vger.kernel.org
Subject: [PATCH v3 24/25] trace-cmd: Compress trace data
Date: Wed, 12 May 2021 09:38:22 +0300 [thread overview]
Message-ID: <20210512063823.324610-25-tz.stoyanov@gmail.com> (raw)
In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com>
When creating a trace.dat file of version 7, compress the trace data.
The data is compressed in chunks. The chunk size is a compile time
parameter, set by default to 1MB.
Signed-off-by: Tzvetomir Stoyanov (VMware) <tz.stoyanov@gmail.com>
---
lib/trace-cmd/trace-output.c | 50 +++++++++++++++++++++++++++++++-----
1 file changed, 43 insertions(+), 7 deletions(-)
diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c
index 08d05744..fab01f56 100644
--- a/lib/trace-cmd/trace-output.c
+++ b/lib/trace-cmd/trace-output.c
@@ -320,6 +320,41 @@ static tsize_t copy_file(struct tracecmd_output *handle,
return size;
}
+#define COMPRESS_CHUNK_SIZE (1*1024*1024)
+static tsize_t copy_file_compress(struct tracecmd_output *handle,
+ const char *file, unsigned long long *write_size)
+{
+ unsigned long long rsize = 0;
+ unsigned long long wsize = 0;
+ tsize_t size;
+ int ret;
+ int fd;
+
+ fd = open(file, O_RDONLY);
+ if (fd < 0) {
+ tracecmd_warning("Can't read '%s'", file);
+ return 0;
+ }
+
+ if (handle->file_version >= 7) {
+ ret = tracecmd_compress_copy_from(handle->compress, fd,
+ COMPRESS_CHUNK_SIZE, &rsize, &wsize);
+ if (ret < 0) {
+ tracecmd_warning("Can't compress '%s'", file);
+ close(fd);
+ return 0;
+ }
+ size = rsize;
+ *write_size = wsize;
+ } else {
+ size = copy_file_fd(handle, fd);
+ *write_size = size;
+ }
+
+ close(fd);
+ return size;
+}
+
/*
* Finds the path to the debugfs/tracing
* Allocates the string and stores it.
@@ -1684,8 +1719,9 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle,
struct data_file_write *data_files = NULL;
off64_t offset;
unsigned long long endian8;
+ unsigned long long read_size;
+ unsigned long long write_size;
char *clock = NULL;
- off64_t check_size;
char *file;
struct stat st;
int ret;
@@ -1743,11 +1779,11 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle,
fprintf(stderr, "CPU%d data recorded at offset=0x%llx\n",
i, (unsigned long long) data_files[i].data_offset);
- check_size = copy_file(handle, cpu_data_files[i]);
- if (check_size != data_files[i].file_size) {
+ read_size = copy_file_compress(handle, cpu_data_files[i], &write_size);
+ if (read_size != data_files[i].file_size) {
errno = EINVAL;
tracecmd_warning("did not match size of %lld to %lld",
- check_size, data_files[i].file_size);
+ read_size, data_files[i].file_size);
goto out_free;
}
/* Write the real CPU data offset inthe file */
@@ -1757,14 +1793,14 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle,
goto out_free;
/* Write the real CPU data size in the file */
offset = lseek64(handle->fd, data_files[i].soffset, SEEK_SET);
- endian8 = convert_endian_8(handle, check_size);
+ endian8 = convert_endian_8(handle, write_size);
if (do_write_check(handle, &endian8, 8))
goto out_free;
- offset = data_files[i].data_offset + check_size;
+ offset = data_files[i].data_offset + write_size;
offset = lseek64(handle->fd, offset, SEEK_SET);
if (!tracecmd_get_quiet(handle))
fprintf(stderr, " %llu bytes in size\n",
- (unsigned long long)check_size);
+ (unsigned long long)write_size);
}
if (lseek64(handle->fd, 0, SEEK_END) == (off64_t)-1)
--
2.31.1
next prev parent reply other threads:[~2021-05-12 6:38 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-05-12 6:37 [PoC PATCH v3 00/25] Add trace file compression Tzvetomir Stoyanov (VMware)
2021-05-12 6:37 ` [PATCH v3 01/25] trace-cmd library: Remove unused private APIs for creating trace files Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 02/25] trace-cmd library: Remove unused API tracecmd_update_option Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 03/25] trace-cmd: Check if file version is supported Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 04/25] trace-cmd library: Add new API to get file version of input handler Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 05/25] trace-cmd library: Select the file version when writing trace file Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 06/25] trace-cmd: Add APIs for library initialization and free Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 07/25] trace-cmd library: Add support for compression algorithms Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 08/25] trace-cmd list: Show supported " Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 09/25] trace-cmd library: Bump the trace file version to 7 Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 10/25] trace-cmd library: Compress part of the trace file Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 11/25] trace-cmd library: Read compressed " Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 12/25] trace-cmd library: Add new API to get compression of input handler Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 13/25] trace-cmd library: Inherit compression algorithm from input file Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 14/25] trace-cmd library: Extend the create file APIs to support different compression Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 15/25] trace-cmd record: Add new parameter --compression Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 16/25] trace-cmd dump: Add support for trace files version 7 Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 17/25] trace-cmd library: Add support for zlib compression library Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 18/25] trace-cmd library: Hide the logic for updating buffer offset Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 19/25] trace-cmd: Move buffers description outside of options Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 20/25] trace-cmd library: Track the offset in the option section in the trace file Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 21/25] trace-cmd library: Add compression of the option section of " Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 22/25] trace-cmd library: Refactor the logic for writing trace data in the file Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` [PATCH v3 23/25] trace-cmd library: Add APIs for read and write compressed data in chunks Tzvetomir Stoyanov (VMware)
2021-05-12 6:38 ` Tzvetomir Stoyanov (VMware) [this message]
2021-05-12 6:38 ` [PATCH v3 25/25] trace-cmd: Read compressed trace data Tzvetomir Stoyanov (VMware)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210512063823.324610-25-tz.stoyanov@gmail.com \
--to=tz.stoyanov@gmail.com \
--cc=linux-trace-devel@vger.kernel.org \
--cc=rostedt@goodmis.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).