From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Tue, 25 Oct 2016 18:27:35 +0000 (UTC) From: Philippe Proulx Message-ID: <1453850286.4431.1477420055622.JavaMail.zimbra@efficios.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: [diamon-discuss] CTF2-FS-1.0: Layout of a CTF 2 trace stored on a file system List-Id: DiaMon diagnostic and monitoring workgroup general discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: diamon-discuss Cc: lttng-dev , tracecompass-dev , etienne bergeron , francois doray HTML version: http://diamon.org/ctf/files/CTF2-FS-1.0.html Philippe Proulx EfficiOS Inc. http://www.efficios.com/ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - =3D CTF2-FS-1.0: Layout of a CTF{nbsp}2 trace stored on a file system Philippe Proulx v1.0, 21 October 2016 :toc: :toclevels: 5 :docid: did:CTF2-FS-1.0 This document defines a standard layout of files for supporting a CTF{nbsp}2 trace (see did:CTF2-SPEC-2.0) on a https://en.wikipedia.org/wiki/File_system[file system] with file name and directory support. The purpose of this standard is to formalize the interoperability of producers writing CTF{nbsp}2 traces on a file system with consumers reading those traces (see <>). The layout defined in this document is compatible with the layout of http://lttng.org/[LTTng]{nbsp}2 traces on a file system, and with the layou= t supported by the http://diamon.org/babeltrace/[Babeltrace] and http://tracecompass.org/[Trace Compass] consumers. .RFC 2119 NOTE: The key words _must_, _must not_, _required_, _shall_, _shall not_, _should_, _should not_, _recommended_, _may_, and _optional_ in this document, when emphasized, are to be interpreted as described in https://www.ietf.org/rfc/rfc2119.txt[RFC 2119]. [[compliance]] =3D=3D Compliance A CTF{nbsp}2 producer is said to _apply {docid}_ if it can produce a CTF{nbsp}2 trace and store it on a file system as defined by this document. A CTF{nbsp}2 consumer is said to _support {docid}_ if it can read the streams of a CTF{nbsp}2 trace stored on a file system as defined by this document. =3D=3D Trace The producer _must_ write the metadata stream and all the data stream packets of a trace to regular files in a dedicated **directory**. In the present text, this dedicated directory is called the _trace directory_. The producer _must not_ create regular files in the trace directory that do not contain the metadata stream or data stream packets, unless the names of those files start with a full stop character (`.`, U+002e). The producer _may_ create subdirectories in the trace directory. The consumer _must_ ignore subdirectories in the trace directory. There are no constraints on the name of the trace directory. =3D=3D=3D Metadata stream The metadata stream _must_ be serialized to a single regular file named `metadata`. The file's content can be either raw, in which case it contains the complete UTF-8 JSON metadata array, or it can be a packetized metadata stream as described in did:CTF2-PMETA-1.0. =3D=3D=3D Data streams If the data stream packet context field type of a given data stream class does not contain a field tagged with the `data-stream-id` tag, then each data stream instantiated by this data stream class _must_ be written to exactly one regular file. There is no constraints on the name of this file, except that it _must not_ be `metadata`. If the data stream packet context field type of a given data stream class does not contain a field tagged with the `data-stream-id` tag, then the individual packets of the data streams referring to this data stream class can be written to one or more files within the trace directory, except to the files described in the previous paragraph. There is no constraints on the name of those files, except that they _must not_ be `metadata`. Within any data stream file, the packets of a given data stream must be written in the order they are produced. =3D=3D Examples Here are a few examples of trace directories following the standard layout defined by this document. .Four data streams (no field tagged with `data-stream-id`) =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D my-trace/ =E2=94=A3=E2=94=81 metadata (contains the raw metadata array) =E2=94=A3=E2=94=81 streamA (contains all the packets of data str= eam 0) =E2=94=A3=E2=94=81 streamB (contains all the packets of data str= eam 1) =E2=94=A3=E2=94=81 streamC (contains all the packets of data str= eam 2) =E2=94=97=E2=94=81 streamD (contains all the packets of data str= eam 3) =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D .One data stream (field tagged with `data-stream-id`) =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D my-trace/ =E2=94=A3=E2=94=81 metadata (contains the packetized metadata arr= ay) =E2=94=A3=E2=94=81 streamA (contains some packets of data stream= 0) =E2=94=A3=E2=94=81 streamB (contains some packets of data stream= 0) =E2=94=A3=E2=94=81 streamC (contains some packets of data stream= 0) =E2=94=97=E2=94=81 streamD (contains some packets of data stream= 0) =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D .Four data streams (field tagged with `data-stream-id`) =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D my-trace/ =E2=94=A3=E2=94=81 metadata (contains the raw metadata array) =E2=94=A3=E2=94=81 streamA0 (contains some packets of data stream= 0) =E2=94=A3=E2=94=81 streamA1 (contains some packets of data stream= 0) =E2=94=A3=E2=94=81 streamB0 (contains some packets of data stream= 1) =E2=94=A3=E2=94=81 streamB1 (contains some packets of data stream= 1) =E2=94=A3=E2=94=81 streamB2 (contains some packets of data stream= 1) =E2=94=A3=E2=94=81 streamC0 (contains some packets of data stream= 2) =E2=94=A3=E2=94=81 streamC1 (contains some packets of data stream= 2) =E2=94=A3=E2=94=81 streamD0 (contains some packets of data stream= 3) =E2=94=A3=E2=94=81 streamD1 (contains some packets of data stream= 3) =E2=94=A3=E2=94=81 streamD2 (contains some packets of data stream= 3) =E2=94=97=E2=94=81 streamD3 (contains some packets of data stream= 3) =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D .Two data streams with a field tagged with `data-stream-id`, two data strea= ms without a field tagged with `data-stream-id` =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D my-trace/ =E2=94=A3=E2=94=81 metadata (contains the packetized metadata arr= ay) =E2=94=A3=E2=94=81 streamAa (contains some packets of data stream= 0) =E2=94=A3=E2=94=81 streamAb (contains some packets of data stream= 0) =E2=94=A3=E2=94=81 streamBa (contains some packets of data stream= 1) =E2=94=A3=E2=94=81 streamBb (contains some packets of data stream= 1) =E2=94=A3=E2=94=81 streamC (contains all the packets of data str= eam 2) =E2=94=97=E2=94=81 streamD (contains all the packets of data str= eam 3) =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D .Two data streams with a field tagged with `data-stream-id` =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D my-trace/ =E2=94=A3=E2=94=81 metadata (contains the raw metadata array) =E2=94=A3=E2=94=81 streamW (contains some packets of data stream= 0 and of data stream 1) =E2=94=A3=E2=94=81 streamX (contains some packets of data stream= 0) =E2=94=A3=E2=94=81 streamY (contains some packets of data stream= 1) =E2=94=97=E2=94=81 streamZ (contains some packets of data stream= 0 and of data stream 1) =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D