From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Wed, 25 Mar 2015 23:07:24 +0000 (UTC) From: Philippe Proulx Message-ID: <1110353632.105212.1427324844044.JavaMail.zimbra@efficios.com> In-Reply-To: <2022179888.104000.1427319707820.JavaMail.zimbra@efficios.com> References: <1932682628.103099.1427308743604.JavaMail.zimbra@efficios.com> <551307FE.8060507@ericsson.com> <369395798.103323.1427311320803.JavaMail.zimbra@efficios.com> <55130F34.10806@ericsson.com> <987784607.103695.1427316285128.JavaMail.zimbra@efficios.com> <939295554.103805.1427317243385.JavaMail.zimbra@efficios.com> <2022179888.104000.1427319707820.JavaMail.zimbra@efficios.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [diamon-discuss] Common Trace Format 1.9 planning List-Id: DiaMon diagnostic and monitoring workgroup general discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Mathieu Desnoyers Cc: diamon-discuss@lists.linuxfoundation.org ----- Original Message ----- > From: "Philippe Proulx" > To: "Mathieu Desnoyers" > Cc: "Simon Marchi" , diamon-discuss@lists.linu= xfoundation.org > Sent: Wednesday, 25 March, 2015 5:41:47 PM > Subject: Re: [diamon-discuss] Common Trace Format 1.9 planning >=20 > ----- Original Message ----- > > From: "Mathieu Desnoyers" > > To: "Simon Marchi" , "Philippe Proulx" > > > > Cc: diamon-discuss@lists.linuxfoundation.org > > Sent: Wednesday, 25 March, 2015 5:00:43 PM > > Subject: Re: [diamon-discuss] Common Trace Format 1.9 planning > >=20 > > Philippe might want to chime in. >=20 > Yes, I might. >=20 > Comments below. >=20 > >=20 > > ----- Original Message ----- > > > ----- Original Message ----- > > > > On 15-03-25 03:22 PM, Mathieu Desnoyers wrote: > > > > > > This is great, so v1.9 is a superset of 1.8. > > > > > > > > > > No. A superset would be a completely backward compatible 1.9. > > > > > This is not the case here. We plan to do incompatible changes > > > > > within 1.9, hence the 1.8 and 1.9 parsers needed. Since the > > > > > version is self-described, it should not be an issue to detect > > > > > the CTF version from the trace metadata. > > > >=20 > > > > From what I understand, it means that a reader for 1.9 won't be abl= e to > > > > read > > > > a 1.8 trace, is that right? > > >=20 > > > Yes, we plan to add incompatible grammar changes. This might have to = be > > > versioned as 2.0 then. However, we can have side-by-side implementati= ons > > > of CTF 1.8 and 2.0 readers within a trace reading lib, and therefore > > > distinguish between those, and use the proper CTF reader implementati= on. > > >=20 > > > >=20 > > > > Do the version numbers mean something? If introducing non backward > > > > compatible > > > > changes only bumps the minor version, what could ever bump the majo= r? > > >=20 > > > Good question! In this case, we might be talking about a CTF 2.0 then= , > > > since we are planning non-compatible changes. > > >=20 > > > >=20 > > > > If they don't have one already, this could be an opportunity to giv= e a > > > > meaning > > > > to the numbers. The major could be for breaking backward compatibil= ity, > > > > while > > > > the minor could be for backward-compatible changes. It would mean t= hat > > > > a > > > > reader > > > > for x.y should be able to read any x.z trace, where z <=3D y. In ot= her > > > > words, > > > > the > > > > x.y format would be a superset of x.z (I think?). Much like semver.= org, > > > > but > > > > without the PATCH level. > > >=20 > > > Yes, I think it's a good approach. > > >=20 > > > We have been hesitating between moving from CTF 1.8 to either 1.9 or = 2.0. > > > Here are the upsides/downsides of each approach: > > >=20 > > > * 1.8 to 1.9: > > > + Compatibility: New CTF 1.9 readers would be able to read CTF 1.8 > > > traces, > > > - Incompatibility: CTF 1.8 readers would not be able to read CTF 1.= 9 > > > traces, > > > - Complexity: The CTF 1.9 specification would need to be an exact > > > subset >=20 > I guess you meant "superset" here. >=20 > > > of 1.8, which means a more complex spec, grammar, and > > > implementations, > > >=20 > > > * 1.8 to 2.0: > > > + Compability: Since we're keeping the version headers, a trace rea= der > > > can implement parsers for both CTF 1.8 and 2.0, and read both tra= ce > > > formats without requiring user interaction, > > > - Incompatibility: CTF 1.8 readers would not be able to read CTF 2.= 0 > > > traces, > > > + Simplicity: New CTF 2.0 readers would be simpler, since they woul= d > > > not > > > need to read CTF 1.8 traces, > > >=20 > > > Since the user-visible impacts of bumping from 1.8 to 1.9 or from 1.8= to > > > 2.0 > > > appear to be the same for existing implementations of the spec, I am > > > really > > > tempted to bump to 2.0 at this stage. Already having the version numb= er > > > in > > > the header makes the transition so much easier to manage. > > >=20 > > > Thoughts ? >=20 > File formats have various ways of identifying their versions. >=20 > The "serial" approach is often found (single version). One example is > Microsoft's Bitmap format: > . > Here, the DIB Header Size field indicates the size of the DIB Header, and > depending on this size, a Bitmap file reader knows what fields are availa= ble. > This header size may be considered as a serial version number. As the Bit= map > format evolved over the years, new fields were added, and thus the DIB He= ader > Size field value was increased each time. Old readers may still read newe= r > Bitmap files here: they just skip the unknown fields thanks to the header > size, > and continue from there. The program will miss some information, but shou= ld > still be able to decode the image. >=20 > I think the major.minor approach should follow this backward-compatibilit= y > approach using the minor version: a CTF reader implemented by reading the > CTF 1.8 specification, should be able to decode CTF 1.9 traces, CTF 1.10 > traces, and so on, with no changes. It is expected that this same reader > would not be able to decode CTF 2.0 traces (major bump). After some discussions with Mathieu, here's what we agreed. Let my definition of the minor version (older reader able to read newer minor versions) become the _patch version_. This means: older readers are a= ble to read newer patch versions (same major and minor versions). Let my definition of the major version (complete break, including changing the TSDL grammar) be split in two: * Minor version: changing the TSDL grammar is allowed, as long as the changes form a superset of the previous grammar (the previous grammar is completely included within the new one). This obviously means that a CTF 2.4 reader (which doesn't know CTF 2.5) won't be able to open a CTF 2.5 trace, since the grammar could have changed. However, a CTF 2.5 reader is necessarily able to read CTF 2.4 traces since the grammar of CTF 2.5 is a superset of CTF 2.4's. * Major version: complete compatibility break, possibly including removing/altering parts of the grammar. A CTF 3.x reader is not necessa= rily able to read CTF 2.x traces, since the grammars of CTF 2.x and CTF 3.x could form disjoint sets. This approach makes sure that the major number will not change often, since adding features to the grammar, as long as it's a superset of the previous = one, only bumps the minor version. Of course, our efforts for the initial 2.0.0 version will focus on making sure the new grammar is relaxed enough to reserve room for future improveme= nts that do not need a change in the grammar (patch version bump). Thus a CTF 2.0.0 reader will be able to parse CTF 2.0.1 traces, CTF 2.0.2 traces, and so on, with no changes. For example, custom type attributes could be prefixed (namespaced), so that new specified attributes may be added withou= t changing the grammar, and without interfering with custom (prefixed) attributes, e.g. (hypothetical syntax): integer < size: 2, align: 4, x-my-custom-attribute: 0x1000, x-my-other-custom-attr: "something", attribute: 42 /* <-- May be added in CTF 2.0.1; CTF 2.0.0 readers will ignore it, while CTF 2.0.1+ readers will treat it specially (cannot be a custom attribu= te, because it's not "x-attribute"). */ > So, again, since we plan major alterations of CTF 1.8's grammar, i.e. CTF 1= .9's grammar won't be a superset of the current one, CTF 2.0 is considered the b= est next release version number for the moment. Phil >=20 > This means that, when a new minor version is released, features may only = be > _added_, and added in a way that makes sure older readers supporting olde= r > versions sharing the same major number can still decode the new format as > they > previously did. Just as a well-designed API may add features when bumping > its minor version, but must ensure older applications will work with no > changes using this new version. >=20 > Here's an example: if hypothetical CTF 1.2 says: >=20 > Integers are defined this way: >=20 > integer { > size =3D ; > align =3D ; > > =C2=A0} >=20 > Other optional attributes may be ignored. >=20 > and CTF 1.3 says: >=20 > Integers are defined this way: >=20 > integer { > size =3D ; > align =3D ; > base =3D ; > > } >=20 > Other optional attributes may be ignored. >=20 > Then, a CTF 1.2 reader would be able to read a CTF 1.3 trace, but it > would ignore the "base" attribute (which falls into an "other optional > attribute" as per CTF 1.2) and display all integers in base 10. >=20 > A CTF 1.3 reader would know this "base" field, and treat it specially, > displaying integers with the provided radix. >=20 > The idea is letting some free space in the specification, like this > "", for future minor revisions of the > format. In binary formats, this is usually done with a fixed-offset field > providing the header size, like Bitmap's DIB Header Size field. As > long as the purpose of this fixed-offset field is known in the first mino= r > version of a given major version, future, custom fields may be added > at will without breaking older decoders. For a text-based format like > TSDL, letting some free space means relaxing the grammar so that > unknown blocks, fields or attributes are still parseable, but have no > attached semantics (yet). >=20 > tl;dr: Unless the current version of Babeltrace/TraceCompass will be > able to read the next version of CTF with no change, it should be > CTF 2.0. >=20 > Phil >=20 > > >=20 > > > Thanks! > > >=20 > > > Mathieu > > >=20 > > > -- > > > Mathieu Desnoyers > > > EfficiOS Inc. > > > http://www.efficios.com > > > _______________________________________________ > > > diamon-discuss mailing list > > > diamon-discuss@lists.linuxfoundation.org > > > https://lists.linuxfoundation.org/mailman/listinfo/diamon-discuss > > >=20 > >=20 > > -- > > Mathieu Desnoyers > > EfficiOS Inc. > > http://www.efficios.com > >