[RELEASE] LTTng 2.13.0-rc2 - Nordicité - Linux kernel and user-space tracer

Jérémie Galarneau
2021-05-15T05:26:40Z

Hi everyone,

Today marks the release of the second LTTng 2.13 - Nordicité release candidate.

A prettyfied version of this annoucement is available here:
https://github.com/lttng/lttng-tools/releases/tag/v2.13.0-rc2

If you were already testing the first release candidate of this release, please
note that LTTng-UST and LTTng-tools must be updated to "rc2" in locked-step
due to ABI changes. Also note that probe providers must be rebuilt against
LTTng-UST 2.13.0-rc2.

Have a look at the first release candidate's release notes for an
overview of the features introduced in LTTng 2.13.

https://github.com/lttng/lttng-tools/releases/tag/v2.13.0-rc1

What's new in LTTng-UST (Linux application tracer) 2.13.0-rc2?
---

- Allow explicit tracepoint instance provider name. Allow re-using a tracepoint
class from tracepoint instances in other providers.

This is a localized API-breaking change introduced very early in the
2.13-rc cycle. The LTTNG_UST_TRACEPOINT_EVENT_INSTANCE macro newly introduced
in LTTng-UST 2.13-rc1 now takes an extra "template_provider" argument.

This changes the ABI exposed by the probe providers, so bump the probe
provider major version, and make newer LTTng-UST reject old incompatible
major.

Probe providers built against LTTng-UST 2.13-rc1 must be rebuilt against
LTTng-UST 2.13-rc2 to be traced.

- Add probe descriptor field to enumeration and event class

Also breaks the ABI exposed by probe providers (provider major version already
bumped for rc2). This enables additional validation of probe provider
compatibility when using event class and enumerations across providers.

- Validate provider version for event class.

- Validate provider version for event enumeration field types.

- Introduce api0 and api1 compilation tests. This tests whether the API 0
compatibility API works. See ust-api-compat.h for details.

- Fix Java (JUL and log4j) application context segmentation fault and leaks.

- liblttng-ust-ctl: implement SIGBUS handling:

This fixes a long-standing DoS where an unprivileged application could cause
a SIGBUS in the consumer daemon by truncating the shared memory files it
receives from the session daemon. This requires that LTTng-tools be upgraded
to 2.13-rc2 in locked-step with LTTng-UST 2.13-rc2.

- Restore communication protocol backward compatibility with LTTng-UST 2.12

- liblttng-ust-ctl: keep using lttng-ust-sock-8 and lttng-ust-wait-8 filename

LTTng-UST 2.13-rc1 erroneously changed the file names of those socket and
pipe files from [...]-8 to [...]-9, even though we intended to keep protocol
compatibility between LTTng-UST 2.12 and LTTng-tools/ust 2.13. This changes
reverts back to the prior filenames to make sure LTTng-UST 2.12 applications
can be traced by a 2.13 LTTng-tools using 2.13 LTTng-UST.

- Fix: add missing fields in struct lttng_ust_abi_channel_config

Three fields were removed by mistake from this structure which is ABI
between the consumer daemon and applications traced with LTTng-UST. Add them
back to allow tracing 2.12 LTTng-UST applications with a 2.13 LTTng-tools
using 2.13 LTTng-UST.

- Fix LTTng-UST communication protocol variable-length data handling

When mixing older (e.g. 2.12) and newer versions of LTTng-UST talking to each
other between application and LTTng-tools, situations where commands are
unknown lead to out-of-sync communication protocol.

Change the protocol for commands new to LTTng-UST 2.13 so they acknowledge the
fixed-size commands before the variable-sized data is sent, so the
communication protocol is kept in sync even when unknown command error
happens. This also happens for more ancient commands released in prior
versions for which we cannot change the protocol. Shutdown the socket from the
session daemon side when we detect such errors, because there is no point in
trying to communicate further on an out-of-sync socket.

- Detect mixed liblttng-ust and liblttng-ust-tracepoint .so.0 and .so.1 libraries

Detect and report with a new "critical" logging message whether .so.0 and
.so.1 tracer libraries are loaded within a single process, which is an
unsupported use-case. Note that the LTTNG_UST_DEBUG environment variable must
be set to allow users to observe those critical messages.

We also introduce a new LTTNG_UST_ABORT_ON_CRITICAL environment variable to
allow the tracer to abort() the process whenever a critical logging statement
is encountered. This is meant to facilitate discovering problematic
deployments where an application indirectly loads different soname major
numbers of LTTng-UST through its instrumented shared libraries.

See the ChangeLog for the full list of changes:
https://github.com/lttng/lttng-ust/blob/13861e2d/ChangeLog

What's new in LTTng-modules (Linux kernel tracer) 2.13.0-rc2?
---

- There are refactoring commits that were introduced after 2.13-rc1 to align the
code of LTTng-modules with the refactoring that was done in LTTng-UST prior to
2.13-rc1. Given that LTTng-modules does not expose its kernel APIs to external
users, it was possible to do this refactoring even after rc1.

- Compiling the system call tracing code has typically been a resource-heavy
endeavour both in terms of memory usage (~1G memory required) and CPU time. It
has been split into smaller compile units to allow parallel compilation and
lessen the memory requirements on the compiler.

- Bitwise enumerations are now disabled by default. Only compiled in with make
CONFIG_LTTNG_EXPERIMENTAL_BITWISE_ENUM=y.

- Fix: increment buffer offset when failing to copy from user-space.

- Implement ring buffer Pascal string copy: allows taking C-strings as inputs
and record them to ctf_array_text and ctf_sequence_text fields (which are
Pascal strings) without causing out-of-bound reads of the C-strings of those
are smaller than the size reserved for the pascal strings. This mirrors the
behaviour introduced in LTTng-UST 2.13-rc1.

- Support for 5.12 Linux kernels,

- Support recent stable kernel branches 4.14, 4.19, 5.4,

- Support for newer Ubuntu 4.15, 5.4, and RHEL 8.2/8.3 kernels,

See the ChangeLog for the full list of changes:
https://github.com/lttng/lttng-modules/blob/f982b51a/ChangeLog

What's new in LTTng-tools 2.13.0-rc2?
---

- The new features introduced in the 2.13 release are now fully documented as
part of the installed man pages. A number of improvements were also made to
the existing documentation such as adding examples for the lttng-add-context
command.

- lttng client messages referring to "event rule hit" conditions have been
changed to use the correct "event-rule matches" condition name.

- An example was added under doc/examples/trigger-condition-event-matches. It
contains an application instrumented with LTTng-UST, a script that adds
triggers to notify external clients whenever the application's events occur,
and a notification client which consumes the resulting notification along with
any captured payload.

See https://github.com/lttng/lttng-tools/tree/c3195dc0/doc/examples/trigger-condition-event-matches#readme

- A bug resulting in the production of erroneous path hierarchies when a
snapshot is streamed to a relay daemon was fixed. This bug resulted in
duplicated domain names being present in the final path of a snapshot
("ust/ust" or "kernel/kernel") which could cause some automation scripts to
fail.

- A bug that caused some trace data to be invisible to live viewers has been
fixed.

See https://lists.lttng.org/pipermail/lttng-dev/2021-May/029953.html

- A number of bugs that could result, under rare circumstances, in a hang when
using the lttng-clear command against a session that contains kernel channels
was fixed.

- Miscellaneous stability improvements.

See the ChangeLog for the full list of changes:
https://github.com/lttng/lttng-tools/blob/4be5babe/ChangeLog

Links
---