Have live session and rotation together

Have live session and rotation together

[Sergei Dyshel via lttng-dev]

Hi all,

When creating live session (lttng create --live) I can not enable rotation:

# lttng enable-rotation --size=10M
Error: Failed to enable size-based rotation schedule on session BX

In LTTNG 2.12 rotation is supported for network streaming mode  but it
was not yet supported in 2.11.

Is there architectural reasons why we can't have live session with rotation?
Or is it just not implemented yet? In that case in what version is it
expected to appear?
-- 
Regards,
Sergei Dyshel

Re: Have live session and rotation together

[Mathieu Desnoyers via lttng-dev]

----- On Apr 20, 2020, at 4:17 PM, lttng-dev lttng-dev@lists.lttng.org wrote:

> Hi all,
> 
> When creating live session (lttng create --live) I can not enable rotation:
> 
> # lttng enable-rotation --size=10M
> Error: Failed to enable size-based rotation schedule on session BX

This is indeed the expected behavior.

> 
> In LTTNG 2.12 rotation is supported for network streaming mode  but it
> was not yet supported in 2.11.

I do not think this statement is factually correct. I recall that session
rotation was available for sessions being streamed over the network. What
makes you think otherwise ?

> 
> Is there architectural reasons why we can't have live session with rotation?
> Or is it just not implemented yet? In that case in what version is it
> expected to appear?

It is not impossible design-wise, but would make the implementation much more
complex and require a very significant testing effort.

Considering that there are few compelling use-cases for using both
features together, and no customer have expressed interest in this,
it is not part of our roadmap.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Re: Have live session and rotation together

[Sergei Dyshel via lttng-dev]

> I do not think this statement is factually correct. I recall that session
> rotation was available for sessions being streamed over the network. What
> makes you think otherwise ?

I think I tried to enable rotation for network streaming session and
failed but I don't have
2.11 version currently to reproduce this.

> Considering that there are few compelling use-cases for using both
> features together, and no customer have expressed interest in this,
> it is not part of our roadmap.

Here is my case: I'm using LTTNG for continuous tracing during the
whole system timeline,
generating pretty big volume of traces.

I'm using live and session and Babeltrace2 plugin which converts
processes these traces,
converts them to text and stores them
in some text file to which I apply "logrotate" so that old traces are
archived when needed.

However, as I see, "lttng-relayd" also writes binary traces in CTF
from to some directory, never discarding old traces. So assuming
system long runtime this directory is going to eat all the free space
sometime?! Rotation would allow me to discard old archives just as
"logrotate" does. How else can I discard old CTF files?

-- 
Regards,
Sergei Dyshel

On Tue, Apr 21, 2020 at 5:23 PM Mathieu Desnoyers
<mathieu.desnoyers@efficios.com> wrote:
>
> ----- On Apr 20, 2020, at 4:17 PM, lttng-dev lttng-dev@lists.lttng.org wrote:
>
> > Hi all,
> >
> > When creating live session (lttng create --live) I can not enable rotation:
> >
> > # lttng enable-rotation --size=10M
> > Error: Failed to enable size-based rotation schedule on session BX
>
> This is indeed the expected behavior.
>
> >
> > In LTTNG 2.12 rotation is supported for network streaming mode  but it
> > was not yet supported in 2.11.
>
> I do not think this statement is factually correct. I recall that session
> rotation was available for sessions being streamed over the network. What
> makes you think otherwise ?
>
> >
> > Is there architectural reasons why we can't have live session with rotation?
> > Or is it just not implemented yet? In that case in what version is it
> > expected to appear?
>
> It is not impossible design-wise, but would make the implementation much more
> complex and require a very significant testing effort.
>
> Considering that there are few compelling use-cases for using both
> features together, and no customer have expressed interest in this,
> it is not part of our roadmap.
>
> Thanks,
>
> Mathieu
>
> --
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com

Re: Have live session and rotation together

[Mathieu Desnoyers via lttng-dev]

----- On Apr 21, 2020, at 5:37 PM, Sergei Dyshel qyron.private@gmail.com wrote:

[...]
> 
>> Considering that there are few compelling use-cases for using both
>> features together, and no customer have expressed interest in this,
>> it is not part of our roadmap.
> 
> Here is my case: I'm using LTTNG for continuous tracing during the
> whole system timeline,
> generating pretty big volume of traces.
> 
> I'm using live and session and Babeltrace2 plugin which converts
> processes these traces,
> converts them to text and stores them
> in some text file to which I apply "logrotate" so that old traces are
> archived when needed.
> 
> However, as I see, "lttng-relayd" also writes binary traces in CTF
> from to some directory, never discarding old traces. So assuming
> system long runtime this directory is going to eat all the free space
> sometime?! Rotation would allow me to discard old archives just as
> "logrotate" does. How else can I discard old CTF files?

The feature you need is called "tracefile rotation", with the channel
attributes "tracefile count" and "tracefile size". It was implemented at
the same time as the live mode for exactly the purpose you describe.

See https://lttng.org/docs/#doc-tracefile-rotation for more information.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Re: Have live session and rotation together

[Sergei Dyshel via lttng-dev]

Thanks! I really missed this feature somehow.

However I see that passing 0 as "tracefile size" will remove size
limitation. Is it possible to disable CTF trace writing altogether?
-- 
Regards,
Sergei Dyshel

On Wed, Apr 22, 2020 at 2:15 AM Mathieu Desnoyers
<mathieu.desnoyers@efficios.com> wrote:
>
> ----- On Apr 21, 2020, at 5:37 PM, Sergei Dyshel qyron.private@gmail.com wrote:
>
> [...]
> >
> >> Considering that there are few compelling use-cases for using both
> >> features together, and no customer have expressed interest in this,
> >> it is not part of our roadmap.
> >
> > Here is my case: I'm using LTTNG for continuous tracing during the
> > whole system timeline,
> > generating pretty big volume of traces.
> >
> > I'm using live and session and Babeltrace2 plugin which converts
> > processes these traces,
> > converts them to text and stores them
> > in some text file to which I apply "logrotate" so that old traces are
> > archived when needed.
> >
> > However, as I see, "lttng-relayd" also writes binary traces in CTF
> > from to some directory, never discarding old traces. So assuming
> > system long runtime this directory is going to eat all the free space
> > sometime?! Rotation would allow me to discard old archives just as
> > "logrotate" does. How else can I discard old CTF files?
>
> The feature you need is called "tracefile rotation", with the channel
> attributes "tracefile count" and "tracefile size". It was implemented at
> the same time as the live mode for exactly the purpose you describe.
>
> See https://lttng.org/docs/#doc-tracefile-rotation for more information.
>
> Thanks,
>
> Mathieu
>
> --
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com

Re: Have live session and rotation together

[Mathieu Desnoyers via lttng-dev]

----- On Apr 22, 2020, at 2:18 PM, Sergei Dyshel qyron.private@gmail.com wrote:

> Thanks! I really missed this feature somehow.
> 
> However I see that passing 0 as "tracefile size" will remove size
> limitation. Is it possible to disable CTF trace writing altogether?

No, because the relay daemon uses the on-disk files as intermediate
storage between the receiving end and the part interacting with live
viewers.

Note that https://bugs.lttng.org/issues/1182 tracks a known issue
where the relay daemon mishandles tracefile-count == 0 and tracefile-size != 0.
It has not been a priority to fix this issue so far.

Thanks,

Mathieu


> --
> Regards,
> Sergei Dyshel
> 
> On Wed, Apr 22, 2020 at 2:15 AM Mathieu Desnoyers
> <mathieu.desnoyers@efficios.com> wrote:
>>
>> ----- On Apr 21, 2020, at 5:37 PM, Sergei Dyshel qyron.private@gmail.com wrote:
>>
>> [...]
>> >
>> >> Considering that there are few compelling use-cases for using both
>> >> features together, and no customer have expressed interest in this,
>> >> it is not part of our roadmap.
>> >
>> > Here is my case: I'm using LTTNG for continuous tracing during the
>> > whole system timeline,
>> > generating pretty big volume of traces.
>> >
>> > I'm using live and session and Babeltrace2 plugin which converts
>> > processes these traces,
>> > converts them to text and stores them
>> > in some text file to which I apply "logrotate" so that old traces are
>> > archived when needed.
>> >
>> > However, as I see, "lttng-relayd" also writes binary traces in CTF
>> > from to some directory, never discarding old traces. So assuming
>> > system long runtime this directory is going to eat all the free space
>> > sometime?! Rotation would allow me to discard old archives just as
>> > "logrotate" does. How else can I discard old CTF files?
>>
>> The feature you need is called "tracefile rotation", with the channel
>> attributes "tracefile count" and "tracefile size". It was implemented at
>> the same time as the live mode for exactly the purpose you describe.
>>
>> See https://lttng.org/docs/#doc-tracefile-rotation for more information.
>>
>> Thanks,
>>
>> Mathieu
>>
>> --
>> Mathieu Desnoyers
>> EfficiOS Inc.
> > http://www.efficios.com

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com