[lttng-dev] How to use lttng-live plugin with babeltrace2 python api for live LTTng trace reading?

[lttng-dev] How to use lttng-live plugin with babeltrace2 python api for live LTTng trace reading?

[LI Ruoxiang via lttng-dev]

[-- Attachment #1.1.1: Type: text/plain, Size: 1021 bytes --]

Hi there,

I am currently involved in a project on designing a Python program for LTTng trace data analysis online. The following figure illustrates the program with a live trace data reader using babeltrace2 Python bindings (the yellow box) connected to the LTTng relay daemon. The program will read (such as periodically) the trace data from the relay daemon and then process them while the LLTng keeps tracing. The above “read” and “process” phases repeat in a loop.

[cid:4435d4c9-8a71-4667-bd25-21ef64401df9]

After reading the babeltrace2 documents, examples, and some source code, I found the lttng-live plugin may be an option for reading trace data from LTTng relay daemon. However, I didn't find any examples for using lttng-live plugin with babeltrace2 Python bindings. And I wonder if the Python bindings support the mentioned live LTTng trace reading for my case. Is it possible to receive any examples about the usage of babeltrace2’s live reading, if any?

Thank you.

Best,
Ruoxiang Li


[-- Attachment #1.1.2: Type: text/html, Size: 5130 bytes --]

[-- Attachment #1.2: image.png --]
[-- Type: image/png, Size: 51776 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] How to use lttng-live plugin with babeltrace2 python api for live LTTng trace reading?

[Erica Bugden via lttng-dev]

[-- Attachment #1.1.1: Type: text/plain, Size: 5710 bytes --]

Hello Ruoxiang!

Thank you for your question. It's true that there are no Python bindings examples specific to lttng live (we've provided a brief example below). Unfortunately, the python bindings documentation is currently incomplete, but information about using lttng live can be pieced together using the various documentation links below (see Useful Links section).

Some adjustments typically needed when using lttng live and the Python bindings are:
- When referring to a trace source, use a URL (e.g. net://localhost/host/luna/my-session) rather than a file path (e.g. /path/to/trace)
- When querying, use the source.ctf.lttng-live component class (rather than the file system class: source.ctf.fs) - source.ctf.lttng-live docs https://babeltrace.org/docs/v2.0/man7/babeltrace2-source.ctf.lttng-live.7/

Hope this helps!

Best,
Erica

----

Useful links

- Python bindings docs (Installation, Examples) - https://babeltrace.org/docs/v2.0/python/bt2/index.html
- LTTng live, General information - https://lttng.org/docs/v2.13/#doc-lttng-live (e.g. How to express a live trace source: net://localhost/host/HOSTNAME/my-session)
- source.ctf.lttng-live component - https://babeltrace.org/docs/v2.0/man7/babeltrace2-source.ctf.lttng-live.7/ (C API doc not Python, but can be used to adapt the source.ctf.fs examples by comparing)

----

Quick Example: Trace reading with python bindings and lttng live

      Note: This is a local example where the tracing and reading with babeltrace are happening on the same machine (luna).

1. REQUIREMENTS

Make sure babeltrace is installed with the python plugins and python bindings: https://babeltrace.org/docs/v2.0/python/bt2/installation.html

2. PROCEDURE

 - Start root session daemon: $sudo lttng-sessiond --daemonize
 - Create live session: $lttng create my-session --live
 - Enable events: $lttng enable-event --kernel sched_switch,sched_process_fork
 - Start tracing: $lttng start
 - Run python script: (see below) $python3 lttng-live.py

3. PYTHON SCRIPT

File name: lttng-live.py
File contents:

      import bt2
      import time

      msg_iter = bt2.TraceCollectionMessageIterator('net://localhost/host/luna/my-session') # The hostname (i.e. machine name) is 'luna'
      while True:
          try:
              for msg in msg_iter:
                  if type(msg) is bt2._EventMessageConst:
                      print(msg.event.name)
          except bt2.TryAgain:
              print('Try again. The iterator has no events to provide right now.')

          time.sleep(0.5)

Reading trace data using the bt2.TraceCollectionMessageIterator and lttng-live: When using the message iterator with live, the iterator never ends by default and can be polled for trace data infinitely (hence the while loop in the example). When there is available data, the iterator will return it. However, there will not always be data available to consume. When this is the case, the iterator returns a "Try again" exception which must be caught in order to continue polling.

4. OUTPUT SNIPPET (Example)

      erica@luna:~$ python3 lttng-live-example.py
      Try again. The iterator has no events to provide right now.
      Try again. The iterator has no events to provide right now.
      Try again. The iterator has no events to provide right now.
      Try again. The iterator has no events to provide right now.
      Try again. The iterator has no events to provide right now.
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_process_fork
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      [...]

________________________________
From: lttng-dev <lttng-dev-bounces@lists.lttng.org> on behalf of LI Ruoxiang via lttng-dev <lttng-dev@lists.lttng.org>
Sent: August 3, 2023 12:44 PM
To: lttng-dev@lists.lttng.org <lttng-dev@lists.lttng.org>
Subject: [lttng-dev] How to use lttng-live plugin with babeltrace2 python api for live LTTng trace reading?

Hi there,

I am currently involved in a project on designing a Python program for LTTng trace data analysis online. The following figure illustrates the program with a live trace data reader using babeltrace2 Python bindings (the yellow box) connected to the LTTng relay daemon. The program will read (such as periodically) the trace data from the relay daemon and then process them while the LLTng keeps tracing. The above “read” and “process” phases repeat in a loop.

[cid:4435d4c9-8a71-4667-bd25-21ef64401df9]


After reading the babeltrace2 documents, examples, and some source code, I found the lttng-live plugin may be an option for reading trace data from LTTng relay daemon. However, I didn't find any examples for using lttng-live plugin with babeltrace2 Python bindings. And I wonder if the Python bindings support the mentioned live LTTng trace reading for my case. Is it possible to receive any examples about the usage of babeltrace2’s live reading, if any?



Thank you.



Best,

Ruoxiang Li


[-- Attachment #1.1.2: Type: text/html, Size: 19524 bytes --]

[-- Attachment #1.2: image.png --]
[-- Type: image/png, Size: 51776 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [Ext] Re: How to use lttng-live plugin with babeltrace2 python api for live LTTng trace reading?

[LI Ruoxiang via lttng-dev]

[-- Attachment #1.1.1: Type: text/plain, Size: 6243 bytes --]

Hi Erica,

Thank you for your kind help.

It works for my case.

Best,
Ruoxiang

From: Erica Bugden <ebugden@efficios.com>
Date: Wednesday, August 9, 2023 at 05:29
To: LI Ruoxiang <ruoxiang.li@my.cityu.edu.hk>, lttng-dev@lists.lttng.org <lttng-dev@lists.lttng.org>
Subject: [Ext] Re: How to use lttng-live plugin with babeltrace2 python api for live LTTng trace reading?
CAUTION: External email. Do not reply, click on links or open attachments unless you recognize the sender and know the content is safe.

Hello Ruoxiang!

Thank you for your question. It's true that there are no Python bindings examples specific to lttng live (we've provided a brief example below). Unfortunately, the python bindings documentation is currently incomplete, but information about using lttng live can be pieced together using the various documentation links below (see Useful Links section).

Some adjustments typically needed when using lttng live and the Python bindings are:
- When referring to a trace source, use a URL (e.g. net://localhost/host/luna/my-session) rather than a file path (e.g. /path/to/trace)
- When querying, use the source.ctf.lttng-live component class (rather than the file system class: source.ctf.fs) - source.ctf.lttng-live docs https://babeltrace.org/docs/v2.0/man7/babeltrace2-source.ctf.lttng-live.7/

Hope this helps!

Best,
Erica

----

Useful links

- Python bindings docs (Installation, Examples) - https://babeltrace.org/docs/v2.0/python/bt2/index.html
- LTTng live, General information - https://lttng.org/docs/v2.13/#doc-lttng-live (e.g. How to express a live trace source: net://localhost/host/HOSTNAME/my-session)
- source.ctf.lttng-live component - https://babeltrace.org/docs/v2.0/man7/babeltrace2-source.ctf.lttng-live.7/ (C API doc not Python, but can be used to adapt the source.ctf.fs examples by comparing)

----

Quick Example: Trace reading with python bindings and lttng live

      Note: This is a local example where the tracing and reading with babeltrace are happening on the same machine (luna).

1. REQUIREMENTS

Make sure babeltrace is installed with the python plugins and python bindings: https://babeltrace.org/docs/v2.0/python/bt2/installation.html

2. PROCEDURE

 - Start root session daemon: $sudo lttng-sessiond --daemonize
 - Create live session: $lttng create my-session --live
 - Enable events: $lttng enable-event --kernel sched_switch,sched_process_fork
 - Start tracing: $lttng start
 - Run python script: (see below) $python3 lttng-live.py

3. PYTHON SCRIPT

File name: lttng-live.py
File contents:

      import bt2
      import time

      msg_iter = bt2.TraceCollectionMessageIterator('net://localhost/host/luna/my-session') # The hostname (i.e. machine name) is 'luna'
      while True:
          try:
              for msg in msg_iter:
                  if type(msg) is bt2._EventMessageConst:
                      print(msg.event.name)
          except bt2.TryAgain:
              print('Try again. The iterator has no events to provide right now.')

          time.sleep(0.5)

Reading trace data using the bt2.TraceCollectionMessageIterator and lttng-live: When using the message iterator with live, the iterator never ends by default and can be polled for trace data infinitely (hence the while loop in the example). When there is available data, the iterator will return it. However, there will not always be data available to consume. When this is the case, the iterator returns a "Try again" exception which must be caught in order to continue polling.

4. OUTPUT SNIPPET (Example)

      erica@luna:~$ python3 lttng-live-example.py
      Try again. The iterator has no events to provide right now.
      Try again. The iterator has no events to provide right now.
      Try again. The iterator has no events to provide right now.
      Try again. The iterator has no events to provide right now.
      Try again. The iterator has no events to provide right now.
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_process_fork
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      sched_switch
      [...]

________________________________
From: lttng-dev <lttng-dev-bounces@lists.lttng.org> on behalf of LI Ruoxiang via lttng-dev <lttng-dev@lists.lttng.org>
Sent: August 3, 2023 12:44 PM
To: lttng-dev@lists.lttng.org <lttng-dev@lists.lttng.org>
Subject: [lttng-dev] How to use lttng-live plugin with babeltrace2 python api for live LTTng trace reading?

Hi there,


I am currently involved in a project on designing a Python program for LTTng trace data analysis online. The following figure illustrates the program with a live trace data reader using babeltrace2 Python bindings (the yellow box) connected to the LTTng relay daemon. The program will read (such as periodically) the trace data from the relay daemon and then process them while the LLTng keeps tracing. The above “read” and “process” phases repeat in a loop.


[cid:4435d4c9-8a71-4667-bd25-21ef64401df9]




After reading the babeltrace2 documents, examples, and some source code, I found the lttng-live plugin may be an option for reading trace data from LTTng relay daemon. However, I didn't find any examples for using lttng-live plugin with babeltrace2 Python bindings. And I wonder if the Python bindings support the mentioned live LTTng trace reading for my case. Is it possible to receive any examples about the usage of babeltrace2’s live reading, if any?



Thank you.



Best,

Ruoxiang Li



[-- Attachment #1.1.2: Type: text/html, Size: 26302 bytes --]

[-- Attachment #1.2: image.png --]
[-- Type: image/png, Size: 51776 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [Ext] Re: How to use lttng-live plugin with babeltrace2 python api for live LTTng trace reading?

[Erica Bugden via lttng-dev]

You're welcome Ruoxiang!

On 2023-08-10 22:50, LI Ruoxiang wrote:
> HiErica,
> 
> Thank you for your kind help.
> 
> It works for my case.
> 
> Best,
> 
> Ruoxiang
> 
> *From: *Erica Bugden <ebugden@efficios.com>
> *Date: *Wednesday, August 9, 2023 at 05:29
> *To: *LI Ruoxiang <ruoxiang.li@my.cityu.edu.hk>, 
> lttng-dev@lists.lttng.org <lttng-dev@lists.lttng.org>
> *Subject: *[Ext] Re: How to use lttng-live plugin with babeltrace2 
> python api for live LTTng trace reading?
> 
> *CAUTION: External email. Do not reply, click on links or open 
> attachments unless you recognize the sender and know the content is safe. *
> 
> Hello Ruoxiang!
> 
> Thank you for your question. It's true that there are no Python bindings 
> examples specific to lttng live (we've provided a brief example below). 
> Unfortunately, the python bindings documentation is currently 
> incomplete, but information about using lttng live can be pieced 
> together using the various documentation links below (see Useful Links 
> section).
> 
> Some adjustments typically needed when using lttng live and the Python 
> bindings are:
> 
> - When referring to a trace source, use a URL (e.g. 
> net://localhost/host/luna/my-session) rather than a file path (e.g. 
> /path/to/trace)
> 
> - When querying, use the source.ctf.lttng-live component class (rather 
> than the file system class: source.ctf.fs) - source.ctf.lttng-live docs 
> https://babeltrace.org/docs/v2.0/man7/babeltrace2-source.ctf.lttng-live.7/ <https://babeltrace.org/docs/v2.0/man7/babeltrace2-source.ctf.lttng-live.7/>
> 
> Hope this helps!
> 
> Best,
> 
> Erica
> 
> ----
> 
> Useful links
> 
> - Python bindings docs (Installation, Examples) - 
> https://babeltrace.org/docs/v2.0/python/bt2/index.html 
> <https://babeltrace.org/docs/v2.0/python/bt2/index.html>
> 
> - LTTng live, General information - 
> https://lttng.org/docs/v2.13/#doc-lttng-live 
> <https://lttng.org/docs/v2.13/#doc-lttng-live> (e.g. How to express a 
> live trace source: net://localhost/host/HOSTNAME/my-session)
> 
> - source.ctf.lttng-live component - 
> https://babeltrace.org/docs/v2.0/man7/babeltrace2-source.ctf.lttng-live.7/ <https://babeltrace.org/docs/v2.0/man7/babeltrace2-source.ctf.lttng-live.7/> (C API doc not Python, but can be used to adapt the source.ctf.fs examples by comparing)
> 
> ----
> 
> Quick Example: Trace reading with python bindings and lttng live
> 
>       Note: This is a local example where the tracing and reading with 
> babeltrace are happening on the same machine (luna).
> 
> 1. REQUIREMENTS
> 
> Make sure babeltrace is installed with the python plugins and python 
> bindings: https://babeltrace.org/docs/v2.0/python/bt2/installation.html 
> <https://babeltrace.org/docs/v2.0/python/bt2/installation.html>
> 
> 2. PROCEDURE
> 
>   - Start root session daemon: $sudo lttng-sessiond --daemonize
> 
>   - Create live session: $lttng create my-session --live
> 
>   - Enable events: $lttng enable-event --kernel 
> sched_switch,sched_process_fork
> 
>   - Start tracing: $lttng start
> 
>   - Run python script: (see below) $python3 lttng-live.py
> 
> 3. PYTHON SCRIPT
> 
> File name: lttng-live.py
> 
> File contents:
> 
>       import bt2
> 
>       import time
> 
>       msg_iter = 
> bt2.TraceCollectionMessageIterator('net://localhost/host/luna/my-session') # The hostname (i.e. machine name) is 'luna'
> 
>       while True:
> 
>           try:
> 
>               for msg in msg_iter:
> 
>                   if type(msg) is bt2._EventMessageConst:
> 
>                       print(msg.event.name)
> 
>           except bt2.TryAgain:
> 
>               print('Try again. The iterator has no events to provide 
> right now.')
> 
>           time.sleep(0.5)
> 
> Reading trace data using the bt2.TraceCollectionMessageIterator and 
> lttng-live: When using the message iterator with live, the iterator 
> never ends by default and can be polled for trace data infinitely (hence 
> the while loop in the example). When there is available data, the 
> iterator will return it. However, there will not always be data 
> available to consume. When this is the case, the iterator returns a "Try 
> again" exception which must be caught in order to continue polling.
> 
> 4. OUTPUT SNIPPET (Example)
> 
>       erica@luna:~$ python3 lttng-live-example.py
> 
>       Try again. The iterator has no events to provide right now.
> 
>       Try again. The iterator has no events to provide right now.
> 
>       Try again. The iterator has no events to provide right now.
> 
>       Try again. The iterator has no events to provide right now.
> 
>       Try again. The iterator has no events to provide right now.
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_process_fork
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       [...]
> 
> ------------------------------------------------------------------------
> 
> *From:*lttng-dev <lttng-dev-bounces@lists.lttng.org> on behalf of LI 
> Ruoxiang via lttng-dev <lttng-dev@lists.lttng.org>
> *Sent:* August 3, 2023 12:44 PM
> *To:* lttng-dev@lists.lttng.org <lttng-dev@lists.lttng.org>
> *Subject:* [lttng-dev] How to use lttng-live plugin with babeltrace2 
> python api for live LTTng trace reading?
> 
> Hi there,
> 
> 
> 
> I am currently involved in a project on designing a Python program for 
> LTTng trace data analysis online. The following figure illustrates the 
> program with a live trace data reader using babeltrace2 Python bindings 
> (the yellow box) connected to the LTTng relay daemon. The program will 
> read (such as periodically)the trace data from the relay daemon and then 
> process them while the LLTng keeps tracing. The above “read” and 
> “process” phases repeat in a loop.
> 
> 
> 
> 
> 
> 
> 
> After reading the babeltrace2 documents, examples, and some source code, 
> I found the lttng-live plugin may be an option for reading trace data 
> from LTTng relay daemon. However, I didn't find any examples for using 
> lttng-live plugin with babeltrace2 Python bindings. And I wonder if the 
> Python bindings support the mentioned live LTTng trace reading for my 
> case. Is it possible to receive any examples about the usage of 
> babeltrace2’s live reading, if any?
> 
> Thank you.
> 
> Best,
> 
> Ruoxiang Li
> 
> 
> 
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev