[diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016, at 11h EDT (15h UTC)

[diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016, at 11h EDT (15h UTC)

[Mathieu Desnoyers]

Hi,

Following the blog post published two weeks ago [1], we would like to
propose organizing a phone meeting with all interested members of this
workgroup on February 9th to gather feedback and ideas for improvement
on the subject of measuring and detecting high response time.

At EfficiOS, we have developed a kernel module for monitoring at run-time
the delay between the moment the kernel starts processing an interrupt
(do_IRQ) and the moment the target task gets scheduled in or has finished
processing the data.

When a high latency is detected, it emits a tracepoint event and can
wakeup a user-space script to take arbitrary actions as soon as
possible.

The main intent is to provide an entry point in a kernel trace. After
that, everyone has their own methodology to process the trace. The blog
post illustrates what we can do with LTTng as an example but the
detection and triggers are not coupled with any tracer.

The proposed agenda is a discussion around these points:
- presentation of the scope of the problem
- limitation of the current tools
- overview of the latency_tracker module applied for this use-case
-- current state
-- use-cases
-- future plans
- from the audience: comments, ideas, other approaches, etc.

If you have other points you would like to discuss around this subject,
please let me know and I will add them.

Also, if you wish to attend but can't make it at the proposed date and
time, let us know.

The details for the conference call will be sent soon.

Thanks,

Julien & Mathieu

[1] https://lttng.org/blog/2016/01/06/monitoring-realtime-latencies/

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Re: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016, at 11h EDT (15h UTC)

[Oberhuber, Martin]

Hi Mathieu,

This looks like a very interesting approach and I've read the Blog,
but unfortunately I can't make the 9th.

I don't think I would have much to contribute, but I am certainly
interested in what others say. Could the session be recorded, or
meeting notes posted? - Otherwise I'd be available from the 15th on,
in case you'd consider moving the session.

Two questions came up for me when reading the blog,

1. Julien writes  "We are not trying to solve a problem related to the interrupts
   being masked for too long (ftrace already has some tools to help with that)."
   Which are those tools? Any pointer would be appreciated.

2. I really like the general usefulness of the latency_tracker and the many clever
    ways it can be configured / leveraged - but it feels like an expert tool to me,
    where you'd very clearly have to know what you're doing. Are there any ideas
    making such expert tools more usable and discoverable by general users ?

Many thanks,
Martin
--
Martin Oberhuber, SMTS / Product Owner - Development Tools, Wind River
direct +43.662.457915.85  fax +43.662.457915.6


-----Original Message-----
From: diamon-discuss-bounces@lists.linuxfoundation.org [mailto:diamon-discuss-bounces@lists.linuxfoundation.org] On Behalf Of Mathieu Desnoyers
Sent: Thursday, January 28, 2016 9:09 PM
To: diamon-discuss@lists.linuxfoundation.org
Subject: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016, at 11h EDT (15h UTC)

Hi,

Following the blog post published two weeks ago [1], we would like to propose organizing a phone meeting with all interested members of this workgroup on February 9th to gather feedback and ideas for improvement on the subject of measuring and detecting high response time.

At EfficiOS, we have developed a kernel module for monitoring at run-time the delay between the moment the kernel starts processing an interrupt
(do_IRQ) and the moment the target task gets scheduled in or has finished processing the data.

When a high latency is detected, it emits a tracepoint event and can wakeup a user-space script to take arbitrary actions as soon as possible.

The main intent is to provide an entry point in a kernel trace. After that, everyone has their own methodology to process the trace. The blog post illustrates what we can do with LTTng as an example but the detection and triggers are not coupled with any tracer.

The proposed agenda is a discussion around these points:
- presentation of the scope of the problem
- limitation of the current tools
- overview of the latency_tracker module applied for this use-case
-- current state
-- use-cases
-- future plans
- from the audience: comments, ideas, other approaches, etc.

If you have other points you would like to discuss around this subject, please let me know and I will add them.

Also, if you wish to attend but can't make it at the proposed date and time, let us know.

The details for the conference call will be sent soon.

Thanks,

Julien & Mathieu

[1] https://lttng.org/blog/2016/01/06/monitoring-realtime-latencies/

--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
_______________________________________________
diamon-discuss mailing list
diamon-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/diamon-discuss

Re: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016, at 11h EDT (15h UTC)

[Julien Desfossez]

Hi,

> I don't think I would have much to contribute, but I am certainly
> interested in what others say. Could the session be recorded, or
> meeting notes posted? - Otherwise I'd be available from the 15th on,
> in case you'd consider moving the session.
We did not receive any other requests for moving the meeting, so we will
do it tomorrow, I will try to record it.

> Two questions came up for me when reading the blog,
> 
> 1. Julien writes  "We are not trying to solve a problem related to the interrupts
>    being masked for too long (ftrace already has some tools to help with that)."
>    Which are those tools? Any pointer would be appreciated.
I was refering to the irqsoff tracer in ftrace.

> 2. I really like the general usefulness of the latency_tracker and the many clever
>     ways it can be configured / leveraged - but it feels like an expert tool to me,
>     where you'd very clearly have to know what you're doing. Are there any ideas
>     making such expert tools more usable and discoverable by general users ?
Indeed for now it is not very user-friendly as we are still defining all
the knobs that we want to expose to the users. When we will have
gathered enough feedback we will see if how we can adapt this to make it
more easy to approach.

Thanks for the feedback,

Julien

Re: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016, at 11h EDT (15h UTC)

[Mathieu Desnoyers]

Hi Martin,

We are trying to clarify with Mike Dolan whether the recording
feature of the conference system was working or not during the
call. As far as I can see, it did not appear to have worked,
even though the option was enabled in the conference system. :-(

Thanks,

Mathieu

----- On Feb 4, 2016, at 4:31 AM, Oberhuber, Martin Martin.Oberhuber@windriver.com wrote:

> Hi Mathieu,
> 
> This looks like a very interesting approach and I've read the Blog,
> but unfortunately I can't make the 9th.
> 
> I don't think I would have much to contribute, but I am certainly
> interested in what others say. Could the session be recorded, or
> meeting notes posted? - Otherwise I'd be available from the 15th on,
> in case you'd consider moving the session.
> 
> Two questions came up for me when reading the blog,
> 
> 1. Julien writes  "We are not trying to solve a problem related to the
> interrupts
>   being masked for too long (ftrace already has some tools to help with that)."
>   Which are those tools? Any pointer would be appreciated.
> 
> 2. I really like the general usefulness of the latency_tracker and the many
> clever
>    ways it can be configured / leveraged - but it feels like an expert tool to me,
>    where you'd very clearly have to know what you're doing. Are there any ideas
>    making such expert tools more usable and discoverable by general users ?
> 
> Many thanks,
> Martin
> --
> Martin Oberhuber, SMTS / Product Owner - Development Tools, Wind River
> direct +43.662.457915.85  fax +43.662.457915.6
> 
> 
> -----Original Message-----
> From: diamon-discuss-bounces@lists.linuxfoundation.org
> [mailto:diamon-discuss-bounces@lists.linuxfoundation.org] On Behalf Of Mathieu
> Desnoyers
> Sent: Thursday, January 28, 2016 9:09 PM
> To: diamon-discuss@lists.linuxfoundation.org
> Subject: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016, at 11h
> EDT (15h UTC)
> 
> Hi,
> 
> Following the blog post published two weeks ago [1], we would like to propose
> organizing a phone meeting with all interested members of this workgroup on
> February 9th to gather feedback and ideas for improvement on the subject of
> measuring and detecting high response time.
> 
> At EfficiOS, we have developed a kernel module for monitoring at run-time the
> delay between the moment the kernel starts processing an interrupt
> (do_IRQ) and the moment the target task gets scheduled in or has finished
> processing the data.
> 
> When a high latency is detected, it emits a tracepoint event and can wakeup a
> user-space script to take arbitrary actions as soon as possible.
> 
> The main intent is to provide an entry point in a kernel trace. After that,
> everyone has their own methodology to process the trace. The blog post
> illustrates what we can do with LTTng as an example but the detection and
> triggers are not coupled with any tracer.
> 
> The proposed agenda is a discussion around these points:
> - presentation of the scope of the problem
> - limitation of the current tools
> - overview of the latency_tracker module applied for this use-case
> -- current state
> -- use-cases
> -- future plans
> - from the audience: comments, ideas, other approaches, etc.
> 
> If you have other points you would like to discuss around this subject, please
> let me know and I will add them.
> 
> Also, if you wish to attend but can't make it at the proposed date and time, let
> us know.
> 
> The details for the conference call will be sent soon.
> 
> Thanks,
> 
> Julien & Mathieu
> 
> [1] https://lttng.org/blog/2016/01/06/monitoring-realtime-latencies/
> 
> --
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com
> _______________________________________________
> diamon-discuss mailing list
> diamon-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/diamon-discuss

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Re: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016, at 11h EDT (15h UTC)

[Julien Desfossez]

Mike confirmed to me that the recording did not work, so I am annotating
the slides to add a little bit more context and I will send them very
soon.

Julien

On 16-Feb-2016 09:15:28 PM, Mathieu Desnoyers wrote:
> Hi Martin,
> 
> We are trying to clarify with Mike Dolan whether the recording
> feature of the conference system was working or not during the
> call. As far as I can see, it did not appear to have worked,
> even though the option was enabled in the conference system. :-(
> 
> Thanks,
> 
> Mathieu
> 
> ----- On Feb 4, 2016, at 4:31 AM, Oberhuber, Martin Martin.Oberhuber@windriver.com wrote:
> 
> > Hi Mathieu,
> > 
> > This looks like a very interesting approach and I've read the Blog,
> > but unfortunately I can't make the 9th.
> > 
> > I don't think I would have much to contribute, but I am certainly
> > interested in what others say. Could the session be recorded, or
> > meeting notes posted? - Otherwise I'd be available from the 15th on,
> > in case you'd consider moving the session.
> > 
> > Two questions came up for me when reading the blog,
> > 
> > 1. Julien writes  "We are not trying to solve a problem related to the
> > interrupts
> >   being masked for too long (ftrace already has some tools to help with that)."
> >   Which are those tools? Any pointer would be appreciated.
> > 
> > 2. I really like the general usefulness of the latency_tracker and the many
> > clever
> >    ways it can be configured / leveraged - but it feels like an expert tool to me,
> >    where you'd very clearly have to know what you're doing. Are there any ideas
> >    making such expert tools more usable and discoverable by general users ?
> > 
> > Many thanks,
> > Martin
> > --
> > Martin Oberhuber, SMTS / Product Owner - Development Tools, Wind River
> > direct +43.662.457915.85  fax +43.662.457915.6
> > 
> > 
> > -----Original Message-----
> > From: diamon-discuss-bounces@lists.linuxfoundation.org
> > [mailto:diamon-discuss-bounces@lists.linuxfoundation.org] On Behalf Of Mathieu
> > Desnoyers
> > Sent: Thursday, January 28, 2016 9:09 PM
> > To: diamon-discuss@lists.linuxfoundation.org
> > Subject: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016, at 11h
> > EDT (15h UTC)
> > 
> > Hi,
> > 
> > Following the blog post published two weeks ago [1], we would like to propose
> > organizing a phone meeting with all interested members of this workgroup on
> > February 9th to gather feedback and ideas for improvement on the subject of
> > measuring and detecting high response time.
> > 
> > At EfficiOS, we have developed a kernel module for monitoring at run-time the
> > delay between the moment the kernel starts processing an interrupt
> > (do_IRQ) and the moment the target task gets scheduled in or has finished
> > processing the data.
> > 
> > When a high latency is detected, it emits a tracepoint event and can wakeup a
> > user-space script to take arbitrary actions as soon as possible.
> > 
> > The main intent is to provide an entry point in a kernel trace. After that,
> > everyone has their own methodology to process the trace. The blog post
> > illustrates what we can do with LTTng as an example but the detection and
> > triggers are not coupled with any tracer.
> > 
> > The proposed agenda is a discussion around these points:
> > - presentation of the scope of the problem
> > - limitation of the current tools
> > - overview of the latency_tracker module applied for this use-case
> > -- current state
> > -- use-cases
> > -- future plans
> > - from the audience: comments, ideas, other approaches, etc.
> > 
> > If you have other points you would like to discuss around this subject, please
> > let me know and I will add them.
> > 
> > Also, if you wish to attend but can't make it at the proposed date and time, let
> > us know.
> > 
> > The details for the conference call will be sent soon.
> > 
> > Thanks,
> > 
> > Julien & Mathieu
> > 
> > [1] https://lttng.org/blog/2016/01/06/monitoring-realtime-latencies/
> > 
> > --
> > Mathieu Desnoyers
> > EfficiOS Inc.
> > http://www.efficios.com
> > _______________________________________________
> > diamon-discuss mailing list
> > diamon-discuss@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/diamon-discuss
> 
> -- 
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com
> _______________________________________________
> diamon-discuss mailing list
> diamon-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/diamon-discuss

Re: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016, at 11h EDT (15h UTC)

[Michael Dolan]

Sorry all, not sure what happened on the last call. I had a conflict so wasn’t there, but Julien, maybe you and I should try a practice run before the next meeting?


---
Mike Dolan
VP of Strategic Programs
The Linux Foundation
Office: +1.330.460.3250   Cell: +1.440.552.5322  Skype: michaelkdolan
mdolan@linuxfoundation.org
---

> On Feb 16, 2016, at 4:22 PM, Julien Desfossez <jdesfossez@efficios.com> wrote:
> 
> Mike confirmed to me that the recording did not work, so I am annotating
> the slides to add a little bit more context and I will send them very
> soon.
> 
> Julien
> 
> On 16-Feb-2016 09:15:28 PM, Mathieu Desnoyers wrote:
>> Hi Martin,
>> 
>> We are trying to clarify with Mike Dolan whether the recording
>> feature of the conference system was working or not during the
>> call. As far as I can see, it did not appear to have worked,
>> even though the option was enabled in the conference system. :-(
>> 
>> Thanks,
>> 
>> Mathieu
>> 
>> ----- On Feb 4, 2016, at 4:31 AM, Oberhuber, Martin Martin.Oberhuber@windriver.com wrote:
>> 
>>> Hi Mathieu,
>>> 
>>> This looks like a very interesting approach and I've read the Blog,
>>> but unfortunately I can't make the 9th.
>>> 
>>> I don't think I would have much to contribute, but I am certainly
>>> interested in what others say. Could the session be recorded, or
>>> meeting notes posted? - Otherwise I'd be available from the 15th on,
>>> in case you'd consider moving the session.
>>> 
>>> Two questions came up for me when reading the blog,
>>> 
>>> 1. Julien writes  "We are not trying to solve a problem related to the
>>> interrupts
>>>  being masked for too long (ftrace already has some tools to help with that)."
>>>  Which are those tools? Any pointer would be appreciated.
>>> 
>>> 2. I really like the general usefulness of the latency_tracker and the many
>>> clever
>>>   ways it can be configured / leveraged - but it feels like an expert tool to me,
>>>   where you'd very clearly have to know what you're doing. Are there any ideas
>>>   making such expert tools more usable and discoverable by general users ?
>>> 
>>> Many thanks,
>>> Martin
>>> --
>>> Martin Oberhuber, SMTS / Product Owner - Development Tools, Wind River
>>> direct +43.662.457915.85  fax +43.662.457915.6
>>> 
>>> 
>>> -----Original Message-----
>>> From: diamon-discuss-bounces@lists.linuxfoundation.org
>>> [mailto:diamon-discuss-bounces@lists.linuxfoundation.org] On Behalf Of Mathieu
>>> Desnoyers
>>> Sent: Thursday, January 28, 2016 9:09 PM
>>> To: diamon-discuss@lists.linuxfoundation.org
>>> Subject: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016, at 11h
>>> EDT (15h UTC)
>>> 
>>> Hi,
>>> 
>>> Following the blog post published two weeks ago [1], we would like to propose
>>> organizing a phone meeting with all interested members of this workgroup on
>>> February 9th to gather feedback and ideas for improvement on the subject of
>>> measuring and detecting high response time.
>>> 
>>> At EfficiOS, we have developed a kernel module for monitoring at run-time the
>>> delay between the moment the kernel starts processing an interrupt
>>> (do_IRQ) and the moment the target task gets scheduled in or has finished
>>> processing the data.
>>> 
>>> When a high latency is detected, it emits a tracepoint event and can wakeup a
>>> user-space script to take arbitrary actions as soon as possible.
>>> 
>>> The main intent is to provide an entry point in a kernel trace. After that,
>>> everyone has their own methodology to process the trace. The blog post
>>> illustrates what we can do with LTTng as an example but the detection and
>>> triggers are not coupled with any tracer.
>>> 
>>> The proposed agenda is a discussion around these points:
>>> - presentation of the scope of the problem
>>> - limitation of the current tools
>>> - overview of the latency_tracker module applied for this use-case
>>> -- current state
>>> -- use-cases
>>> -- future plans
>>> - from the audience: comments, ideas, other approaches, etc.
>>> 
>>> If you have other points you would like to discuss around this subject, please
>>> let me know and I will add them.
>>> 
>>> Also, if you wish to attend but can't make it at the proposed date and time, let
>>> us know.
>>> 
>>> The details for the conference call will be sent soon.
>>> 
>>> Thanks,
>>> 
>>> Julien & Mathieu
>>> 
>>> [1] https://lttng.org/blog/2016/01/06/monitoring-realtime-latencies/
>>> 
>>> --
>>> Mathieu Desnoyers
>>> EfficiOS Inc.
>>> http://www.efficios.com
>>> _______________________________________________
>>> diamon-discuss mailing list
>>> diamon-discuss@lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/diamon-discuss
>> 
>> -- 
>> Mathieu Desnoyers
>> EfficiOS Inc.
>> http://www.efficios.com
>> _______________________________________________
>> diamon-discuss mailing list
>> diamon-discuss@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/diamon-discuss

Re: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016, at 11h EDT (15h UTC)

[Julien Desfossez]

Hi,

Here is the information for tomorrow's call:
February 9th, 2016, at 11h EDT (15h UTC)

Conference: 415-906-5657 PIN: 88326
URL for the screen sharing: www.uberconference.com/mdolan

Local International phone numbers are available at:
https://www.uberconference.com/international

Please note anyone dialing using the international numbers need to dial
the local international number for the country they are in, then enter
the US conference number (4159065657), and then enter the PIN
(88326).

Julien

On 28-Jan-2016 08:08:45 PM, Mathieu Desnoyers wrote:
> Hi,
> 
> Following the blog post published two weeks ago [1], we would like to
> propose organizing a phone meeting with all interested members of this
> workgroup on February 9th to gather feedback and ideas for improvement
> on the subject of measuring and detecting high response time.
> 
> At EfficiOS, we have developed a kernel module for monitoring at run-time
> the delay between the moment the kernel starts processing an interrupt
> (do_IRQ) and the moment the target task gets scheduled in or has finished
> processing the data.
> 
> When a high latency is detected, it emits a tracepoint event and can
> wakeup a user-space script to take arbitrary actions as soon as
> possible.
> 
> The main intent is to provide an entry point in a kernel trace. After
> that, everyone has their own methodology to process the trace. The blog
> post illustrates what we can do with LTTng as an example but the
> detection and triggers are not coupled with any tracer.
> 
> The proposed agenda is a discussion around these points:
> - presentation of the scope of the problem
> - limitation of the current tools
> - overview of the latency_tracker module applied for this use-case
> -- current state
> -- use-cases
> -- future plans
> - from the audience: comments, ideas, other approaches, etc.
> 
> If you have other points you would like to discuss around this subject,
> please let me know and I will add them.
> 
> Also, if you wish to attend but can't make it at the proposed date and
> time, let us know.
> 
> The details for the conference call will be sent soon.
> 
> Thanks,
> 
> Julien & Mathieu
> 
> [1] https://lttng.org/blog/2016/01/06/monitoring-realtime-latencies/
> 
> -- 
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com

Re: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016, at 11h EDT (15h UTC)

[Mathieu Desnoyers]

The conference call is starting now. See you there,

Thanks,

Mathieu

----- On Feb 8, 2016, at 2:13 PM, Julien Desfossez jdesfossez@efficios.com wrote:

> Hi,
> 
> Here is the information for tomorrow's call:
> February 9th, 2016, at 11h EDT (15h UTC)
> 
> Conference: 415-906-5657 PIN: 88326
> URL for the screen sharing: www.uberconference.com/mdolan
> 
> Local International phone numbers are available at:
> https://www.uberconference.com/international
> 
> Please note anyone dialing using the international numbers need to dial
> the local international number for the country they are in, then enter
> the US conference number (4159065657), and then enter the PIN
> (88326).
> 
> Julien
> 
> On 28-Jan-2016 08:08:45 PM, Mathieu Desnoyers wrote:
>> Hi,
>> 
>> Following the blog post published two weeks ago [1], we would like to
>> propose organizing a phone meeting with all interested members of this
>> workgroup on February 9th to gather feedback and ideas for improvement
>> on the subject of measuring and detecting high response time.
>> 
>> At EfficiOS, we have developed a kernel module for monitoring at run-time
>> the delay between the moment the kernel starts processing an interrupt
>> (do_IRQ) and the moment the target task gets scheduled in or has finished
>> processing the data.
>> 
>> When a high latency is detected, it emits a tracepoint event and can
>> wakeup a user-space script to take arbitrary actions as soon as
>> possible.
>> 
>> The main intent is to provide an entry point in a kernel trace. After
>> that, everyone has their own methodology to process the trace. The blog
>> post illustrates what we can do with LTTng as an example but the
>> detection and triggers are not coupled with any tracer.
>> 
>> The proposed agenda is a discussion around these points:
>> - presentation of the scope of the problem
>> - limitation of the current tools
>> - overview of the latency_tracker module applied for this use-case
>> -- current state
>> -- use-cases
>> -- future plans
>> - from the audience: comments, ideas, other approaches, etc.
>> 
>> If you have other points you would like to discuss around this subject,
>> please let me know and I will add them.
>> 
>> Also, if you wish to attend but can't make it at the proposed date and
>> time, let us know.
>> 
>> The details for the conference call will be sent soon.
>> 
>> Thanks,
>> 
>> Julien & Mathieu
>> 
>> [1] https://lttng.org/blog/2016/01/06/monitoring-realtime-latencies/
>> 
>> --
>> Mathieu Desnoyers
>> EfficiOS Inc.
> > http://www.efficios.com

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Re: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016, at 11h EDT (15h UTC)

[Al Grant]

Hi,

I have read the blog and this looks a very useful technique but unfortunately I
couldn't make the call.  Are slides/minutes available?

My main question is not actually technical - it's about the prospects for getting
this into mainline Linux.

Thanks,

Al


> -----Original Message-----
> From: diamon-discuss-bounces@lists.linuxfoundation.org [mailto:diamon-
> discuss-bounces@lists.linuxfoundation.org] On Behalf Of Julien Desfossez
> Sent: 08 February 2016 19:14
> To: Mathieu Desnoyers
> Cc: diamon-discuss@lists.linuxfoundation.org
> Subject: Re: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016,
> at 11h EDT (15h UTC)
>
> Hi,
>
> Here is the information for tomorrow's call:
> February 9th, 2016, at 11h EDT (15h UTC)
>
> Conference: 415-906-5657 PIN: 88326
> URL for the screen sharing: www.uberconference.com/mdolan
>
> Local International phone numbers are available at:
> https://www.uberconference.com/international
>
> Please note anyone dialing using the international numbers need to dial the
> local international number for the country they are in, then enter the US
> conference number (4159065657), and then enter the PIN (88326).
>
> Julien
>
> On 28-Jan-2016 08:08:45 PM, Mathieu Desnoyers wrote:
> > Hi,
> >
> > Following the blog post published two weeks ago [1], we would like to
> > propose organizing a phone meeting with all interested members of this
> > workgroup on February 9th to gather feedback and ideas for improvement
> > on the subject of measuring and detecting high response time.
> >
> > At EfficiOS, we have developed a kernel module for monitoring at
> > run-time the delay between the moment the kernel starts processing an
> > interrupt
> > (do_IRQ) and the moment the target task gets scheduled in or has
> > finished processing the data.
> >
> > When a high latency is detected, it emits a tracepoint event and can
> > wakeup a user-space script to take arbitrary actions as soon as
> > possible.
> >
> > The main intent is to provide an entry point in a kernel trace. After
> > that, everyone has their own methodology to process the trace. The
> > blog post illustrates what we can do with LTTng as an example but the
> > detection and triggers are not coupled with any tracer.
> >
> > The proposed agenda is a discussion around these points:
> > - presentation of the scope of the problem
> > - limitation of the current tools
> > - overview of the latency_tracker module applied for this use-case
> > -- current state
> > -- use-cases
> > -- future plans
> > - from the audience: comments, ideas, other approaches, etc.
> >
> > If you have other points you would like to discuss around this
> > subject, please let me know and I will add them.
> >
> > Also, if you wish to attend but can't make it at the proposed date and
> > time, let us know.
> >
> > The details for the conference call will be sent soon.
> >
> > Thanks,
> >
> > Julien & Mathieu
> >
> > [1] https://lttng.org/blog/2016/01/06/monitoring-realtime-latencies/
> >
> > --
> > Mathieu Desnoyers
> > EfficiOS Inc.
> > http://www.efficios.com
> _______________________________________________
> diamon-discuss mailing list
> diamon-discuss@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/diamon-discuss

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Re: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016, at 11h EDT (15h UTC)

[Julien Desfossez]

Hi,

We are currently setting up a place on the diamon website so we can
upload the slides, I will send the link soon.

Regarding the mainlining, we intend to push this projet upstream. Before
that, we are working on more measurements/benchmarks, stability testing
and usability improvements.

Also, one things that we would like to investigate before sending the
patches is the possibility of using eBPF/bcc to handle the probes while
keeping the latency-tracker as the backend to keep the state and all the
more advanced interactions with the kernel. That way, we could combine
the flexibility/safety of eBPF and the efficiency of the
latency-tracker.

Thanks for your feedback, I will keep this list updated of the progress
we make on that front.

Julien

On 14-Feb-2016 11:37:44 AM, Al Grant wrote:
> Hi,
> 
> I have read the blog and this looks a very useful technique but unfortunately I
> couldn't make the call.  Are slides/minutes available?
> 
> My main question is not actually technical - it's about the prospects for getting
> this into mainline Linux.
> 
> Thanks,
> 
> Al
> 
> 
> > -----Original Message-----
> > From: diamon-discuss-bounces@lists.linuxfoundation.org [mailto:diamon-
> > discuss-bounces@lists.linuxfoundation.org] On Behalf Of Julien Desfossez
> > Sent: 08 February 2016 19:14
> > To: Mathieu Desnoyers
> > Cc: diamon-discuss@lists.linuxfoundation.org
> > Subject: Re: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016,
> > at 11h EDT (15h UTC)
> >
> > Hi,
> >
> > Here is the information for tomorrow's call:
> > February 9th, 2016, at 11h EDT (15h UTC)
> >
> > Conference: 415-906-5657 PIN: 88326
> > URL for the screen sharing: www.uberconference.com/mdolan
> >
> > Local International phone numbers are available at:
> > https://www.uberconference.com/international
> >
> > Please note anyone dialing using the international numbers need to dial the
> > local international number for the country they are in, then enter the US
> > conference number (4159065657), and then enter the PIN (88326).
> >
> > Julien
> >
> > On 28-Jan-2016 08:08:45 PM, Mathieu Desnoyers wrote:
> > > Hi,
> > >
> > > Following the blog post published two weeks ago [1], we would like to
> > > propose organizing a phone meeting with all interested members of this
> > > workgroup on February 9th to gather feedback and ideas for improvement
> > > on the subject of measuring and detecting high response time.
> > >
> > > At EfficiOS, we have developed a kernel module for monitoring at
> > > run-time the delay between the moment the kernel starts processing an
> > > interrupt
> > > (do_IRQ) and the moment the target task gets scheduled in or has
> > > finished processing the data.
> > >
> > > When a high latency is detected, it emits a tracepoint event and can
> > > wakeup a user-space script to take arbitrary actions as soon as
> > > possible.
> > >
> > > The main intent is to provide an entry point in a kernel trace. After
> > > that, everyone has their own methodology to process the trace. The
> > > blog post illustrates what we can do with LTTng as an example but the
> > > detection and triggers are not coupled with any tracer.
> > >
> > > The proposed agenda is a discussion around these points:
> > > - presentation of the scope of the problem
> > > - limitation of the current tools
> > > - overview of the latency_tracker module applied for this use-case
> > > -- current state
> > > -- use-cases
> > > -- future plans
> > > - from the audience: comments, ideas, other approaches, etc.
> > >
> > > If you have other points you would like to discuss around this
> > > subject, please let me know and I will add them.
> > >
> > > Also, if you wish to attend but can't make it at the proposed date and
> > > time, let us know.
> > >
> > > The details for the conference call will be sent soon.
> > >
> > > Thanks,
> > >
> > > Julien & Mathieu
> > >
> > > [1] https://lttng.org/blog/2016/01/06/monitoring-realtime-latencies/
> > >
> > > --
> > > Mathieu Desnoyers
> > > EfficiOS Inc.
> > > http://www.efficios.com
> > _______________________________________________
> > diamon-discuss mailing list
> > diamon-discuss@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/diamon-discuss
> 
> IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
> 

Re: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016, at 11h EDT (15h UTC)

[Mathieu Desnoyers]

Adding Alexei and Brendan in CC, they will likely be interested
in the eBPF discussion.

Thanks,

Mathieu

----- On Feb 17, 2016, at 11:41 PM, Julien Desfossez jdesfossez@efficios.com wrote:

> Hi,
> 
> We are currently setting up a place on the diamon website so we can
> upload the slides, I will send the link soon.
> 
> Regarding the mainlining, we intend to push this projet upstream. Before
> that, we are working on more measurements/benchmarks, stability testing
> and usability improvements.
> 
> Also, one things that we would like to investigate before sending the
> patches is the possibility of using eBPF/bcc to handle the probes while
> keeping the latency-tracker as the backend to keep the state and all the
> more advanced interactions with the kernel. That way, we could combine
> the flexibility/safety of eBPF and the efficiency of the
> latency-tracker.
> 
> Thanks for your feedback, I will keep this list updated of the progress
> we make on that front.
> 
> Julien
> 
> On 14-Feb-2016 11:37:44 AM, Al Grant wrote:
>> Hi,
>> 
>> I have read the blog and this looks a very useful technique but unfortunately I
>> couldn't make the call.  Are slides/minutes available?
>> 
>> My main question is not actually technical - it's about the prospects for
>> getting
>> this into mainline Linux.
>> 
>> Thanks,
>> 
>> Al
>> 
>> 
>> > -----Original Message-----
>> > From: diamon-discuss-bounces@lists.linuxfoundation.org [mailto:diamon-
>> > discuss-bounces@lists.linuxfoundation.org] On Behalf Of Julien Desfossez
>> > Sent: 08 February 2016 19:14
>> > To: Mathieu Desnoyers
>> > Cc: diamon-discuss@lists.linuxfoundation.org
>> > Subject: Re: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016,
>> > at 11h EDT (15h UTC)
>> >
>> > Hi,
>> >
>> > Here is the information for tomorrow's call:
>> > February 9th, 2016, at 11h EDT (15h UTC)
>> >
>> > Conference: 415-906-5657 PIN: 88326
>> > URL for the screen sharing: www.uberconference.com/mdolan
>> >
>> > Local International phone numbers are available at:
>> > https://www.uberconference.com/international
>> >
>> > Please note anyone dialing using the international numbers need to dial the
>> > local international number for the country they are in, then enter the US
>> > conference number (4159065657), and then enter the PIN (88326).
>> >
>> > Julien
>> >
>> > On 28-Jan-2016 08:08:45 PM, Mathieu Desnoyers wrote:
>> > > Hi,
>> > >
>> > > Following the blog post published two weeks ago [1], we would like to
>> > > propose organizing a phone meeting with all interested members of this
>> > > workgroup on February 9th to gather feedback and ideas for improvement
>> > > on the subject of measuring and detecting high response time.
>> > >
>> > > At EfficiOS, we have developed a kernel module for monitoring at
>> > > run-time the delay between the moment the kernel starts processing an
>> > > interrupt
>> > > (do_IRQ) and the moment the target task gets scheduled in or has
>> > > finished processing the data.
>> > >
>> > > When a high latency is detected, it emits a tracepoint event and can
>> > > wakeup a user-space script to take arbitrary actions as soon as
>> > > possible.
>> > >
>> > > The main intent is to provide an entry point in a kernel trace. After
>> > > that, everyone has their own methodology to process the trace. The
>> > > blog post illustrates what we can do with LTTng as an example but the
>> > > detection and triggers are not coupled with any tracer.
>> > >
>> > > The proposed agenda is a discussion around these points:
>> > > - presentation of the scope of the problem
>> > > - limitation of the current tools
>> > > - overview of the latency_tracker module applied for this use-case
>> > > -- current state
>> > > -- use-cases
>> > > -- future plans
>> > > - from the audience: comments, ideas, other approaches, etc.
>> > >
>> > > If you have other points you would like to discuss around this
>> > > subject, please let me know and I will add them.
>> > >
>> > > Also, if you wish to attend but can't make it at the proposed date and
>> > > time, let us know.
>> > >
>> > > The details for the conference call will be sent soon.
>> > >
>> > > Thanks,
>> > >
>> > > Julien & Mathieu
>> > >
>> > > [1] https://lttng.org/blog/2016/01/06/monitoring-realtime-latencies/
>> > >
>> > > --
>> > > Mathieu Desnoyers
>> > > EfficiOS Inc.
>> > > http://www.efficios.com
>> > _______________________________________________
>> > diamon-discuss mailing list
>> > diamon-discuss@lists.linuxfoundation.org
>> > https://lists.linuxfoundation.org/mailman/listinfo/diamon-discuss
>> 
>> IMPORTANT NOTICE: The contents of this email and any attachments are
>> confidential and may also be privileged. If you are not the intended recipient,
>> please notify the sender immediately and do not disclose the contents to any
>> other person, use it for any purpose, or store or copy the information in any
>> medium. Thank you.

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Re: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016, at 11h EDT (15h UTC)

[Alexei Starovoitov]

On Thu, Feb 18, 2016 at 2:44 PM, Mathieu Desnoyers
<mathieu.desnoyers@efficios.com> wrote:
> Adding Alexei and Brendan in CC, they will likely be interested
> in the eBPF discussion.

yep :)
please cc iovisor-dev as well for ideas you guys have around
making use of bpf kernel bits with 'latency-tracker'.

> ----- On Feb 17, 2016, at 11:41 PM, Julien Desfossez jdesfossez@efficios.com wrote:
>
>> Hi,
>>
>> We are currently setting up a place on the diamon website so we can
>> upload the slides, I will send the link soon.
>>
>> Regarding the mainlining, we intend to push this projet upstream. Before
>> that, we are working on more measurements/benchmarks, stability testing
>> and usability improvements.
>>
>> Also, one things that we would like to investigate before sending the
>> patches is the possibility of using eBPF/bcc to handle the probes while
>> keeping the latency-tracker as the backend to keep the state and all the
>> more advanced interactions with the kernel. That way, we could combine
>> the flexibility/safety of eBPF and the efficiency of the
>> latency-tracker.
>>
>> Thanks for your feedback, I will keep this list updated of the progress
>> we make on that front.
>>
>> Julien
>>
>> On 14-Feb-2016 11:37:44 AM, Al Grant wrote:
>>> Hi,
>>>
>>> I have read the blog and this looks a very useful technique but unfortunately I
>>> couldn't make the call.  Are slides/minutes available?
>>>
>>> My main question is not actually technical - it's about the prospects for
>>> getting
>>> this into mainline Linux.
>>>
>>> Thanks,
>>>
>>> Al
>>>
>>>
>>> > -----Original Message-----
>>> > From: diamon-discuss-bounces@lists.linuxfoundation.org [mailto:diamon-
>>> > discuss-bounces@lists.linuxfoundation.org] On Behalf Of Julien Desfossez
>>> > Sent: 08 February 2016 19:14
>>> > To: Mathieu Desnoyers
>>> > Cc: diamon-discuss@lists.linuxfoundation.org
>>> > Subject: Re: [diamon-discuss] Diamon Meeting on Tuesday February 9th, 2016,
>>> > at 11h EDT (15h UTC)
>>> >
>>> > Hi,
>>> >
>>> > Here is the information for tomorrow's call:
>>> > February 9th, 2016, at 11h EDT (15h UTC)
>>> >
>>> > Conference: 415-906-5657 PIN: 88326
>>> > URL for the screen sharing: www.uberconference.com/mdolan
>>> >
>>> > Local International phone numbers are available at:
>>> > https://www.uberconference.com/international
>>> >
>>> > Please note anyone dialing using the international numbers need to dial the
>>> > local international number for the country they are in, then enter the US
>>> > conference number (4159065657), and then enter the PIN (88326).
>>> >
>>> > Julien
>>> >
>>> > On 28-Jan-2016 08:08:45 PM, Mathieu Desnoyers wrote:
>>> > > Hi,
>>> > >
>>> > > Following the blog post published two weeks ago [1], we would like to
>>> > > propose organizing a phone meeting with all interested members of this
>>> > > workgroup on February 9th to gather feedback and ideas for improvement
>>> > > on the subject of measuring and detecting high response time.
>>> > >
>>> > > At EfficiOS, we have developed a kernel module for monitoring at
>>> > > run-time the delay between the moment the kernel starts processing an
>>> > > interrupt
>>> > > (do_IRQ) and the moment the target task gets scheduled in or has
>>> > > finished processing the data.
>>> > >
>>> > > When a high latency is detected, it emits a tracepoint event and can
>>> > > wakeup a user-space script to take arbitrary actions as soon as
>>> > > possible.
>>> > >
>>> > > The main intent is to provide an entry point in a kernel trace. After
>>> > > that, everyone has their own methodology to process the trace. The
>>> > > blog post illustrates what we can do with LTTng as an example but the
>>> > > detection and triggers are not coupled with any tracer.
>>> > >
>>> > > The proposed agenda is a discussion around these points:
>>> > > - presentation of the scope of the problem
>>> > > - limitation of the current tools
>>> > > - overview of the latency_tracker module applied for this use-case
>>> > > -- current state
>>> > > -- use-cases
>>> > > -- future plans
>>> > > - from the audience: comments, ideas, other approaches, etc.
>>> > >
>>> > > If you have other points you would like to discuss around this
>>> > > subject, please let me know and I will add them.
>>> > >
>>> > > Also, if you wish to attend but can't make it at the proposed date and
>>> > > time, let us know.
>>> > >
>>> > > The details for the conference call will be sent soon.
>>> > >
>>> > > Thanks,
>>> > >
>>> > > Julien & Mathieu
>>> > >
>>> > > [1] https://lttng.org/blog/2016/01/06/monitoring-realtime-latencies/
>>> > >
>>> > > --
>>> > > Mathieu Desnoyers
>>> > > EfficiOS Inc.
>>> > > http://www.efficios.com
>>> > _______________________________________________
>>> > diamon-discuss mailing list
>>> > diamon-discuss@lists.linuxfoundation.org
>>> > https://lists.linuxfoundation.org/mailman/listinfo/diamon-discuss
>>>
>>> IMPORTANT NOTICE: The contents of this email and any attachments are
>>> confidential and may also be privileged. If you are not the intended recipient,
>>> please notify the sender immediately and do not disclose the contents to any
>>> other person, use it for any purpose, or store or copy the information in any
>>> medium. Thank you.
>
> --
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com