multiple telemetry designs

multiple telemetry designs

[Brad Bishop]

There are two telemetry / metric designs under review right now:

https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/22257
https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24357

I would like to see one or both of them merged.  Neither seem to have any support….there is a single +1 on the latter review from Shawn McCarney.  If you support one of these designs, please go review it and indicate your support.

I also can’t figure out what the path forward for OpenBMC is.  Maybe to start with, we could all level set on where we are at with our thinking:

Kun: How far along are you in the implementation of your proposal?
Piotr: How far along are you in the implementation of your proposal?
James: OpenBMC can support one or both proposals.  If we support both, this means multiple implementations for the same thing in bmcweb.  One that gets data from dbus objects, and another that gets it from librrd.  As the maintainer of bmcweb are you open to accepting one or both of these options?

Without any discussion and resolution, I’m afraid both of these proposals are just going to sit here, unmerged, indefinitely.

thx - brad

Re: multiple telemetry designs

[James Feist]

On 10/23/19 9:38 AM, Brad Bishop wrote:
> There are two telemetry / metric designs under review right now:
> 
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/22257
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24357
> 
> I would like to see one or both of them merged.  Neither seem to have any support….there is a single +1 on the latter review from Shawn McCarney.  If you support one of these designs, please go review it and indicate your support.

It looks like Kun has +1ed Piotr's design as well, not sure if that 
means we can go with one design?

> 
> I also can’t figure out what the path forward for OpenBMC is.  Maybe to start with, we could all level set on where we are at with our thinking:
> 
> Kun: How far along are you in the implementation of your proposal?
> Piotr: How far along are you in the implementation of your proposal?
> James: OpenBMC can support one or both proposals.  If we support both, this means multiple implementations for the same thing in bmcweb.  One that gets data from dbus objects, and another that gets it from librrd.  As the maintainer of bmcweb are you open to accepting one or both of these options?

With 0 previous knowledge, I would suggest using a way that works with 
previous OBMC practices, and that is using dbus. If there has to be 2 
separate designs, then if both produce the same d-bus interfaces, it 
shouldn't matter to bmcweb which one is being used. That being said, if 
this doesn't work for some reason, we've used compile switches in the 
past, although this would be the least preferable option. Truthfully I 
haven't looked at these designs yet as I've only recently taken a larger 
role in bmcweb reviews, so I'll try to look at them soon.


> 
> Without any discussion and resolution, I’m afraid both of these proposals are just going to sit here, unmerged, indefinitely.
> 
> thx - brad
> 

Re: multiple telemetry designs

[James Feist]

On 10/23/19 10:39 AM, James Feist wrote:
> On 10/23/19 9:38 AM, Brad Bishop wrote:
>> There are two telemetry / metric designs under review right now:

I've been informed there are actually 3:

https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749

Added Appu to this conversation as well.


>>
>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/22257
>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24357
>>
>> I would like to see one or both of them merged.  Neither seem to have 
>> any support….there is a single +1 on the latter review from Shawn 
>> McCarney.  If you support one of these designs, please go review it 
>> and indicate your support.
> 
> It looks like Kun has +1ed Piotr's design as well, not sure if that 
> means we can go with one design?
> 
>>
>> I also can’t figure out what the path forward for OpenBMC is.  Maybe 
>> to start with, we could all level set on where we are at with our 
>> thinking:
>>
>> Kun: How far along are you in the implementation of your proposal?
>> Piotr: How far along are you in the implementation of your proposal?
>> James: OpenBMC can support one or both proposals.  If we support both, 
>> this means multiple implementations for the same thing in bmcweb.  One 
>> that gets data from dbus objects, and another that gets it from 
>> librrd.  As the maintainer of bmcweb are you open to accepting one or 
>> both of these options?
> 
> With 0 previous knowledge, I would suggest using a way that works with 
> previous OBMC practices, and that is using dbus. If there has to be 2 
> separate designs, then if both produce the same d-bus interfaces, it 
> shouldn't matter to bmcweb which one is being used. That being said, if 
> this doesn't work for some reason, we've used compile switches in the 
> past, although this would be the least preferable option. Truthfully I 
> haven't looked at these designs yet as I've only recently taken a larger 
> role in bmcweb reviews, so I'll try to look at them soon.
> 
> 
>>
>> Without any discussion and resolution, I’m afraid both of these 
>> proposals are just going to sit here, unmerged, indefinitely.
>>
>> thx - brad
>>

Re: multiple telemetry designs

[Justin Thaler]

On 10/23/19 12:47 PM, James Feist wrote:
> On 10/23/19 10:39 AM, James Feist wrote:
>> On 10/23/19 9:38 AM, Brad Bishop wrote:
>>> There are two telemetry / metric designs under review right now:

There's a fourth option that can also be used, but more than just sensor 
readings. This isn't redfish compliant and relies on secure websockets 
however.

https://github.com/openbmc/docs/blob/master/rest-api.md#event-subscription-protocol

> 
> I've been informed there are actually 3:
> 
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749
> 
> Added Appu to this conversation as well.
> 
> 
>>>
>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/22257
>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24357
>>>
>>> I would like to see one or both of them merged.  Neither seem to have 
>>> any support….there is a single +1 on the latter review from Shawn 
>>> McCarney.  If you support one of these designs, please go review it 
>>> and indicate your support.
>>
>> It looks like Kun has +1ed Piotr's design as well, not sure if that 
>> means we can go with one design?
>>
>>>
>>> I also can’t figure out what the path forward for OpenBMC is.  Maybe 
>>> to start with, we could all level set on where we are at with our 
>>> thinking:
>>>
>>> Kun: How far along are you in the implementation of your proposal?
>>> Piotr: How far along are you in the implementation of your proposal?
>>> James: OpenBMC can support one or both proposals.  If we support 
>>> both, this means multiple implementations for the same thing in 
>>> bmcweb.  One that gets data from dbus objects, and another that gets 
>>> it from librrd.  As the maintainer of bmcweb are you open to 
>>> accepting one or both of these options?
>>
>> With 0 previous knowledge, I would suggest using a way that works with 
>> previous OBMC practices, and that is using dbus. If there has to be 2 
>> separate designs, then if both produce the same d-bus interfaces, it 
>> shouldn't matter to bmcweb which one is being used. That being said, 
>> if this doesn't work for some reason, we've used compile switches in 
>> the past, although this would be the least preferable option. 
>> Truthfully I haven't looked at these designs yet as I've only recently 
>> taken a larger role in bmcweb reviews, so I'll try to look at them soon.
>>
>>
>>>
>>> Without any discussion and resolution, I’m afraid both of these 
>>> proposals are just going to sit here, unmerged, indefinitely.
>>>
>>> thx - brad
>>>

RE: multiple telemetry designs

[Matuszczak, Piotr]

As for the two telemetry design proposals:
https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/22257
https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24357
I talked with Kun and we agreed that a common bmcweb interface would be great in order to be able to support either the monitoring service or the collectd. In order to to do that collectd will have to be modified to expose D-Bus interface or bmcweb will have to support libraries to handle collectd directly. We at Intel prefer D-Bus for the OpenBMC architecture consistency. 
As for the implementation, we have monitoring service and Redfish telemetry service implemented (currently without triggers support), however it require some refactoring to be production code quality. 

The Redfish Event Service (https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749) is something different than telemetry service, so I wouldn't consider it as third telemetry design. It is rather prepared to cooperate with the Redfish Telemetry Service and there is reference to telemetry design https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24357 . 

-----Original Message-----
From: Justin Thaler [mailto:thalerj@linux.vnet.ibm.com] 
Sent: Wednesday, October 23, 2019 10:30 PM
To: James Feist <james.feist@linux.intel.com>; Brad Bishop <bradleyb@fuzziesquirrel.com>; Matuszczak, Piotr <piotr.matuszczak@intel.com>; kunyi@google.com; apparao.puli@linux.intel.com
Cc: neladk@microsoft.com; openbmc@lists.ozlabs.org; Mihm, James <james.mihm@intel.com>
Subject: Re: multiple telemetry designs



On 10/23/19 12:47 PM, James Feist wrote:
> On 10/23/19 10:39 AM, James Feist wrote:
>> On 10/23/19 9:38 AM, Brad Bishop wrote:
>>> There are two telemetry / metric designs under review right now:

There's a fourth option that can also be used, but more than just sensor readings. This isn't redfish compliant and relies on secure websockets however.

https://github.com/openbmc/docs/blob/master/rest-api.md#event-subscription-protocol

> 
> I've been informed there are actually 3:
> 
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749
> 
> Added Appu to this conversation as well.
> 
> 
>>>
>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/22257
>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24357
>>>
>>> I would like to see one or both of them merged.  Neither seem to 
>>> have any support….there is a single +1 on the latter review from 
>>> Shawn McCarney.  If you support one of these designs, please go 
>>> review it and indicate your support.
>>
>> It looks like Kun has +1ed Piotr's design as well, not sure if that 
>> means we can go with one design?
>>
>>>
>>> I also can’t figure out what the path forward for OpenBMC is.  Maybe 
>>> to start with, we could all level set on where we are at with our
>>> thinking:
>>>
>>> Kun: How far along are you in the implementation of your proposal?
>>> Piotr: How far along are you in the implementation of your proposal?
>>> James: OpenBMC can support one or both proposals.  If we support 
>>> both, this means multiple implementations for the same thing in 
>>> bmcweb.  One that gets data from dbus objects, and another that gets 
>>> it from librrd.  As the maintainer of bmcweb are you open to 
>>> accepting one or both of these options?
>>
>> With 0 previous knowledge, I would suggest using a way that works 
>> with previous OBMC practices, and that is using dbus. If there has to 
>> be 2 separate designs, then if both produce the same d-bus 
>> interfaces, it shouldn't matter to bmcweb which one is being used. 
>> That being said, if this doesn't work for some reason, we've used 
>> compile switches in the past, although this would be the least preferable option.
>> Truthfully I haven't looked at these designs yet as I've only 
>> recently taken a larger role in bmcweb reviews, so I'll try to look at them soon.
>>
>>
>>>
>>> Without any discussion and resolution, I’m afraid both of these 
>>> proposals are just going to sit here, unmerged, indefinitely.
>>>
>>> thx - brad
>>>

Re: multiple telemetry designs

[James Feist]

On 10/24/19 1:48 AM, Matuszczak, Piotr wrote:
> As for the two telemetry design proposals:
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/22257
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24357
> I talked with Kun and we agreed that a common bmcweb interface would be great in order to be able to support either the monitoring service or the collectd. In order to to do that collectd will have to be modified to expose D-Bus interface or bmcweb will have to support libraries to handle collectd directly. We at Intel prefer D-Bus for the OpenBMC architecture consistency.
> As for the implementation, we have monitoring service and Redfish telemetry service implemented (currently without triggers support), however it require some refactoring to be production code quality.

Is there a WIP review somewhere?

> 
> The Redfish Event Service (https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749) is something different than telemetry service, so I wouldn't consider it as third telemetry design. It is rather prepared to cooperate with the Redfish Telemetry Service and there is reference to telemetry design https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24357 .
> 
> -----Original Message-----
> From: Justin Thaler [mailto:thalerj@linux.vnet.ibm.com]
> Sent: Wednesday, October 23, 2019 10:30 PM
> To: James Feist <james.feist@linux.intel.com>; Brad Bishop <bradleyb@fuzziesquirrel.com>; Matuszczak, Piotr <piotr.matuszczak@intel.com>; kunyi@google.com; apparao.puli@linux.intel.com
> Cc: neladk@microsoft.com; openbmc@lists.ozlabs.org; Mihm, James <james.mihm@intel.com>
> Subject: Re: multiple telemetry designs
> 
> 
> 
> On 10/23/19 12:47 PM, James Feist wrote:
>> On 10/23/19 10:39 AM, James Feist wrote:
>>> On 10/23/19 9:38 AM, Brad Bishop wrote:
>>>> There are two telemetry / metric designs under review right now:
> 
> There's a fourth option that can also be used, but more than just sensor readings. This isn't redfish compliant and relies on secure websockets however.
> 
> https://github.com/openbmc/docs/blob/master/rest-api.md#event-subscription-protocol
> 
>>
>> I've been informed there are actually 3:
>>
>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749
>>
>> Added Appu to this conversation as well.
>>
>>
>>>>
>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/22257
>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24357
>>>>
>>>> I would like to see one or both of them merged.  Neither seem to
>>>> have any support….there is a single +1 on the latter review from
>>>> Shawn McCarney.  If you support one of these designs, please go
>>>> review it and indicate your support.
>>>
>>> It looks like Kun has +1ed Piotr's design as well, not sure if that
>>> means we can go with one design?
>>>
>>>>
>>>> I also can’t figure out what the path forward for OpenBMC is.  Maybe
>>>> to start with, we could all level set on where we are at with our
>>>> thinking:
>>>>
>>>> Kun: How far along are you in the implementation of your proposal?
>>>> Piotr: How far along are you in the implementation of your proposal?
>>>> James: OpenBMC can support one or both proposals.  If we support
>>>> both, this means multiple implementations for the same thing in
>>>> bmcweb.  One that gets data from dbus objects, and another that gets
>>>> it from librrd.  As the maintainer of bmcweb are you open to
>>>> accepting one or both of these options?
>>>
>>> With 0 previous knowledge, I would suggest using a way that works
>>> with previous OBMC practices, and that is using dbus. If there has to
>>> be 2 separate designs, then if both produce the same d-bus
>>> interfaces, it shouldn't matter to bmcweb which one is being used.
>>> That being said, if this doesn't work for some reason, we've used
>>> compile switches in the past, although this would be the least preferable option.
>>> Truthfully I haven't looked at these designs yet as I've only
>>> recently taken a larger role in bmcweb reviews, so I'll try to look at them soon.
>>>
>>>
>>>>
>>>> Without any discussion and resolution, I’m afraid both of these
>>>> proposals are just going to sit here, unmerged, indefinitely.
>>>>
>>>> thx - brad
>>>>

Re: multiple telemetry designs

[Brad Bishop]

> On Oct 24, 2019, at 12:14 PM, James Feist <james.feist@linux.intel.com> wrote:
> 
> On 10/24/19 1:48 AM, Matuszczak, Piotr wrote:
>> 
>> As for the implementation, we have monitoring service and Redfish telemetry service implemented (currently without triggers support), however it require some refactoring to be production code quality.
> 
> Is there a WIP review somewhere?

I have this same question.  I wouldn’t mind being able to have a look.  Is there anything you need from the community to be able to do your work in the open?

RE: multiple telemetry designs

[Matuszczak, Piotr]

Hi,

Unfortunately, this is only a PoC quality, thus there is no open repository with the code.
We plan to do the OpenBMC implementation in the open repository. Will it work for you?

-----Original Message-----
From: Brad Bishop [mailto:bradleyb@fuzziesquirrel.com]
Sent: Friday, October 25, 2019 2:08 PM
To: Matuszczak, Piotr <piotr.matuszczak@intel.com>
Cc: Justin Thaler <thalerj@linux.vnet.ibm.com>; kunyi@google.com; apparao.puli@linux.intel.com; neladk@microsoft.com; openbmc@lists.ozlabs.org; Mihm, James <james.mihm@intel.com>; James Feist <james.feist@linux.intel.com>
Subject: Re: multiple telemetry designs

> On Oct 24, 2019, at 12:14 PM, James Feist <james.feist@linux.intel.com> wrote:
>
> On 10/24/19 1:48 AM, Matuszczak, Piotr wrote:
>>
>> As for the implementation, we have monitoring service and Redfish telemetry service implemented (currently without triggers support), however it require some refactoring to be production code quality.
>
> Is there a WIP review somewhere?

I have this same question.  I wouldn’t mind being able to have a look.  Is there anything you need from the community to be able to do your work in the open?

Re: multiple telemetry designs

[Brad Bishop]

> On Oct 25, 2019, at 12:46 PM, Matuszczak, Piotr <piotr.matuszczak@intel.com> wrote:
> 
> Hi,
> 
> Unfortunately, this is only a PoC quality, thus there is no open repository with the code.

Where did you get the idea that PoC code can’t be shared with the community?

> We plan to do the OpenBMC implementation in the open repository. Will it work for you?

Not really.  I can’t make you do anything but this is not how open source software development works.  If Intel develops the code and then shares it, that is not the OpenBMC implementation that is the Intel implementation.

thx -brad

RE: multiple telemetry designs

[Matuszczak, Piotr]

The PoC was done internally, thus it wasn't opened. We plan to use its code as some reference implementing Monitoring Service and Redfish telemetry service. 
We don't want to do all the development internally and then show it as, in fact, Intel's implementation. I would like to make the code opened from the very beginning. 

-----Original Message-----
From: Brad Bishop [mailto:bradleyb@fuzziesquirrel.com] 
Sent: Friday, October 25, 2019 6:59 PM
To: Matuszczak, Piotr <piotr.matuszczak@intel.com>
Cc: Justin Thaler <thalerj@linux.vnet.ibm.com>; openbmc@lists.ozlabs.org; neladk@microsoft.com; James Feist <james.feist@linux.intel.com>; Mihm, James <james.mihm@intel.com>; apparao.puli@linux.intel.com
Subject: Re: multiple telemetry designs



> On Oct 25, 2019, at 12:46 PM, Matuszczak, Piotr <piotr.matuszczak@intel.com> wrote:
> 
> Hi,
> 
> Unfortunately, this is only a PoC quality, thus there is no open repository with the code.

Where did you get the idea that PoC code can’t be shared with the community?

> We plan to do the OpenBMC implementation in the open repository. Will it work for you?

Not really.  I can’t make you do anything but this is not how open source software development works.  If Intel develops the code and then shares it, that is not the OpenBMC implementation that is the Intel implementation.

thx -brad

Re: multiple telemetry designs

[Brad Bishop]

> On Oct 28, 2019, at 12:35 PM, Matuszczak, Piotr <piotr.matuszczak@intel.com> wrote:
> 
> I would like to make the code opened from the very beginning. 

Glad to hear it - that sounds like the best way to me :-)

FWIW, whenever you are ready to share it, I’d still like to see whatever code Intel has for the monitoring service.  It will help me understand your design better.  It is fine if it has bugs or it isn’t polished.  Thanks Piotr.

-brad

Re: multiple telemetry designs

[vishwa]

There is also this version from Dell: 
https://gerrit.openbmc-project.xyz/#/c/openbmc/docs/+/23758/. Was this 
considered in this discussion ?.

Also, from IBM's standpoint, Justin Thaler was mentioning that we wanted 
a "true subscription" model, in that, clients can pick and chose the 
specific sensors.

Justin: Could you add here please ?

!! Vishwa !!

On 10/28/19 10:12 PM, Brad Bishop wrote:
>> On Oct 28, 2019, at 12:35 PM, Matuszczak, Piotr <piotr.matuszczak@intel.com> wrote:
>>
>> I would like to make the code opened from the very beginning.
> Glad to hear it - that sounds like the best way to me :-)
>
> FWIW, whenever you are ready to share it, I’d still like to see whatever code Intel has for the monitoring service.  It will help me understand your design better.  It is fine if it has bugs or it isn’t polished.  Thanks Piotr.
>
> -brad

RE: multiple telemetry designs

[Matuszczak, Piotr]

Hi,

I looked at this design briefly and it seems to be focusing on Redfish Telemetry Service implementation, which our design (https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24357) also covers. Dell's design assumes using collecd for gathering sensor readings.

-----Original Message-----
From: vishwa [mailto:vishwa@linux.vnet.ibm.com]
Sent: Tuesday, November 5, 2019 8:31 AM
To: Brad Bishop <bradleyb@fuzziesquirrel.com>
Cc: Mihm, James <james.mihm@intel.com>; Justin Thaler <thalerj@linux.vnet.ibm.com>; openbmc@lists.ozlabs.org; neladk@microsoft.com; James Feist <james.feist@linux.intel.com>; apparao.puli@linux.intel.com; Matuszczak, Piotr <piotr.matuszczak@intel.com>
Subject: Re: multiple telemetry designs

There is also this version from Dell:
https://gerrit.openbmc-project.xyz/#/c/openbmc/docs/+/23758/. Was this considered in this discussion ?.

Also, from IBM's standpoint, Justin Thaler was mentioning that we wanted a "true subscription" model, in that, clients can pick and chose the specific sensors.

Justin: Could you add here please ?

!! Vishwa !!

On 10/28/19 10:12 PM, Brad Bishop wrote:
>> On Oct 28, 2019, at 12:35 PM, Matuszczak, Piotr <piotr.matuszczak@intel.com> wrote:
>>
>> I would like to make the code opened from the very beginning.
> Glad to hear it - that sounds like the best way to me :-)
>
> FWIW, whenever you are ready to share it, I’d still like to see whatever code Intel has for the monitoring service.  It will help me understand your design better.  It is fine if it has bugs or it isn’t polished.  Thanks Piotr.
>
> -brad

Re: multiple telemetry designs

[vishwa]

Thanks.

So, looks like we are getting zeroed in on Intel's proposal ?. I see Kun 
approving Intel version.

Paul: Did you have anything ?

!! Vishwa !!

On 11/5/19 2:26 PM, Matuszczak, Piotr wrote:
> Hi,
>
> I looked at this design briefly and it seems to be focusing on Redfish Telemetry Service implementation, which our design (https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24357) also covers. Dell's design assumes using collecd for gathering sensor readings.

> -----Original Message-----
> From: vishwa [mailto:vishwa@linux.vnet.ibm.com]
> Sent: Tuesday, November 5, 2019 8:31 AM
> To: Brad Bishop <bradleyb@fuzziesquirrel.com>
> Cc: Mihm, James <james.mihm@intel.com>; Justin Thaler <thalerj@linux.vnet.ibm.com>; openbmc@lists.ozlabs.org; neladk@microsoft.com; James Feist <james.feist@linux.intel.com>; apparao.puli@linux.intel.com; Matuszczak, Piotr <piotr.matuszczak@intel.com>
> Subject: Re: multiple telemetry designs
>
> There is also this version from Dell:
> https://gerrit.openbmc-project.xyz/#/c/openbmc/docs/+/23758/. Was this considered in this discussion ?.
>
> Also, from IBM's standpoint, Justin Thaler was mentioning that we wanted a "true subscription" model, in that, clients can pick and chose the specific sensors.
>
> Justin: Could you add here please ?
>
> !! Vishwa !!
>
> On 10/28/19 10:12 PM, Brad Bishop wrote:
>>> On Oct 28, 2019, at 12:35 PM, Matuszczak, Piotr <piotr.matuszczak@intel.com> wrote:
>>>
>>> I would like to make the code opened from the very beginning.
>> Glad to hear it - that sounds like the best way to me :-)
>>
>> FWIW, whenever you are ready to share it, I’d still like to see whatever code Intel has for the monitoring service.  It will help me understand your design better.  It is fine if it has bugs or it isn’t polished.  Thanks Piotr.
>>
>> -brad

Re: multiple telemetry designs

[Justin Thaler]

On 11/5/19 10:58 AM, vishwa wrote:
> Thanks.
> 
> So, looks like we are getting zeroed in on Intel's proposal ?. I see Kun 
> approving Intel version.
> 
> Paul: Did you have anything ?
> 
> !! Vishwa !!
> 
> On 11/5/19 2:26 PM, Matuszczak, Piotr wrote:
>> Hi,
>>
>> I looked at this design briefly and it seems to be focusing on Redfish 
>> Telemetry Service implementation, which our design 
>> (https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24357) also 
>> covers. Dell's design assumes using collecd for gathering sensor 
>> readings.
> 
>> -----Original Message-----
>> From: vishwa [mailto:vishwa@linux.vnet.ibm.com]
>> Sent: Tuesday, November 5, 2019 8:31 AM
>> To: Brad Bishop <bradleyb@fuzziesquirrel.com>
>> Cc: Mihm, James <james.mihm@intel.com>; Justin Thaler 
>> <thalerj@linux.vnet.ibm.com>; openbmc@lists.ozlabs.org; 
>> neladk@microsoft.com; James Feist <james.feist@linux.intel.com>; 
>> apparao.puli@linux.intel.com; Matuszczak, Piotr 
>> <piotr.matuszczak@intel.com>
>> Subject: Re: multiple telemetry designs
>>
>> There is also this version from Dell:
>> https://gerrit.openbmc-project.xyz/#/c/openbmc/docs/+/23758/. Was this 
>> considered in this discussion ?.
>>
>> Also, from IBM's standpoint, Justin Thaler was mentioning that we 
>> wanted a "true subscription" model, in that, clients can pick and 
>> chose the specific sensors.
>>
>> Justin: Could you add here please ?
Sorry for the slow response. Piotr was kind enough to walk me through 
how the proposal works and it does allow for a true subscription model. 
I still have a to do to determine how much data we will be using with 
this model so I can understand how well it scales. This is a concern for 
us as we are shifting from receiving sensor updates in an "on-change" 
model to updates every second, regardless of change. There's also 
changes in the data format that's sent, which will likely make this less 
of a concern.

Thanks,
Justin

>>
>> !! Vishwa !!
>>
>> On 10/28/19 10:12 PM, Brad Bishop wrote:
>>>> On Oct 28, 2019, at 12:35 PM, Matuszczak, Piotr 
>>>> <piotr.matuszczak@intel.com> wrote:
>>>>
>>>> I would like to make the code opened from the very beginning.
>>> Glad to hear it - that sounds like the best way to me :-)
>>>
>>> FWIW, whenever you are ready to share it, I’d still like to see 
>>> whatever code Intel has for the monitoring service.  It will help me 
>>> understand your design better.  It is fine if it has bugs or it isn’t 
>>> polished.  Thanks Piotr.
>>>
>>> -brad

Re: multiple telemetry designs

[Paul Vancil]

[-- Attachment #1: Type: text/plain, Size: 2855 bytes --]

Paul here.
I also commented and approved the intel design.
It looks good to me.


On Tue, Nov 12, 2019 at 9:36 AM Justin Thaler <thalerj@linux.vnet.ibm.com>
wrote:

>
>
> On 11/5/19 10:58 AM, vishwa wrote:
> > Thanks.
> >
> > So, looks like we are getting zeroed in on Intel's proposal ?. I see Kun
> > approving Intel version.
> >
> > Paul: Did you have anything ?
> >
> > !! Vishwa !!
> >
> > On 11/5/19 2:26 PM, Matuszczak, Piotr wrote:
> >> Hi,
> >>
> >> I looked at this design briefly and it seems to be focusing on Redfish
> >> Telemetry Service implementation, which our design
> >> (https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24357) also
> >> covers. Dell's design assumes using collecd for gathering sensor
> >> readings.
> >
> >> -----Original Message-----
> >> From: vishwa [mailto:vishwa@linux.vnet.ibm.com]
> >> Sent: Tuesday, November 5, 2019 8:31 AM
> >> To: Brad Bishop <bradleyb@fuzziesquirrel.com>
> >> Cc: Mihm, James <james.mihm@intel.com>; Justin Thaler
> >> <thalerj@linux.vnet.ibm.com>; openbmc@lists.ozlabs.org;
> >> neladk@microsoft.com; James Feist <james.feist@linux.intel.com>;
> >> apparao.puli@linux.intel.com; Matuszczak, Piotr
> >> <piotr.matuszczak@intel.com>
> >> Subject: Re: multiple telemetry designs
> >>
> >> There is also this version from Dell:
> >> https://gerrit.openbmc-project.xyz/#/c/openbmc/docs/+/23758/. Was this
> >> considered in this discussion ?.
> >>
> >> Also, from IBM's standpoint, Justin Thaler was mentioning that we
> >> wanted a "true subscription" model, in that, clients can pick and
> >> chose the specific sensors.
> >>
> >> Justin: Could you add here please ?
> Sorry for the slow response. Piotr was kind enough to walk me through
> how the proposal works and it does allow for a true subscription model.
> I still have a to do to determine how much data we will be using with
> this model so I can understand how well it scales. This is a concern for
> us as we are shifting from receiving sensor updates in an "on-change"
> model to updates every second, regardless of change. There's also
> changes in the data format that's sent, which will likely make this less
> of a concern.
>
> Thanks,
> Justin
>
> >>
> >> !! Vishwa !!
> >>
> >> On 10/28/19 10:12 PM, Brad Bishop wrote:
> >>>> On Oct 28, 2019, at 12:35 PM, Matuszczak, Piotr
> >>>> <piotr.matuszczak@intel.com> wrote:
> >>>>
> >>>> I would like to make the code opened from the very beginning.
> >>> Glad to hear it - that sounds like the best way to me :-)
> >>>
> >>> FWIW, whenever you are ready to share it, I’d still like to see
> >>> whatever code Intel has for the monitoring service.  It will help me
> >>> understand your design better.  It is fine if it has bugs or it isn’t
> >>> polished.  Thanks Piotr.
> >>>
> >>> -brad
>

[-- Attachment #2: Type: text/html, Size: 4637 bytes --]

RE: multiple telemetry designs

[Matuszczak, Piotr]

Hi, 

Thank you all for reviews and comments. Is there anything else required in order to push this design to the Github repository?

Regards
Piotr

From: Paul Vancil [mailto:pwvancil@gmail.com] 
Sent: Tuesday, November 12, 2019 3:39 PM
To: Justin Thaler <thalerj@linux.vnet.ibm.com>
Cc: Brad Bishop <bradleyb@fuzziesquirrel.com>; James Feist <james.feist@linux.intel.com>; Matuszczak, Piotr <piotr.matuszczak@intel.com>; Mihm, James <james.mihm@intel.com>; apparao.puli@linux.intel.com; neladk@microsoft.com; openbmc@lists.ozlabs.org; vishwa <vishwa@linux.vnet.ibm.com>
Subject: Re: multiple telemetry designs

Paul here. 
I also commented and approved the intel design. 
It looks good to me. 


On Tue, Nov 12, 2019 at 9:36 AM Justin Thaler <thalerj@linux.vnet.ibm.com> wrote:


On 11/5/19 10:58 AM, vishwa wrote:
> Thanks.
> 
> So, looks like we are getting zeroed in on Intel's proposal ?. I see Kun 
> approving Intel version.
> 
> Paul: Did you have anything ?
> 
> !! Vishwa !!
> 
> On 11/5/19 2:26 PM, Matuszczak, Piotr wrote:
>> Hi,
>>
>> I looked at this design briefly and it seems to be focusing on Redfish 
>> Telemetry Service implementation, which our design 
>> (https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24357) also 
>> covers. Dell's design assumes using collecd for gathering sensor 
>> readings.
> 
>> -----Original Message-----
>> From: vishwa [mailto:vishwa@linux.vnet.ibm.com]
>> Sent: Tuesday, November 5, 2019 8:31 AM
>> To: Brad Bishop <bradleyb@fuzziesquirrel.com>
>> Cc: Mihm, James <james.mihm@intel.com>; Justin Thaler 
>> <thalerj@linux.vnet.ibm.com>; openbmc@lists.ozlabs.org; 
>> neladk@microsoft.com; James Feist <james.feist@linux.intel.com>; 
>> apparao.puli@linux.intel.com; Matuszczak, Piotr 
>> <piotr.matuszczak@intel.com>
>> Subject: Re: multiple telemetry designs
>>
>> There is also this version from Dell:
>> https://gerrit.openbmc-project.xyz/#/c/openbmc/docs/+/23758/. Was this 
>> considered in this discussion ?.
>>
>> Also, from IBM's standpoint, Justin Thaler was mentioning that we 
>> wanted a "true subscription" model, in that, clients can pick and 
>> chose the specific sensors.
>>
>> Justin: Could you add here please ?
Sorry for the slow response. Piotr was kind enough to walk me through 
how the proposal works and it does allow for a true subscription model. 
I still have a to do to determine how much data we will be using with 
this model so I can understand how well it scales. This is a concern for 
us as we are shifting from receiving sensor updates in an "on-change" 
model to updates every second, regardless of change. There's also 
changes in the data format that's sent, which will likely make this less 
of a concern.

Thanks,
Justin

>>
>> !! Vishwa !!
>>
>> On 10/28/19 10:12 PM, Brad Bishop wrote:
>>>> On Oct 28, 2019, at 12:35 PM, Matuszczak, Piotr 
>>>> <piotr.matuszczak@intel.com> wrote:
>>>>
>>>> I would like to make the code opened from the very beginning.
>>> Glad to hear it - that sounds like the best way to me :-)
>>>
>>> FWIW, whenever you are ready to share it, I’d still like to see 
>>> whatever code Intel has for the monitoring service.  It will help me 
>>> understand your design better.  It is fine if it has bugs or it isn’t 
>>> polished.  Thanks Piotr.
>>>
>>> -brad

Re: multiple telemetry designs

[Shawn McCarney]

I've reviewed both designs, although I cannot say I understand them both 
in depth.

With that disclaimer, here is my 2 cents:

* Both proposals are thoughtful with a lot work put into them.

* bmcweb has a lot of a sensor code that is quite complex that is 
dependent on the current D-Bus sensors and associations.  It would 
require a lot of work and re-testing to ensure a different interface to 
sensor data doesn't break current systems.  The code would be even more 
complex if it had to support two different sensor data interfaces.

* There are sensor readings that cannot be collected by reading files in 
the file system.  Some are collected by direct I2C reads or other 
methods.  If my surface understanding of collectd is correct, plug-ins 
would need to be written to handle these "non-file" sensors.

* For the reasons above, I'd prefer to see D-Bus continue to be the 
"public API" to sensor data.  D-Bus is the central data sharing 
repository on the OpenBMC.  How the sensor data gets on D-Bus is 
implementation detail and can vary by system and by project.  It can be 
obtained by hwmon, collectd, and many other ways.  As long as it is 
published on D-Bus, other applications (like bmcweb) can easily consume it.

* It sounds like the RRD format would be an efficient way to store 
sensor data.  I do worry about the space and CPU required to store 
telemetry data.  The OpenBMC stack is going to be used on some big 
servers, and they are going to have a large number of sensors.

* Could the two proposals be merged, with D-Bus providing the public API 
to the data?  Maybe something like the following?  1) Continue to store 
current sensor values on D-Bus using the existing architecture.  Sensor 
values come from a variety of sources.  2) An application obtains 
current sensor values from D-Bus and stores them with timestamps in RRD 
to provide efficient history/telemetry.  3) A new D-Bus interface/method 
is created to obtain the history/telemetry data.  4) bmcweb uses the 
current D-Bus interfaces for the Sensor URIs (as it does today) and uses 
the new D-Bus interface/method for Telemetry URIs?

Thanks,

Shawn

Re: multiple telemetry designs

[Kun Yi]

[-- Attachment #1: Type: text/plain, Size: 3160 bytes --]

On Thu, Oct 24, 2019 at 10:13 AM Shawn McCarney <shawnmm@linux.vnet.ibm.com>
wrote:

> I've reviewed both designs, although I cannot say I understand them both
> in depth.
>
> With that disclaimer, here is my 2 cents:
>
> * Both proposals are thoughtful with a lot work put into them.
>
> * bmcweb has a lot of a sensor code that is quite complex that is
> dependent on the current D-Bus sensors and associations.  It would
> require a lot of work and re-testing to ensure a different interface to
> sensor data doesn't break current systems.  The code would be even more
> complex if it had to support two different sensor data interfaces.
>
> * There are sensor readings that cannot be collected by reading files in
> the file system.  Some are collected by direct I2C reads or other
> methods.  If my surface understanding of collectd is correct, plug-ins
> would need to be written to handle these "non-file" sensors.
>
> * For the reasons above, I'd prefer to see D-Bus continue to be the
> "public API" to sensor data.  D-Bus is the central data sharing
> repository on the OpenBMC.  How the sensor data gets on D-Bus is
> implementation detail and can vary by system and by project.  It can be
> obtained by hwmon, collectd, and many other ways.  As long as it is
> published on D-Bus, other applications (like bmcweb) can easily consume it.
>
> * It sounds like the RRD format would be an efficient way to store
> sensor data.  I do worry about the space and CPU required to store
> telemetry data.  The OpenBMC stack is going to be used on some big
> servers, and they are going to have a large number of sensors.
>
> * Could the two proposals be merged, with D-Bus providing the public API
> to the data?  Maybe something like the following?  1) Continue to store
> current sensor values on D-Bus using the existing architecture.  Sensor
> values come from a variety of sources.  2) An application obtains
> current sensor values from D-Bus and stores them with timestamps in RRD
> to provide efficient history/telemetry.  3) A new D-Bus interface/method
> is created to obtain the history/telemetry data.  4) bmcweb uses the
> current D-Bus interfaces for the Sensor URIs (as it does today) and uses
> the new D-Bus interface/method for Telemetry URIs?
>
> Thanks,
>
> Shawn
>
>
(author of the collectd/RRD based design here)
First of all, I have been silent on the mailing list for a while, without
any progress on collectd. There are some fires that I need to put out
first, unfortunately :(

I have discussed with Piotr in the telemetry meeting. Basically we'd like
to rephrase it as this: Piotr's design doesn't prevent future extension
such as using collectd/rrdtool as a backend to provide telemetry data, and
I reviewed the Redfish API that the design would provide, which LGTM.
Therefore I +1'ed Piotr's design, given that there is already concrete work
behind it, and collectd didn't work for his requirements.

To be able to merge the designs, either Bmcweb can use RRD library or
collectd/librrd can talk D-Bus, which is some work but not insurmountable.
Piotr maybe you want to call that out explicitly in your design doc?

Regards,
Kun

[-- Attachment #2: Type: text/html, Size: 3883 bytes --]

RE: multiple telemetry designs

[Neeraj Ladkani]

[-- Attachment #1: Type: text/plain, Size: 3700 bytes --]

This is great discussion. can we have a deep dive on this during next telemetry sync up call ?

Neeraj

From: Kun Yi <kunyi@google.com>
Sent: Thursday, October 24, 2019 10:27 AM
To: Shawn McCarney <shawnmm@linux.vnet.ibm.com>
Cc: Brad Bishop <bradleyb@fuzziesquirrel.com>; James Feist <james.feist@linux.intel.com>; piotr.matuszczak@intel.com; thalerj@linux.vnet.ibm.com; OpenBMC Maillist <openbmc@lists.ozlabs.org>; james.mihm@intel.com; Neeraj Ladkani <neladk@microsoft.com>
Subject: Re: multiple telemetry designs



On Thu, Oct 24, 2019 at 10:13 AM Shawn McCarney <shawnmm@linux.vnet.ibm.com<mailto:shawnmm@linux.vnet.ibm.com>> wrote:
I've reviewed both designs, although I cannot say I understand them both
in depth.

With that disclaimer, here is my 2 cents:

* Both proposals are thoughtful with a lot work put into them.

* bmcweb has a lot of a sensor code that is quite complex that is
dependent on the current D-Bus sensors and associations.  It would
require a lot of work and re-testing to ensure a different interface to
sensor data doesn't break current systems.  The code would be even more
complex if it had to support two different sensor data interfaces.

* There are sensor readings that cannot be collected by reading files in
the file system.  Some are collected by direct I2C reads or other
methods.  If my surface understanding of collectd is correct, plug-ins
would need to be written to handle these "non-file" sensors.

* For the reasons above, I'd prefer to see D-Bus continue to be the
"public API" to sensor data.  D-Bus is the central data sharing
repository on the OpenBMC.  How the sensor data gets on D-Bus is
implementation detail and can vary by system and by project.  It can be
obtained by hwmon, collectd, and many other ways.  As long as it is
published on D-Bus, other applications (like bmcweb) can easily consume it.

* It sounds like the RRD format would be an efficient way to store
sensor data.  I do worry about the space and CPU required to store
telemetry data.  The OpenBMC stack is going to be used on some big
servers, and they are going to have a large number of sensors.

* Could the two proposals be merged, with D-Bus providing the public API
to the data?  Maybe something like the following?  1) Continue to store
current sensor values on D-Bus using the existing architecture.  Sensor
values come from a variety of sources.  2) An application obtains
current sensor values from D-Bus and stores them with timestamps in RRD
to provide efficient history/telemetry.  3) A new D-Bus interface/method
is created to obtain the history/telemetry data.  4) bmcweb uses the
current D-Bus interfaces for the Sensor URIs (as it does today) and uses
the new D-Bus interface/method for Telemetry URIs?

Thanks,

Shawn

(author of the collectd/RRD based design here)
First of all, I have been silent on the mailing list for a while, without any progress on collectd. There are some fires that I need to put out first, unfortunately :(

I have discussed with Piotr in the telemetry meeting. Basically we'd like to rephrase it as this: Piotr's design doesn't prevent future extension such as using collectd/rrdtool as a backend to provide telemetry data, and I reviewed the Redfish API that the design would provide, which LGTM. Therefore I +1'ed Piotr's design, given that there is already concrete work behind it, and collectd didn't work for his requirements.

To be able to merge the designs, either Bmcweb can use RRD library or collectd/librrd can talk D-Bus, which is some work but not insurmountable. Piotr maybe you want to call that out explicitly in your design doc?

Regards,
Kun

[-- Attachment #2: Type: text/html, Size: 7148 bytes --]

Re: multiple telemetry designs

[Brad Bishop]

> On Oct 24, 2019, at 1:31 PM, Neeraj Ladkani <neladk@microsoft.com> wrote:
> 
> This is great discussion. can we have a deep dive on this during next telemetry sync up call ?

FWIW you will have more engagement and a better end result if you have the discussion here on the list.  That way everyone can participate (not everyone can attend meetings) and the conversation is automatically recorded and archived.

-brad

Re: multiple telemetry designs

[Brad Bishop]

> On Oct 24, 2019, at 1:27 PM, Kun Yi <kunyi@google.com> wrote:
> 
> (author of the collectd/RRD based design here)
> First of all, I have been silent on the mailing list for a while, without any progress on collectd. There are some fires that I need to put out first, unfortunately :(
> 
> I have discussed with Piotr in the telemetry meeting. Basically we'd like to rephrase it as this: Piotr's design doesn't prevent future extension such as using collectd/rrdtool as a backend to provide telemetry data, and I reviewed the Redfish API that the design would provide, which LGTM. Therefore I +1'ed Piotr's design, given that there is already concrete work behind it, and collectd didn't work for his requirements.

That’s great.  This is exactly the sort of consensus I was looking for!  Thanks for documenting it Kun (apologies if this was already common knowledge to the telemetry workgroup).

I’ll re-state to make sure I have it right - we’ll merge Piotr’s design now, and leave yours open for the time being, until we sort out how to have a collectd based back-end live side-by-side with the custom monitor service from Piotr.

> To be able to merge the designs, either Bmcweb can use RRD library or collectd/librrd can talk D-Bus, which is some work but not insurmountable. Piotr maybe you want to call that out explicitly in your design doc?
> 
> Regards,
> Kun

Thanks Piotr, James, Shawn, Kun, Justin, and Neeraj for the replies!

-brad

Re: multiple telemetry designs

[Brad Bishop]

> On Oct 24, 2019, at 1:13 PM, Shawn McCarney <shawnmm@linux.vnet.ibm.com> wrote:
> 
> * Could the two proposals be merged, with D-Bus providing the public API to the data?  Maybe something like the following?  1) Continue to store current sensor values on D-Bus using the existing architecture.  Sensor values come from a variety of sources.  2) An application obtains current sensor values from D-Bus and stores them with timestamps in RRD to provide efficient history/telemetry.  3) A new D-Bus interface/method is created to obtain the history/telemetry data.  4) bmcweb uses the current D-Bus interfaces for the Sensor URIs (as it does today) and uses the new D-Bus interface/method for Telemetry URIs?

I think there is room for a merged/split design.  There are two use cases to consider though while we formulate that.

1 - I want to expose sensor data with collectd (collectd-binary).
2 - I want to expose sensor data with bmcweb (Redfish).

I would suggest that for #1, dbus be avoided and go right to the source of the data, whatever that may be.  That would be more performant and it avoids dependencies on OpenBMC software which means collectd plugins that are written would have a chance at being accepted in upstream collectd.  This would be irrespective of how #2 is designed and implemented.

I don’t think I need #1 though.  Does anyone plan on doing #1 on their BMC?  When I originally suggested collectd, it was because I thought:

1 - collectd is a preexisting implementation of the proposed custom monitoring service...

therefore

2 - it would save effort and be a smaller maintenance burden to use pre-existing software rather than writing it from scratch.

It is certainly possible though that collectd can’t be made to fit the bill or I am imagining more of an overlap between the custom monitoring service and a collectd based one than there really is.

thx - brad

RE: multiple telemetry designs

[Matuszczak, Piotr]

Hi, 

Our design is tailored to support the Redfish Telemetry Service, but it is not limited to it and either, the telemetry configuration and presentation is done via the Redfish. Also, the monitoring service is designed to support Redfish triggers leveraging the Redfish Event Service for sending events. 

If the collectd is to exist independently of Redfish and D-Bus it is possible to use both solutions. I have some questions, though:
1. How the telemetry gathering will be configured by the user, when collectd is used?
2. What is the use case for storing historical readings in the BMC, using non-volatile storage (or did I misunderstood the rrd files)? 

As for merging the two proposals, the common D-Bus API for the telemetry back-end (either collectd or monitoring service) was what I thought about at first. Of course, the back-end will have to support metric report and triggers management, because the bmcweb is only the presentation layer (Redfish Telemetry Service) for the telemetry data. 

Regards
Piotr

-----Original Message-----
From: Brad Bishop [mailto:bradleyb@fuzziesquirrel.com] 
Sent: Friday, October 25, 2019 2:59 PM
To: OpenBMC Maillist <openbmc@lists.ozlabs.org>
Cc: James Feist <james.feist@linux.intel.com>; Matuszczak, Piotr <piotr.matuszczak@intel.com>; Kun Yi <kunyi@google.com>; Justin Thaler <thalerj@linux.vnet.ibm.com>; Mihm, James <james.mihm@intel.com>; Neeraj Ladkani <neladk@microsoft.com>; Shawn McCarney <shawnmm@linux.vnet.ibm.com>
Subject: Re: multiple telemetry designs


> On Oct 24, 2019, at 1:13 PM, Shawn McCarney <shawnmm@linux.vnet.ibm.com> wrote:
> 
> * Could the two proposals be merged, with D-Bus providing the public API to the data?  Maybe something like the following?  1) Continue to store current sensor values on D-Bus using the existing architecture.  Sensor values come from a variety of sources.  2) An application obtains current sensor values from D-Bus and stores them with timestamps in RRD to provide efficient history/telemetry.  3) A new D-Bus interface/method is created to obtain the history/telemetry data.  4) bmcweb uses the current D-Bus interfaces for the Sensor URIs (as it does today) and uses the new D-Bus interface/method for Telemetry URIs?

I think there is room for a merged/split design.  There are two use cases to consider though while we formulate that.

1 - I want to expose sensor data with collectd (collectd-binary).
2 - I want to expose sensor data with bmcweb (Redfish).

I would suggest that for #1, dbus be avoided and go right to the source of the data, whatever that may be.  That would be more performant and it avoids dependencies on OpenBMC software which means collectd plugins that are written would have a chance at being accepted in upstream collectd.  This would be irrespective of how #2 is designed and implemented.

I don’t think I need #1 though.  Does anyone plan on doing #1 on their BMC?  When I originally suggested collectd, it was because I thought:

1 - collectd is a preexisting implementation of the proposed custom monitoring service...

therefore

2 - it would save effort and be a smaller maintenance burden to use pre-existing software rather than writing it from scratch.

It is certainly possible though that collectd can’t be made to fit the bill or I am imagining more of an overlap between the custom monitoring service and a collectd based one than there really is.

thx - brad

Re: multiple telemetry designs

[Brad Bishop]

Hi Piotr

> On Oct 25, 2019, at 11:08 AM, Matuszczak, Piotr <piotr.matuszczak@intel.com> wrote:
> 
> Hi, 
> 
> Our design is tailored to support the Redfish Telemetry Service, but it is not limited to it and either, the telemetry configuration and presentation is done via the Redfish. Also, the monitoring service is designed to support Redfish triggers leveraging the Redfish Event Service for sending events. 

I wasn’t trying to suggest that your design was inflexible.  I apologize if that seemed to be the case.

> If the collectd is to exist independently of Redfish and D-Bus it is possible to use both solutions. I have some questions, though:
> 1. How the telemetry gathering will be configured by the user, when collectd is used?

-OEM extension to any management protocol
-ssh + collectd config file
-no configuration (baked into the image - think per-system images deployed to 10000 systems).

but I’m just blasting ideas - I don’t have any need for running the collectd network output plugin.

> 2. What is the use case for storing historical readings in the BMC, using non-volatile storage (or did I misunderstood the rrd files)? 

Clients can have have transient disconnects and not miss any sensor readings.

> As for merging the two proposals, the common D-Bus API for the telemetry back-end (either collectd or monitoring service) was what I thought about at first. Of course, the back-end will have to support metric report and triggers management, because the bmcweb is only the presentation layer (Redfish Telemetry Service) for the telemetry data. 

That makes sense.  And I assume you believe that it is more work to use collectd as that back-end than writing one from scratch?  I guess at this point it is a little moot - from the other thread it sounds like you have the code mostly done already anyway...

thx - brad

RE: multiple telemetry designs

[Matuszczak, Piotr]

Hi Brad, 

Thank you for your answers. I believe, that there was some misunderstanding. I've said, that there is PoC implementation, but I've also said, that the code has to be rewritten. Another thing is, that Monitoring Service is relatively simple and I didn't assume that writing our back-end (the Monitoirng Service) will be less work than using collectd. Our main concern was the collectd storage,  performance requirements and Redfish compatibility. 

I will incorporate in my design the proposal to use collectd as an alternative back-end for telemetry. 

Thank you
Piotr

-----Original Message-----
From: Brad Bishop [mailto:bradleyb@fuzziesquirrel.com] 
Sent: Friday, October 25, 2019 7:31 PM
To: Matuszczak, Piotr <piotr.matuszczak@intel.com>
Cc: OpenBMC Maillist <openbmc@lists.ozlabs.org>; James Feist <james.feist@linux.intel.com>; Kun Yi <kunyi@google.com>; Justin Thaler <thalerj@linux.vnet.ibm.com>; Mihm, James <james.mihm@intel.com>; Neeraj Ladkani <neladk@microsoft.com>; Shawn McCarney <shawnmm@linux.vnet.ibm.com>
Subject: Re: multiple telemetry designs

Hi Piotr

> On Oct 25, 2019, at 11:08 AM, Matuszczak, Piotr <piotr.matuszczak@intel.com> wrote:
> 
> Hi,
> 
> Our design is tailored to support the Redfish Telemetry Service, but it is not limited to it and either, the telemetry configuration and presentation is done via the Redfish. Also, the monitoring service is designed to support Redfish triggers leveraging the Redfish Event Service for sending events. 

I wasn’t trying to suggest that your design was inflexible.  I apologize if that seemed to be the case.

> If the collectd is to exist independently of Redfish and D-Bus it is possible to use both solutions. I have some questions, though:
> 1. How the telemetry gathering will be configured by the user, when collectd is used?

-OEM extension to any management protocol -ssh + collectd config file -no configuration (baked into the image - think per-system images deployed to 10000 systems).

but I’m just blasting ideas - I don’t have any need for running the collectd network output plugin.

> 2. What is the use case for storing historical readings in the BMC, using non-volatile storage (or did I misunderstood the rrd files)? 

Clients can have have transient disconnects and not miss any sensor readings.

> As for merging the two proposals, the common D-Bus API for the telemetry back-end (either collectd or monitoring service) was what I thought about at first. Of course, the back-end will have to support metric report and triggers management, because the bmcweb is only the presentation layer (Redfish Telemetry Service) for the telemetry data. 

That makes sense.  And I assume you believe that it is more work to use collectd as that back-end than writing one from scratch?  I guess at this point it is a little moot - from the other thread it sounds like you have the code mostly done already anyway...

thx - brad