Patch for telemetry design

Patch for telemetry design

[Matuszczak, Piotr]

Hi, 

I've uploaded patch for telemetry service design some time ago. I would like to ask you for review. 
https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/31690


Piotr Matuszczak
---------------------------------------------------------------------
Intel Technology Poland sp. z o.o. 
ul. Slowackiego 173, 80-298 Gdansk
KRS 101882
NIP 957-07-52-316

Re: Patch for telemetry design

[Justin Thaler]

Hi Piotr,
	I've taken a read through the design proposal as more of a client of 
the telemetry data. I like this proposal quite a bit as it is pretty 
clear. I had a couple of questions and given the broad level, I wasn't 
sure if gerrit was the right place for them.

On Timestamps, are the timestamps done per metric(sensor reading) or per 
report. This wasn't clear to me from the design proposal, and also it 
was hard to tell where the timestamp was being set.

 From the performance tests section, is there a limit on the number of 
sensors per report, seemingly it is 10, or was this done to simplify the 
performance testing?

The other general question I had was around the amount of data being 
transmitted. For instance, if you're getting reports on every sensor in 
the system (100s) the report(s) could be huge at scale. Would there be 
an option of considering compressed data like BEJSON? Or is this 
feedback that should go to the DMTF?

Thanks,
Justin Thaler
IBM RAS Engineer

On 6/16/20 4:31 AM, Matuszczak, Piotr wrote:
> Hi,
> 
> I've uploaded patch for telemetry service design some time ago. I would like to ask you for review.
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/31690
> 
> 
> Piotr Matuszczak
> ---------------------------------------------------------------------
> Intel Technology Poland sp. z o.o.
> ul. Slowackiego 173, 80-298 Gdansk
> KRS 101882
> NIP 957-07-52-316
> 

Re: Patch for telemetry design

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 475 bytes --]

On Tue, Jun 16, 2020 at 01:05:10PM -0500, Justin Thaler wrote:
> Would there be an option of considering compressed data like BEJSON? 

Did you mean BSON here or something else?  I saw BSON come through in
one design or interface review ~3 months back and I recommended we use
CBOR instead and that change was made.  CBOR is a standardized binary
representation of JSON that has wider support than BSON.

https://tools.ietf.org/html/rfc7049

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Patch for telemetry design

[Justin Thaler]

On 6/16/20 1:42 PM, Patrick Williams wrote:
> On Tue, Jun 16, 2020 at 01:05:10PM -0500, Justin Thaler wrote:
>> Would there be an option of considering compressed data like BEJSON?
> 
> Did you mean BSON here or something else?  I saw BSON come through in
> one design or interface review ~3 months back and I recommended we use
> CBOR instead and that change was made.  CBOR is a standardized binary
> representation of JSON that has wider support than BSON.
> 
> https://tools.ietf.org/html/rfc7049

I was just referring to Binary Encoded JSON generally. CBOR, would be a 
good suggestion I think. Thanks for helping to clarify!

> 

RE: Patch for telemetry design

[Matuszczak, Piotr]

Hi Justin, 

>I've taken a read through the design proposal as more of a client of the telemetry data. I like this proposal quite a bit as it is pretty clear. I had a couple of >questions and given the broad level, I wasn't sure if gerrit was the right place for them.

In fact, this patch is covering few changes that we've made to D-Bus API during the development and integration. 

>On Timestamps, are the timestamps done per metric(sensor reading) or per report. This wasn't clear to me from the design proposal, and also it was hard to >tell where the timestamp was being set.

Timestamps are gathered for both, metrics and report. Timestamp for metric is the timestamp of D-Bus senor update, while timestamp of the report is for the report update. It corresponds with the Redfish Metric Report resource, where both, individual metrics and the whole report have their update timestamps. 

>From the performance tests section, is there a limit on the number of sensors per report, seemingly it is 10, or was this done to simplify the performance >testing?

It was done because of limited sensors support on the test platform.

>The other general question I had was around the amount of data being transmitted. For instance, if you're getting reports on every sensor in the system >(100s) the report(s) could be huge at scale. Would there be an option of considering compressed data like BEJSON? Or is this feedback that should go to the >DMTF?

First of all, the max number of supported reports will be limited to around 50 due to limited BMC resources. As for the using compressed data, if we would like to compress data for the IPC (D-Bus) it is possible for us to implement any format we like as long as we document it in the design. As for the using compressed format for the Redfish Metric report it will require acceptance from the DMTF, because it will require changes in the schema. 
You were referring to using BSON on D-Bus or in Redfish Metric Report schema?

Piotr Matuszczak
---------------------------------------------------------------------
Intel Technology Poland sp. z o.o. 
ul. Slowackiego 173, 80-298 Gdansk
KRS 101882
NIP 957-07-52-316

-----Original Message-----
From: openbmc <openbmc-bounces+piotr.matuszczak=intel.com@lists.ozlabs.org> On Behalf Of Justin Thaler
Sent: Tuesday, June 16, 2020 8:05 PM
To: openbmc@lists.ozlabs.org
Subject: Re: Patch for telemetry design

Hi Piotr,
	I've taken a read through the design proposal as more of a client of the telemetry data. I like this proposal quite a bit as it is pretty clear. I had a couple of questions and given the broad level, I wasn't sure if gerrit was the right place for them.

On Timestamps, are the timestamps done per metric(sensor reading) or per report. This wasn't clear to me from the design proposal, and also it was hard to tell where the timestamp was being set.

 From the performance tests section, is there a limit on the number of sensors per report, seemingly it is 10, or was this done to simplify the performance testing?

The other general question I had was around the amount of data being transmitted. For instance, if you're getting reports on every sensor in the system (100s) the report(s) could be huge at scale. Would there be an option of considering compressed data like BEJSON? Or is this feedback that should go to the DMTF?

Thanks,
Justin Thaler
IBM RAS Engineer

On 6/16/20 4:31 AM, Matuszczak, Piotr wrote:
> Hi,
> 
> I've uploaded patch for telemetry service design some time ago. I would like to ask you for review.
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/31690
> 
> 
> Piotr Matuszczak
> ---------------------------------------------------------------------
> Intel Technology Poland sp. z o.o.
> ul. Slowackiego 173, 80-298 Gdansk
> KRS 101882
> NIP 957-07-52-316
> 

Re: Patch for telemetry design

[Justin Thaler]

Hi Piotr,
    Thanks for the updates here.

On 6/17/20 6:08 AM, Matuszczak, Piotr wrote:
> Hi Justin,
> 
>> I've taken a read through the design proposal as more of a client of the telemetry data. I like this proposal quite a bit as it is pretty clear. I had a couple of >questions and given the broad level, I wasn't sure if gerrit was the right place for them.
> 
> In fact, this patch is covering few changes that we've made to D-Bus API during the development and integration.
> 
>> On Timestamps, are the timestamps done per metric(sensor reading) or per report. This wasn't clear to me from the design proposal, and also it was hard to >tell where the timestamp was being set.
> 
> Timestamps are gathered for both, metrics and report. Timestamp for metric is the timestamp of D-Bus senor update, while timestamp of the report is for the report update. It corresponds with the Redfish Metric Report resource, where both, individual metrics and the whole report have their update timestamps.

This is good news! On the metric Timestamps, is the timestamp set at the 
time of reading (HWMON/otherapps) or is it set by the new Monitoring 
Service.
> 
>>From the performance tests section, is there a limit on the number of sensors per report, seemingly it is 10, or was this done to simplify the performance >testing?
> 
> It was done because of limited sensors support on the test platform.
> 
>> The other general question I had was around the amount of data being transmitted. For instance, if you're getting reports on every sensor in the system >(100s) the report(s) could be huge at scale. Would there be an option of considering compressed data like BEJSON? Or is this feedback that should go to the >DMTF?
> 
> First of all, the max number of supported reports will be limited to around 50 due to limited BMC resources. As for the using compressed data, if we would like to compress data for the IPC (D-Bus) it is possible for us to implement any format we like as long as we document it in the design. As for the using compressed format for the Redfish Metric report it will require acceptance from the DMTF, because it will require changes in the schema.
> You were referring to using BSON on D-Bus or in Redfish Metric Report schema?

Thanks for the information on the max number of reports and how the test 
was conducted. In terms of the BSON, I was referring to using it with 
the Redfish Metric Report Schema.

> 
> Piotr Matuszczak
> ---------------------------------------------------------------------
> Intel Technology Poland sp. z o.o.
> ul. Slowackiego 173, 80-298 Gdansk
> KRS 101882
> NIP 957-07-52-316
> 
> -----Original Message-----
> From: openbmc <openbmc-bounces+piotr.matuszczak=intel.com@lists.ozlabs.org> On Behalf Of Justin Thaler
> Sent: Tuesday, June 16, 2020 8:05 PM
> To: openbmc@lists.ozlabs.org
> Subject: Re: Patch for telemetry design
> 
> Hi Piotr,
> 	I've taken a read through the design proposal as more of a client of the telemetry data. I like this proposal quite a bit as it is pretty clear. I had a couple of questions and given the broad level, I wasn't sure if gerrit was the right place for them.
> 
> On Timestamps, are the timestamps done per metric(sensor reading) or per report. This wasn't clear to me from the design proposal, and also it was hard to tell where the timestamp was being set.
> 
>   From the performance tests section, is there a limit on the number of sensors per report, seemingly it is 10, or was this done to simplify the performance testing?
> 
> The other general question I had was around the amount of data being transmitted. For instance, if you're getting reports on every sensor in the system (100s) the report(s) could be huge at scale. Would there be an option of considering compressed data like BEJSON? Or is this feedback that should go to the DMTF?
> 
> Thanks,
> Justin Thaler
> IBM RAS Engineer
> 
> On 6/16/20 4:31 AM, Matuszczak, Piotr wrote:
>> Hi,
>>
>> I've uploaded patch for telemetry service design some time ago. I would like to ask you for review.
>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/31690
>>
>>
>> Piotr Matuszczak
>> ---------------------------------------------------------------------
>> Intel Technology Poland sp. z o.o.
>> ul. Slowackiego 173, 80-298 Gdansk
>> KRS 101882
>> NIP 957-07-52-316
>>

Thanks,
Justin Thaler