Re: Contributing ARM tests results to KCIDB

Nikolai Kondrashov
2020-09-18T15:30:30Z

On 9/18/20 6:21 PM, Cristian Marussi wrote:
> So in order to carry on my experiments, I've just tried to push a new dataset
> with a few changes in my data-layout to mimic what I see other origins do; this
> contained something like 38 builds across 4 different revisions (with brand new
> revisions IDs), but I cannot see anything on the UI: I just keep seeing the old
> push from yesterday.
>
> JSON seems valid and kcidb-submit does not report any error even using -l DEBUG.
> (I pushed >30mins ago)
>
> Any idea ?

Yes, I think it's one of the problems you uncovered :)

The schema allows for fully-compliant RFC3339 timestamps, but the BigQuery
database on the backend doesn't understand some of them. In particular it
doesn't understand the date-only timestamps you send. E.g. "2020-09-13".
That's what I wanted to fix today, but ran out of time.

Additionally, the backend doesn't have a way to report a problem to the
submitter at the moment. We intend to fix that, but for now it's possible only
through us looking at the logs and sending a message to the submitter :)

To work around this you can pad your timestamps with dummy date and time
data.

E.g. instead of sending:

2020-09-13

you can send:

2020-09-13 00:00:00+00:00

Hopefully that's the only problem. It could be, since you managed to send data
before :)

Nick

On 9/18/20 6:21 PM, Cristian Marussi wrote:
> Hi Nikolai,
>
> On Thu, Sep 17, 2020 at 08:26:15PM +0300, Nikolai Kondrashov wrote:
>> On 9/17/20 7:22 PM, Cristian Marussi wrote:
>>> It works too ... :D
>>>
>>> https://staging.kernelci.org:3000/d/build/build?orgId=1&var-dataset=playground_kernelci04&var-id=arm:2020-07-07:d3d7689c2cc9503266cac3bc777bb4ddae2e5f2e
>>
>> Whoa, awesome!
>>
>> And you have already uncovered a few issues we need to fix, too!
>> I will deal with them tomorrow.
>>
>>> ..quick question though....given that now I'll have to play quite a bit
>>> with it and see how's better to present our data, if anythinjg missing etc etc,
>>> is there any chance (or way) that if I submmit the same JSON report multiple
>>> times with slight differences here and there (but with the same IDs clearly)
>>> I'll get my DB updated in the bits I have changed: as an example I've just
>>> resubmitted the same report with added discovery_time and descriptions, and got
>>> NO errors, but I cannot see the changes in the UI (unless they have still to
>>> propagate...)..or maybe I can obtain the same effect by dropping my dataset
>>> before re-submitting ?
>>
>> Right now it's not supported (with various possible quirks if attempted).
>> So, preferably, submit only one, complete and final instance of each object
>> (with unique ID) for now.
>>
>> We have a plan to support merging missing properties across multiple reported
>> objects with the same ID.
>>
>> Object A Object B Dashboard/Notifications
>>
>> FieldX: Foo Foo Foo
>> FieldY: Bar Bar
>> FieldZ: Baz Baz
>> FieldU: Red Blue Red/Blue
>>
>> Since we're using a distributed database we cannot really maintain order
>> (without introducing artificial global lock), so the order of the reports
>> doesn't matter. We can only guarantee that a present value would override
>> missing value. It would be undefined which value would be picked among
>> multiple different values.
>>
>> This would allow gradual reporting of each object, but no editing, sorry.
>>
>> However, once again, this is a plan with some research done, only.
>> I plan to start implementing it within a few weeks.
>>
>
> So in order to carry on my experiments, I've just tried to push a new dataset
> with a few changes in my data-layout to mimic what I see other origins do; this
> contained something like 38 builds across 4 different revisions (with brand new
> revisions IDs), but I cannot see anything on the UI: I just keep seeing the old
> push from yesterday.
>
> JSON seems valid and kcidb-submit does not report any error even using -l DEBUG.
> (I pushed >30mins ago)
>
> Any idea ?
>
> Thanks
>
> Cristian
>
>> Nick
>>
>> On 9/17/20 7:22 PM, Cristian Marussi wrote:
>>> On Thu, Sep 17, 2020 at 04:52:30PM +0300, Nikolai Kondrashov wrote:
>>>> Hi Christian,
>>>>
>>>> On 9/17/20 3:50 PM, Cristian Marussi wrote:
>>>>> Hi Nikolai,
>>>>>
>>>>> I work at ARM in the Kernel team and, in short, we'd like certainly to
>>>>> contribute our internal Kernel test results to KCIDB.
>>>>
>>>> Wonderful!
>>>>
>>>>> After having attended your LPC2020 TestMC and KernelCI/BoF, I've now cooked
>>>>> up some KCIDB JSON test report (seemingly valid against your KCIDB v3 schema)
>>>>> and I'd like to start experimenting with kci-submit (on non-production
>>>>> instances), so as to assess how to fit our results into your schema and maybe
>>>>> contribute with some new KCIDB requirements if strictly needed.
>>>>
>>>> Great, this is exactly what we need, welcome aboard :)
>>>>
>>>> Please don't hesitate to reach out on kernelci@groups.io or on #kernelci on
>>>> freenode.net, if you have any questions, problems, or requirements.
>>>>
>>>>> Is it possible to get some valid credentials and a playground instance to
>>>>> point at ?
>>>>
>>>> Absolutely, I created credentials for you and sent them in a separate message.
>>>>
>>>> You can use origin "arm" for the start, unless you have multiple CI systems
>>>> and want to differentiate them somehow in your reports.
>>>>
>>>> Nick
>>>>
>>> Thanks !
>>>
>>> It works too ... :D
>>>
>>> https://staging.kernelci.org:3000/d/build/build?orgId=1&var-dataset=playground_kernelci04&var-id=arm:2020-07-07:d3d7689c2cc9503266cac3bc777bb4ddae2e5f2e
>>>
>>> ..quick question though....given that now I'll have to play quite a bit
>>> with it and see how's better to present our data, if anythinjg missing etc etc,
>>> is there any chance (or way) that if I submmit the same JSON report multiple
>>> times with slight differences here and there (but with the same IDs clearly)
>>> I'll get my DB updated in the bits I have changed: as an example I've just
>>> resubmitted the same report with added discovery_time and descriptions, and got
>>> NO errors, but I cannot see the changes in the UI (unless they have still to
>>> propagate...)..or maybe I can obtain the same effect by dropping my dataset
>>> before re-submitting ?
>>>
>>> Regards
>>>
>>> Thanks
>>>
>>> Cristian
>>>
>>>> On 9/17/20 3:50 PM, Cristian Marussi wrote:
>>>>> Hi Nikolai,
>>>>>
>>>>> I work at ARM in the Kernel team and, in short, we'd like certainly to
>>>>> contribute our internal Kernel test results to KCIDB.
>>>>>
>>>>> After having attended your LPC2020 TestMC and KernelCI/BoF, I've now cooked
>>>>> up some KCIDB JSON test report (seemingly valid against your KCIDB v3 schema)
>>>>> and I'd like to start experimenting with kci-submit (on non-production
>>>>> instances), so as to assess how to fit our results into your schema and maybe
>>>>> contribute with some new KCIDB requirements if strictly needed.
>>>>>
>>>>> Is it possible to get some valid credentials and a playground instance to
>>>>> point at ?
>>>>>
>>>>> Thanks
>>>>>
>>>>> Regards
>>>>>
>>>>> Cristian
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
>
>
>
>