From mboxrd@z Thu Jan 1 00:00:00 1970 Subject: Re: Contributing ARM tests results to KCIDB References: <20200917125044.GA29636@e119603-lin.cambridge.arm.com> <20200917162242.GA18067@e119603-lin.cambridge.arm.com> <20200918152135.GA13088@e119603-lin.cambridge.arm.com> From: "Nikolai Kondrashov" Message-ID: <3e86960e-9780-3e18-3d12-cb4ec3959d63@redhat.com> Date: Fri, 18 Sep 2020 18:30:30 +0300 MIME-Version: 1.0 In-Reply-To: <20200918152135.GA13088@e119603-lin.cambridge.arm.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US List-ID: To: kernelci@groups.io, cristian.marussi@arm.com Cc: broonie@kernel.org, basil.eljuse@arm.com On 9/18/20 6:21 PM, Cristian Marussi wrote: > So in order to carry on my experiments, I've just tried to push a new dataset > with a few changes in my data-layout to mimic what I see other origins do; this > contained something like 38 builds across 4 different revisions (with brand new > revisions IDs), but I cannot see anything on the UI: I just keep seeing the old > push from yesterday. > > JSON seems valid and kcidb-submit does not report any error even using -l DEBUG. > (I pushed >30mins ago) > > Any idea ? Yes, I think it's one of the problems you uncovered :) The schema allows for fully-compliant RFC3339 timestamps, but the BigQuery database on the backend doesn't understand some of them. In particular it doesn't understand the date-only timestamps you send. E.g. "2020-09-13". That's what I wanted to fix today, but ran out of time. Additionally, the backend doesn't have a way to report a problem to the submitter at the moment. We intend to fix that, but for now it's possible only through us looking at the logs and sending a message to the submitter :) To work around this you can pad your timestamps with dummy date and time data. E.g. instead of sending: 2020-09-13 you can send: 2020-09-13 00:00:00+00:00 Hopefully that's the only problem. It could be, since you managed to send data before :) Nick On 9/18/20 6:21 PM, Cristian Marussi wrote: > Hi Nikolai, > > On Thu, Sep 17, 2020 at 08:26:15PM +0300, Nikolai Kondrashov wrote: >> On 9/17/20 7:22 PM, Cristian Marussi wrote: >>> It works too ... :D >>> >>> https://staging.kernelci.org:3000/d/build/build?orgId=1&var-dataset=playground_kernelci04&var-id=arm:2020-07-07:d3d7689c2cc9503266cac3bc777bb4ddae2e5f2e >> >> Whoa, awesome! >> >> And you have already uncovered a few issues we need to fix, too! >> I will deal with them tomorrow. >> >>> ..quick question though....given that now I'll have to play quite a bit >>> with it and see how's better to present our data, if anythinjg missing etc etc, >>> is there any chance (or way) that if I submmit the same JSON report multiple >>> times with slight differences here and there (but with the same IDs clearly) >>> I'll get my DB updated in the bits I have changed: as an example I've just >>> resubmitted the same report with added discovery_time and descriptions, and got >>> NO errors, but I cannot see the changes in the UI (unless they have still to >>> propagate...)..or maybe I can obtain the same effect by dropping my dataset >>> before re-submitting ? >> >> Right now it's not supported (with various possible quirks if attempted). >> So, preferably, submit only one, complete and final instance of each object >> (with unique ID) for now. >> >> We have a plan to support merging missing properties across multiple reported >> objects with the same ID. >> >> Object A Object B Dashboard/Notifications >> >> FieldX: Foo Foo Foo >> FieldY: Bar Bar >> FieldZ: Baz Baz >> FieldU: Red Blue Red/Blue >> >> Since we're using a distributed database we cannot really maintain order >> (without introducing artificial global lock), so the order of the reports >> doesn't matter. We can only guarantee that a present value would override >> missing value. It would be undefined which value would be picked among >> multiple different values. >> >> This would allow gradual reporting of each object, but no editing, sorry. >> >> However, once again, this is a plan with some research done, only. >> I plan to start implementing it within a few weeks. >> > > So in order to carry on my experiments, I've just tried to push a new dataset > with a few changes in my data-layout to mimic what I see other origins do; this > contained something like 38 builds across 4 different revisions (with brand new > revisions IDs), but I cannot see anything on the UI: I just keep seeing the old > push from yesterday. > > JSON seems valid and kcidb-submit does not report any error even using -l DEBUG. > (I pushed >30mins ago) > > Any idea ? > > Thanks > > Cristian > >> Nick >> >> On 9/17/20 7:22 PM, Cristian Marussi wrote: >>> On Thu, Sep 17, 2020 at 04:52:30PM +0300, Nikolai Kondrashov wrote: >>>> Hi Christian, >>>> >>>> On 9/17/20 3:50 PM, Cristian Marussi wrote: >>>>> Hi Nikolai, >>>>> >>>>> I work at ARM in the Kernel team and, in short, we'd like certainly to >>>>> contribute our internal Kernel test results to KCIDB. >>>> >>>> Wonderful! >>>> >>>>> After having attended your LPC2020 TestMC and KernelCI/BoF, I've now cooked >>>>> up some KCIDB JSON test report (seemingly valid against your KCIDB v3 schema) >>>>> and I'd like to start experimenting with kci-submit (on non-production >>>>> instances), so as to assess how to fit our results into your schema and maybe >>>>> contribute with some new KCIDB requirements if strictly needed. >>>> >>>> Great, this is exactly what we need, welcome aboard :) >>>> >>>> Please don't hesitate to reach out on kernelci@groups.io or on #kernelci on >>>> freenode.net, if you have any questions, problems, or requirements. >>>> >>>>> Is it possible to get some valid credentials and a playground instance to >>>>> point at ? >>>> >>>> Absolutely, I created credentials for you and sent them in a separate message. >>>> >>>> You can use origin "arm" for the start, unless you have multiple CI systems >>>> and want to differentiate them somehow in your reports. >>>> >>>> Nick >>>> >>> Thanks ! >>> >>> It works too ... :D >>> >>> https://staging.kernelci.org:3000/d/build/build?orgId=1&var-dataset=playground_kernelci04&var-id=arm:2020-07-07:d3d7689c2cc9503266cac3bc777bb4ddae2e5f2e >>> >>> ..quick question though....given that now I'll have to play quite a bit >>> with it and see how's better to present our data, if anythinjg missing etc etc, >>> is there any chance (or way) that if I submmit the same JSON report multiple >>> times with slight differences here and there (but with the same IDs clearly) >>> I'll get my DB updated in the bits I have changed: as an example I've just >>> resubmitted the same report with added discovery_time and descriptions, and got >>> NO errors, but I cannot see the changes in the UI (unless they have still to >>> propagate...)..or maybe I can obtain the same effect by dropping my dataset >>> before re-submitting ? >>> >>> Regards >>> >>> Thanks >>> >>> Cristian >>> >>>> On 9/17/20 3:50 PM, Cristian Marussi wrote: >>>>> Hi Nikolai, >>>>> >>>>> I work at ARM in the Kernel team and, in short, we'd like certainly to >>>>> contribute our internal Kernel test results to KCIDB. >>>>> >>>>> After having attended your LPC2020 TestMC and KernelCI/BoF, I've now cooked >>>>> up some KCIDB JSON test report (seemingly valid against your KCIDB v3 schema) >>>>> and I'd like to start experimenting with kci-submit (on non-production >>>>> instances), so as to assess how to fit our results into your schema and maybe >>>>> contribute with some new KCIDB requirements if strictly needed. >>>>> >>>>> Is it possible to get some valid credentials and a playground instance to >>>>> point at ? >>>>> >>>>> Thanks >>>>> >>>>> Regards >>>>> >>>>> Cristian >>>>> >>>>> >>>>> >>>>> >>>>> >>>> >>> >> > > > > >