Re: Contributing ARM tests results to KCIDB

Nikolai Kondrashov
2020-12-10T18:17:42Z

Hi Cristian,

On 12/10/20 7:23 PM, Cristian Marussi wrote:
> I fixed the issue about uniqueness of the tests IDs but left the valid
> flag on the revision undefined as of now given the revision hash is
> temporarily faked (as I told you)...just to have an indication that the
> revision is bogus.
> Anyway I'll have that fixed in our backend soon, and once I'll start
> receiving a proper real hash the system 'should' automatically start
> tagging revisions as valid: True.

Good plan!

> Moreover, after fixing a few more annoyances on my side, today I switched to
> KCIDB production and pushed December results; from tomorrow morning it should
> start feeding daily data to KCIDB production.

Woo-hoo! Wonderful, this is a nice Christmas present :)

> Thanks for the support and patience.

Thank you for your work, Cristian!

I notice a bit of strange data: failed builds have one (failed) boot test
submitted. Is this on purpose, does this mean something special? Logically, we
can't boot a build if it hasn't completed, don't we?

Here's an example:

https://staging.kernelci.org:3000/d/build/build?orgId=1&var-id=arm:2020-12-08:d6051b14fced47d1983fd70171b9bcd7170491ce

Nick

On 12/10/20 7:23 PM, Cristian Marussi wrote:
> Hi Nick
>
> On Wed, Dec 02, 2020 at 03:38:19PM +0200, Nikolai Kondrashov via groups.io wrote:
>> On 12/2/20 2:01 PM, Cristian Marussi wrote:
>>>> From POV of KCIDB, what you're sending now is overwriting the same test runs
>>>> over and over, and we can't really tell which one of those objects is the
>>>> final version.
>>>
>>>
>>> Ah, that was exactly what I used to do in my first initial experiments and then,
>>> looking at the data on the UI, I was dumb enough to decide that I should have got
>>> it wrong and I started using the test_id instead of the test_execution_id, because
>>> I thought that, anyway, you can recognize the different test executions of the
>>> same test_id looking at the different build_id is part of (which for us represent
>>> the different test suite runs)....but I suppose this wrong assumption of mine
>>> sparked from the relational data model I use on our side. I'll fix it.
>>
>> Yes, that would work, but then we get a "foreign key explosion" as we start
>> linking to tests from other objects beside builds. So, for now we're sticking
>> to the "one ID column per table" policy.
>>
>> Thanks for bearing with us, and am glad to hear you already have
>> `test_execution_id` in your database, so the fix shouldn't take long :)
>>
>>> Sure, in fact, as of now I still have to ask for some changes in our reporting
>>> backend, (which generates the original data stored in our DB and then pushed
>>> to you), so I have to admit the git commit hash are partially faked (since I
>>> have only a git describe string to start from) and as a consequence they won't
>>> really be so much useful for comparisons amongst different origins (given
>>> they don't refer real kernel commits), BUT I thought this NOT to be a
>>> blocking problem for now, so that I can start pushing data to KCIDB and
>>> then later on (once I get real full hashes on my side) I'll start pushing the
>>> real valid ones, does it sounds good ?
>>
>> Yes, no problem. We don't have maintainers/developers to get angry yet :D
>>
>> I'm looking forward to having four-origin revisions in the dashboard, though,
>> one more than e.g. this one:
>>
>> https://staging.kernelci.org:3000/d/revision/revision?var-id=3650b228f83adda7e5ee532e2b90429c03f7b9ec
>>
>
> I fixed the issue about uniqueness of the tests IDs but left the valid
> flag on the revision undefined as of now given the revision hash is
> temporarily faked (as I told you)...just to have an indication that the
> revision is bogus.
> Anyway I'll have that fixed in our backend soon, and once I'll start
> receiving a proper real hash the system 'should' automatically start
> tagging revisions as valid: True.
>
>>> Side question...for dynamic schema validation purposes...is there any URL
>>> where I can fetch the latest currently valid schema ... something like:
>>>
>>> https://github.com/kernelci/kcidb/releases/kcidb.latest.schema.json
>>>
>>> so that I can check automatically against the latest greatest instead of
>>> using a builtin predownloaded one (or is it a bad idea in your opinion ?)
>>
>> The JSON schemas we generate with `kcidb-schema`, and use inside KCIDB, only
>> validate *one* major version. So v3 data would only validate with v3 schema,
>> but not with e.g. v4.
>>
>> So if you e.g. download and validate against the latest-release schema
>> automatically, validation will start failing the moment a release with v4
>> comes out.
>>
>> Automatic data upgrades between major versions are done in Python whenever we
>> see a difference between the numbers.
>>
>> OTOH, minor version bumps of the schema are backwards-compatible, and you
>> would be fine upgrading validation to those. However, we don't have many of
>> those at all yet, as we're still changing the schema a lot.
>>
>> So, I think a reasonable workflow right now is to download and switch to a new
>> version at the same time you're upgrading your submission code to the next
>> major release of the schema. You'll need more work on the code than just
>> switching the schema, anyway.
>>
>> However, let's get back to this further along the way, perhaps we can think of
>> something smoother and more automated. E.g. set up a way to have automatic
>> upgrades between minor versions.
>
> Agreed, using v3 for the moment.
>
> Moreover, after fixing a few more annoyances on my side, today I switched to
> KCIDB production and pushed December results; from tomorrow morning it should
> start feeding daily data to KCIDB production.
>
> Thanks for the support and patience.
>
> Cristian
>
>>
>> Thanks :)
>> Nick
>>
>> On 12/2/20 2:01 PM, Cristian Marussi wrote:
>>> On Wed, Dec 02, 2020 at 12:16:10PM +0200, Nikolai Kondrashov wrote:
>>>> On 12/2/20 11:23 AM, Cristian Marussi wrote:
>>>>> On Wed, Dec 02, 2020 at 10:05:05AM +0200, Nikolai Kondrashov via groups.io wrote:
>>>>>> On 11/5/20 8:46 PM, Cristian Marussi wrote:
>>>>>>> after past month few experiments on ARM KCIDB submissions against your
>>>>>>> KCIDB staging instance , I was dragged a bit away from this by other stuff
>>>>>>> before effectively deploying some real automation on our side to push our
>>>>>>> daily results to KCIDB...now I'm back at it and I'll keep on testing
>>>>>>> some automation on our side for a bit against your KCIDB staging instance
>>>>>>> before asking you to move to production eventually.
>>>>>>
>>>>>> I see your data has been steadily trickling into our playground database and
>>>>>> it looks quite good. Would you like to move to the production instance?
>>>>>>
>>>>>> I can review your data for you, we can fix the remaining issues if we find
>>>>>> them, and I can give you the permissions to push to production. Then you will
>>>>>> only need to change the topic you push to from "playground_kernelci_new" to
>>>>>> "kernelci_new".
>>>>>
>>>>> In fact I left one staging instance on our side to push data on your
>>>>> staging instance to verify remaining issues on our side *and there are a
>>>>> couple of minor ones I spotted that I'd like to fix indeed);
>>>>
>>>> Sure, it's up to you when you decide to switch. However, if you'd like, list
>>>> your issues here, and I would be able to tell you if those are important from
>>>> KCIDB POV.
>>>>
>>>> Looking at your data, I can only find one serious issue: the test run ("test")
>>>> IDs are not unique. E.g. there are 1460 objects with ID "arm:LTP:11" which
>>>> use 643 distinct build_id's among them.
>>>>
>>>> The test run IDs should correspond to a single execution of a test. Otherwise
>>>> we won't be able to tell them apart. You can send multiple reports containing
>>>> test runs ("tests") with the same ID, but that would still mean the same
>>>> execution, only repeating the same data, or adding more.
>>>>
>>>> A little more explanation:
>>>> https://github.com/kernelci/kcidb/blob/master/SUBMISSION_HOWTO.md#submitting-objects-multiple-times
>>>>
>>>> From POV of KCIDB, what you're sending now is overwriting the same test runs
>>>> over and over, and we can't really tell which one of those objects is the
>>>> final version.
>>>
>>>
>>> Ah, that was exactly what I used to do in my first initial experiments and then,
>>> looking at the data on the UI, I was dumb enough to decide that I should have got
>>> it wrong and I started using the test_id instead of the test_execution_id, because
>>> I thought that, anyway, you can recognize the different test executions of the
>>> same test_id looking at the different build_id is part of (which for us represent
>>> the different test suite runs)....but I suppose this wrong assumption of mine
>>> sparked from the relational data model I use on our side. I'll fix it.
>>>
>>>>
>>>> Aside from that, you might want to add `"valid": true` to your "revision"
>>>> objects to indicate they're alright. You never seem to send patched revisions,
>>>> so it should always be true for you. Then instead of the blank "Status" field:
>>>>
>>>> https://staging.kernelci.org:3000/d/revision/revision?orgId=1&var-dataset=playground_kernelci04&var-id=f0d5c8f71bbb1aa1e98cb1a89adb9d57c04ede3d
>>>>
>>>> you would get a nice green check mark, like this:
>>>>
>>>> https://staging.kernelci.org:3000/d/revision/revision?orgId=1&var-dataset=kernelci04&var-id=8af5fe40bd59d8aa26dd76d9971435177aacbfce
>>>>
>>>
>>> Ah I missed this valid flag on revision too, I'll fix.
>>>
>>>> Finally, at this stage we really need a breadth of data coming from
>>>> different CI system, rather than its depth or precision, so we can understand
>>>> the problem at hand better and faster. It would do us no good to concentrate
>>>> on just a few, and solidify the design around them. That would make it more
>>>> difficult for others to join.
>>>>
>>>> You can refine and add more data afterwards.
>>>>
>>>
>>> Sure, in fact, as of now I still have to ask for some changes in our reporting
>>> backend, (which generates the original data stored in our DB and then pushed
>>> to you), so I have to admit the git commit hash are partially faked (since I
>>> have only a git describe string to start from) and as a consequence they won't
>>> really be so much useful for comparisons amongst different origins (given
>>> they don't refer real kernel commits), BUT I thought this NOT to be a
>>> blocking problem for now, so that I can start pushing data to KCIDB and
>>> then later on (once I get real full hashes on my side) I'll start pushing the
>>> real valid ones, does it sounds good ?
>>>
>>>
>>>>> moreover I saw a little while a go that you're going to switch to schema v4
>>>>> with some minor changes in revisions and commit_hashes so I wanted to
>>>>> conform to that once it's published (even though you're back compatible with
>>>>> v3 AFAIU)....
>>>>
>>>> I would rather you didn't wait for that, as I'm neck deep in research for the
>>>> next release right now, and it doesn't seem like it's gonna come out soon.
>>>> I'm concentrating on getting our result notifications in a good shape so we
>>>> can reach actual kernel developers ASAP.
>>>>
>>>> We can work on upgrading your setup later, when it comes out. And there are
>>>> going to be other changes, anyway. So, I'd rather we released early and
>>>> iterated.
>>>>
>>>
>>> Good I'l stick to v3.
>>>
>>> Side question...for dynamic schema validation purposes...is there any URL
>>> where I can fetch the latest currently valid schema ... something like:
>>>
>>> https://github.com/kernelci/kcidb/releases/kcidb.latest.schema.json
>>>
>>> so that I can check automatically against the latest greatest instead of
>>> using a builtin predownloaded one (or is it a bad idea in your opinion ?)
>>>
>>>>> ... then I've got dragged away again from this past week :D
>>>>>
>>>>> In fact my next steps (possibly next week) would have been (beside my fixes)
>>>>> to ask you how to proceed further to production KCIDB.
>>>>
>>>> There's never enough time for everything :)
>>>>
>>>
>>> eh..
>>>
>>>>> Would you want me to stop flooding your staging instance in the meantime (:D)
>>>>> till I'm back at it at least , I think I have enugh data now to debug anyway.
>>>>> (I could made a few more check next week though)
>>>>
>>>> Don't worry about that, and keep pushing, maybe you'll manage to break it
>>>> again and then we can fix it :)
>>>>
>>>
>>> Fine :D
>>>
>>>>> If it's just a matter of switching project (once got enhanced permissions
>>>>> from you) please do it, and I'll try to finalize all next week on our
>>>>> side and move to production.
>>>>
>>>> Permission granted! Switch when you feel ready, and don't hesitate to ping me
>>>> for another review, if you need it.
>>>>
>>>> Just replace "playground_kernelci_new" topic with "kernelci_new" in your
>>>> setup when you're ready.
>>>>
>>>
>>> Cool, thanks.
>>>
>>>>> Thanks for the patience
>>>>
>>>> Thank you for your effort, we need your data :D
>>>>
>>>> Nick
>>>>
>>>
>>> Thank you Nick
>>>
>>> Cheers,
>>>
>>> Cristian
>>>
>>>
>>>> On 12/2/20 11:23 AM, Cristian Marussi wrote:
>>>>> Hi Nick
>>>>>
>>>>> On Wed, Dec 02, 2020 at 10:05:05AM +0200, Nikolai Kondrashov via groups.io wrote:
>>>>>> Hi Cristian,
>>>>>>
>>>>>> On 11/5/20 8:46 PM, Cristian Marussi wrote:
>>>>>>> Hi Nick,
>>>>>>>
>>>>>>> after past month few experiments on ARM KCIDB submissions against your
>>>>>>> KCIDB staging instance , I was dragged a bit away from this by other stuff
>>>>>>> before effectively deploying some real automation on our side to push our
>>>>>>> daily results to KCIDB...now I'm back at it and I'll keep on testing
>>>>>>> some automation on our side for a bit against your KCIDB staging instance
>>>>>>> before asking you to move to production eventually.
>>>>>>
>>>>>> I see your data has been steadily trickling into our playground database and
>>>>>> it looks quite good. Would you like to move to the production instance?
>>>>>>
>>>>>> I can review your data for you, we can fix the remaining issues if we find
>>>>>> them, and I can give you the permissions to push to production. Then you will
>>>>>> only need to change the topic you push to from "playground_kernelci_new" to
>>>>>> "kernelci_new".
>>>>>
>>>>> In fact I left one staging instance on our side to push data on your
>>>>> staging instance to verify remaining issues on our side *and there are a
>>>>> couple of minor ones I spotted that I'd like to fix indeed); moreover I saw
>>>>> a little while a go that you're going to switch to schema v4 with some minor
>>>>> changes in revisions and commit_hashes so I wanted to conform to that once
>>>>> it's published (even though you're back compatible with v3 AFAIU)....
>>>>>
>>>>> ... then I've got dragged away again from this past week :D
>>>>>
>>>>> In fact my next steps (possibly next week) would have been (beside my fixes)
>>>>> to ask you how to proceed further to production KCIDB.
>>>>>
>>>>> Would you want me to stop flooding your staging instance in the meantime (:D)
>>>>> till I'm back at it at least , I think I have enugh data now to debug anyway.
>>>>> (I could made a few more check next week though)
>>>>>
>>>>> If it's just a matter of switching project (once got enhanced permissions
>>>>> from you) please do it, and I'll try to finalize all next week on our
>>>>> side and move to production.
>>>>>
>>>>> Thanks for the patience
>>>>>
>>>>> Cristian
>>>>>
>>>>>
>>>>>>
>>>>>> Nick
>>>>>>
>>>>>> On 11/5/20 8:46 PM, Cristian Marussi wrote:
>>>>>>> Hi Nick,
>>>>>>>
>>>>>>> after past month few experiments on ARM KCIDB submissions against your
>>>>>>> KCIDB staging instance , I was dragged a bit away from this by other stuff
>>>>>>> before effectively deploying some real automation on our side to push our
>>>>>>> daily results to KCIDB...now I'm back at it and I'll keep on testing
>>>>>>> some automation on our side for a bit against your KCIDB staging instance
>>>>>>> before asking you to move to production eventually.
>>>>>>>
>>>>>>> But, today I realized, though, that I cannot push anymore data successfully
>>>>>>> into staging even using the same test script I used one month ago to push
>>>>>>> some new test data seems to fail now (I tested a few different days and
>>>>>>> JSON validates fine with jsonschema...with proper dates with hours...)...
>>>>>>> ...I cannot see any of my today tests' pushes on:
>>>>>>>
>>>>>>> https://staging.kernelci.org:3000/d/home/home?orgId=1&from=now-1y&to=now&refresh=30m&var-origin=arm&var-git_repository_url=All&var-dataset=playground_kernelci04
>>>>>>>
>>>>>>> Auth seems to proceed fine, but I cannot find any submission dated after
>>>>>>> the old ~15/18-09-2020 submissions. I'm using the same kci-submit tools
>>>>>>> version installed past months from your github though.
>>>>>>>
>>>>>>> Do you see any errors on your side that can shed a light on this ?
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> Regards
>>>>>>>
>>>>>>> Cristian
>>>>>>>
>>>>>>> On Fri, Sep 18, 2020 at 05:42:28PM +0100, Cristian Marussi wrote:
>>>>>>>> Hi Nick,
>>>>>>>>
>>>>>>>> On Fri, Sep 18, 2020 at 06:53:28PM +0300, Nikolai Kondrashov wrote:
>>>>>>>>> On 9/18/20 6:30 PM, Nikolai Kondrashov wrote:
>>>>>>>>>> Yes, I think it's one of the problems you uncovered :)
>>>>>>>>>>
>>>>>>>>>> The schema allows for fully-compliant RFC3339 timestamps, but the BigQuery
>>>>>>>>>> database on the backend doesn't understand some of them. In particular it
>>>>>>>>>> doesn't understand the date-only timestamps you send. E.g. "2020-09-13".
>>>>>>>>>> That's what I wanted to fix today, but ran out of time.
>>>>>>>>>
>>>>>>>>> Looking at this more it seems that Python's jsonschema module simply doesn't
>>>>>>>>> enforce the requirements we put on those fields 🤦. You can send essentially
>>>>>>>>> what you want and then hit BigQuery, which is serious about them.
>>>>>>>>
>>>>>>>> ...in fact on my side I check too with jsonschema in my script before using kcidb :D
>>>>>>>>>
>>>>>>>>> Sorry about that.
>>>>>>>>>
>>>>>>>>
>>>>>>>> No worries.
>>>>>>>>
>>>>>>>>> I opened an issue for this: https://github.com/kernelci/kcidb/issues/108
>>>>>>>>>
>>>>>>>>> For now please just make sure your timestamp comply with RFC3339.
>>>>>>>>>
>>>>>>>>> You can produce such a timestamp e.g. using "date --rfc-3339=s".
>>>>>>>>
>>>>>>>> I'll anyway fix my data on my side too, to have the real discovery timestamp.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Nick
>>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> Cristian
>>>>>>>>
>>>>>>>>> On 9/18/20 6:30 PM, Nikolai Kondrashov wrote:
>>>>>>>>>> On 9/18/20 6:21 PM, Cristian Marussi wrote:
>>>>>>>>>> > So in order to carry on my experiments, I've just tried to push a new dataset
>>>>>>>>>> > with a few changes in my data-layout to mimic what I see other origins do; this
>>>>>>>>>> > contained something like 38 builds across 4 different revisions (with brand new
>>>>>>>>>> > revisions IDs), but I cannot see anything on the UI: I just keep seeing the old
>>>>>>>>>> > push from yesterday.
>>>>>>>>>> >
>>>>>>>>>> > JSON seems valid and kcidb-submit does not report any error even using -l DEBUG.
>>>>>>>>>> > (I pushed >30mins ago)
>>>>>>>>>> >
>>>>>>>>>> > Any idea ?
>>>>>>>>>>
>>>>>>>>>> Yes, I think it's one of the problems you uncovered :)
>>>>>>>>>>
>>>>>>>>>> The schema allows for fully-compliant RFC3339 timestamps, but the BigQuery
>>>>>>>>>> database on the backend doesn't understand some of them. In particular it
>>>>>>>>>> doesn't understand the date-only timestamps you send. E.g. "2020-09-13".
>>>>>>>>>> That's what I wanted to fix today, but ran out of time.
>>>>>>>>>>
>>>>>>>>>> Additionally, the backend doesn't have a way to report a problem to the
>>>>>>>>>> submitter at the moment. We intend to fix that, but for now it's possible only
>>>>>>>>>> through us looking at the logs and sending a message to the submitter :)
>>>>>>>>>>
>>>>>>>>>> To work around this you can pad your timestamps with dummy date and time
>>>>>>>>>> data.
>>>>>>>>>>
>>>>>>>>>> E.g. instead of sending:
>>>>>>>>>>
>>>>>>>>>> 2020-09-13
>>>>>>>>>>
>>>>>>>>>> you can send:
>>>>>>>>>>
>>>>>>>>>> 2020-09-13 00:00:00+00:00
>>>>>>>>>>
>>>>>>>>>> Hopefully that's the only problem. It could be, since you managed to send data
>>>>>>>>>> before :)
>>>>>>>>>>
>>>>>>>>>> Nick
>>>>>>>>>>
>>>>>>>>>> On 9/18/20 6:21 PM, Cristian Marussi wrote:
>>>>>>>>>> > Hi Nikolai,
>>>>>>>>>> >
>>>>>>>>>> > On Thu, Sep 17, 2020 at 08:26:15PM +0300, Nikolai Kondrashov wrote:
>>>>>>>>>> >> On 9/17/20 7:22 PM, Cristian Marussi wrote:
>>>>>>>>>> >>> It works too ... :D
>>>>>>>>>> >>>
>>>>>>>>>> >>> https://staging.kernelci.org:3000/d/build/build?orgId=1&var-dataset=playground_kernelci04&var-id=arm:2020-07-07:d3d7689c2cc9503266cac3bc777bb4ddae2e5f2e
>>>>>>>>>> >>
>>>>>>>>>> >> Whoa, awesome!
>>>>>>>>>> >>
>>>>>>>>>> >> And you have already uncovered a few issues we need to fix, too!
>>>>>>>>>> >> I will deal with them tomorrow.
>>>>>>>>>> >>
>>>>>>>>>> >>> ..quick question though....given that now I'll have to play quite a bit
>>>>>>>>>> >>> with it and see how's better to present our data, if anythinjg missing etc etc,
>>>>>>>>>> >>> is there any chance (or way) that if I submmit the same JSON report multiple
>>>>>>>>>> >>> times with slight differences here and there (but with the same IDs clearly)
>>>>>>>>>> >>> I'll get my DB updated in the bits I have changed: as an example I've just
>>>>>>>>>> >>> resubmitted the same report with added discovery_time and descriptions, and got
>>>>>>>>>> >>> NO errors, but I cannot see the changes in the UI (unless they have still to
>>>>>>>>>> >>> propagate...)..or maybe I can obtain the same effect by dropping my dataset
>>>>>>>>>> >>> before re-submitting ?
>>>>>>>>>> >>
>>>>>>>>>> >> Right now it's not supported (with various possible quirks if attempted).
>>>>>>>>>> >> So, preferably, submit only one, complete and final instance of each object
>>>>>>>>>> >> (with unique ID) for now.
>>>>>>>>>> >>
>>>>>>>>>> >> We have a plan to support merging missing properties across multiple reported
>>>>>>>>>> >> objects with the same ID.
>>>>>>>>>> >>
>>>>>>>>>> >> Object A Object B Dashboard/Notifications
>>>>>>>>>> >>
>>>>>>>>>> >> FieldX: Foo Foo Foo
>>>>>>>>>> >> FieldY: Bar Bar
>>>>>>>>>> >> FieldZ: Baz Baz
>>>>>>>>>> >> FieldU: Red Blue Red/Blue
>>>>>>>>>> >>
>>>>>>>>>> >> Since we're using a distributed database we cannot really maintain order
>>>>>>>>>> >> (without introducing artificial global lock), so the order of the reports
>>>>>>>>>> >> doesn't matter. We can only guarantee that a present value would override
>>>>>>>>>> >> missing value. It would be undefined which value would be picked among
>>>>>>>>>> >> multiple different values.
>>>>>>>>>> >>
>>>>>>>>>> >> This would allow gradual reporting of each object, but no editing, sorry.
>>>>>>>>>> >>
>>>>>>>>>> >> However, once again, this is a plan with some research done, only.
>>>>>>>>>> >> I plan to start implementing it within a few weeks.
>>>>>>>>>> >>
>>>>>>>>>> >
>>>>>>>>>> > So in order to carry on my experiments, I've just tried to push a new dataset
>>>>>>>>>> > with a few changes in my data-layout to mimic what I see other origins do; this
>>>>>>>>>> > contained something like 38 builds across 4 different revisions (with brand new
>>>>>>>>>> > revisions IDs), but I cannot see anything on the UI: I just keep seeing the old
>>>>>>>>>> > push from yesterday.
>>>>>>>>>> >
>>>>>>>>>> > JSON seems valid and kcidb-submit does not report any error even using -l DEBUG.
>>>>>>>>>> > (I pushed >30mins ago)
>>>>>>>>>> >
>>>>>>>>>> > Any idea ?
>>>>>>>>>> >
>>>>>>>>>> > Thanks
>>>>>>>>>> >
>>>>>>>>>> > Cristian
>>>>>>>>>> >
>>>>>>>>>> >> Nick
>>>>>>>>>> >>
>>>>>>>>>> >> On 9/17/20 7:22 PM, Cristian Marussi wrote:
>>>>>>>>>> >>> On Thu, Sep 17, 2020 at 04:52:30PM +0300, Nikolai Kondrashov wrote:
>>>>>>>>>> >>>> Hi Christian,
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> On 9/17/20 3:50 PM, Cristian Marussi wrote:
>>>>>>>>>> >>>>> Hi Nikolai,
>>>>>>>>>> >>>>>
>>>>>>>>>> >>>>> I work at ARM in the Kernel team and, in short, we'd like certainly to
>>>>>>>>>> >>>>> contribute our internal Kernel test results to KCIDB.
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> Wonderful!
>>>>>>>>>> >>>>
>>>>>>>>>> >>>>> After having attended your LPC2020 TestMC and KernelCI/BoF, I've now cooked
>>>>>>>>>> >>>>> up some KCIDB JSON test report (seemingly valid against your KCIDB v3 schema)
>>>>>>>>>> >>>>> and I'd like to start experimenting with kci-submit (on non-production
>>>>>>>>>> >>>>> instances), so as to assess how to fit our results into your schema and maybe
>>>>>>>>>> >>>>> contribute with some new KCIDB requirements if strictly needed.
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> Great, this is exactly what we need, welcome aboard :)
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> Please don't hesitate to reach out on kernelci@groups.io or on #kernelci on
>>>>>>>>>> >>>> freenode.net, if you have any questions, problems, or requirements.
>>>>>>>>>> >>>>
>>>>>>>>>> >>>>> Is it possible to get some valid credentials and a playground instance to
>>>>>>>>>> >>>>> point at ?
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> Absolutely, I created credentials for you and sent them in a separate message.
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> You can use origin "arm" for the start, unless you have multiple CI systems
>>>>>>>>>> >>>> and want to differentiate them somehow in your reports.
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> Nick
>>>>>>>>>> >>>>
>>>>>>>>>> >>> Thanks !
>>>>>>>>>> >>>
>>>>>>>>>> >>> It works too ... :D
>>>>>>>>>> >>>
>>>>>>>>>> >>> https://staging.kernelci.org:3000/d/build/build?orgId=1&var-dataset=playground_kernelci04&var-id=arm:2020-07-07:d3d7689c2cc9503266cac3bc777bb4ddae2e5f2e
>>>>>>>>>> >>>
>>>>>>>>>> >>> ..quick question though....given that now I'll have to play quite a bit
>>>>>>>>>> >>> with it and see how's better to present our data, if anythinjg missing etc etc,
>>>>>>>>>> >>> is there any chance (or way) that if I submmit the same JSON report multiple
>>>>>>>>>> >>> times with slight differences here and there (but with the same IDs clearly)
>>>>>>>>>> >>> I'll get my DB updated in the bits I have changed: as an example I've just
>>>>>>>>>> >>> resubmitted the same report with added discovery_time and descriptions, and got
>>>>>>>>>> >>> NO errors, but I cannot see the changes in the UI (unless they have still to
>>>>>>>>>> >>> propagate...)..or maybe I can obtain the same effect by dropping my dataset
>>>>>>>>>> >>> before re-submitting ?
>>>>>>>>>> >>>
>>>>>>>>>> >>> Regards
>>>>>>>>>> >>>
>>>>>>>>>> >>> Thanks
>>>>>>>>>> >>>
>>>>>>>>>> >>> Cristian
>>>>>>>>>> >>>
>>>>>>>>>> >>>> On 9/17/20 3:50 PM, Cristian Marussi wrote:
>>>>>>>>>> >>>>> Hi Nikolai,
>>>>>>>>>> >>>>>
>>>>>>>>>> >>>>> I work at ARM in the Kernel team and, in short, we'd like certainly to
>>>>>>>>>> >>>>> contribute our internal Kernel test results to KCIDB.
>>>>>>>>>> >>>>>
>>>>>>>>>> >>>>> After having attended your LPC2020 TestMC and KernelCI/BoF, I've now cooked
>>>>>>>>>> >>>>> up some KCIDB JSON test report (seemingly valid against your KCIDB v3 schema)
>>>>>>>>>> >>>>> and I'd like to start experimenting with kci-submit (on non-production
>>>>>>>>>> >>>>> instances), so as to assess how to fit our results into your schema and maybe
>>>>>>>>>> >>>>> contribute with some new KCIDB requirements if strictly needed.
>>>>>>>>>> >>>>>
>>>>>>>>>> >>>>> Is it possible to get some valid credentials and a playground instance to
>>>>>>>>>> >>>>> point at ?
>>>>>>>>>> >>>>>
>>>>>>>>>> >>>>> Thanks
>>>>>>>>>> >>>>>
>>>>>>>>>> >>>>> Regards
>>>>>>>>>> >>>>>
>>>>>>>>>> >>>>> Cristian
>>>>>>>>>> >>>>>
>>>>>>>>>> >>>>>
>>>>>>>>>> >>>>>
>>>>>>>>>> >>>>>
>>>>>>>>>> >>>>>
>>>>>>>>>> >>>>
>>>>>>>>>> >>>
>>>>>>>>>> >>
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > >
>>>>>>>>>> >
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>>
>>
>>
>>
>