Re: kcidb vs KernelCI APIs

Re: kcidb vs KernelCI APIs

[Nikolai Kondrashov]

Hi Minas,

Nice to hear from you again!
Please see my answers below.

On 4/5/23 19:30, Hambardzumyan, Minas wrote:
> Hi Nikolai,
> 
> I am a bit confused between kernel-ci api (https://api.kernelci.org/schema-test.html) and kcidb interfaces, can you help me better understand this? Let me know if my questions are too convoluted or I am not asking the right questions :) -- I'll try to clarify.
> 
> 1. Do kernel-ci api and ckidb interfaces operate on the same database? Based on #3 below I assume the answer is yes, but want to get your confirmation.

No. Rather both the current KernelCI Native API and the new KernelCI API 
systems are submitting their testing results to KCIDB, just like the rest of 
CI system participants.

In addition, KernelCI Native (and, I assume, the new KernelCI API in the 
future) support communicating with third-party labs maintaining their own 
hardware pools, accepting tests for execution, and reporting results.

Guillaume (CC'd) would be able to tell you much more about the latter two.

 From POV of KCIDB things look something like shown on this (somewhat 
outdated) slide:

https://docs.google.com/presentation/d/1dLs6SZfoNQ4BYYd-XfhWSqoJeYFBEUbiUYaQeVYj3EA/edit#slide=id.g1fb506119f1_0_91

I assume that the KernelCI (Native) API you were using accesses "KernelCI 
Backend" on that slide. You could also see there that its data is forwarded to 
KCIDB, to mix with the data coming from all other CI systems there.

> 
> 2. kernel-ci api defines schemas for "test_group" and "test_case" object types, while kcidb defines "tests" only. Can you tell me what is the relationship between these?

There's no direct relationship. However, the structure described by test 
group/case objects in KernelCI Native can be described by the universal "test" 
object in KCIDB, to an extent.

> 3. I am able to extract job/build/test_group/test_case objects using kernel-ci api interfaces. I was also able to testcase ids returned from kernel-ci api using a kcidb-query command like this (the downloaded object is attached):
> 
> kcidb-query -d bigquery:playground_kcidb_01 -t kernelci:staging.kernelci.org:642d52183d363bbbfd0b9bd3
> 
> Looking at the retuned json, I see the "id" is "kernelci:staging.kernelci.org:642d52183d363bbbfd0b9bd3".
> 	a) Where did this id string come from, and how do I create such an ID if I want to submit my own test result using kcidb?

This string was generated by KernelCI Native backend when submitting its data 
to KCIDB. I assume it took its internal test result ID 
"642d52183d363bbbfd0b9bd3", added "staging.kernelci.org:" to indicate it's 
coming from the staging instance, and to differentiate it from production 
results, and then added "kernelci:" to indicate this test result is coming 
from KernelCI itself, and to differentiate it from other CI systems.

The code doing that is likely here:
https://github.com/kernelci/kernelci-backend/blob/main/app/utils/kcidb.py

> 	b) Does it have to follow this syntax to be accessible from the  staging.kernelci.org web page?

I assume you can get your data on staging.kernelci.org (and 
linux.kernelci.org), only if you connect a pool of your machines as a 
KernelCI-controlled lab, but I'd leave it to Guillaume, or others to clarify.

> 	c) Do I need to first use kernel-ci api to create test_group/test_case with and get their ids, and then update it using kcidb?

If you simply want to have your testing results published in KCIDB, you don't 
need to talk to KernelCI Native API at all. You only need to follow the KCIDB 
Submitter Guide:

https://kernelci.org/docs/kcidb/submitter_guide/

to format your results as JSON and then push them to KCIDB.

Here's an example of what Syzkaller/Syzbot does:
https://github.com/google/syzkaller/blob/master/pkg/kcidb/client.go

Here's how GKernelCI does their submissions:
https://github.com/GKernelCI/Ghelper/tree/main/kcidb

However, if you want to have tests executed on your hardware by KernelCI, then 
you need to integrate with KernelCI Native API (I hope I'm not spouting total 
nonsense in this last part, Guillaume 🙈).

If you'd like, we can have a video call and go over all the questions you 
have, and hopefully clarify things. Perhaps Guillaume could join as well, 
depending on what you want to do.

Hope this helps!
Nick

Re: kcidb vs KernelCI APIs

[Nikolai Kondrashov]

On 4/6/23 11:40, Nikolai Kondrashov wrote:
>> Looking at the retuned json, I see the "id" is 
>> "kernelci:staging.kernelci.org:642d52183d363bbbfd0b9bd3".
>>     a) Where did this id string come from, and how do I create such an ID if 
>> I want to submit my own test result using kcidb?
> 
> This string was generated by KernelCI Native backend when submitting its data 
> to KCIDB. I assume it took its internal test result ID 
> "642d52183d363bbbfd0b9bd3", added "staging.kernelci.org:" to indicate it's 
> coming from the staging instance, and to differentiate it from production 
> results, and then added "kernelci:" to indicate this test result is coming 
> from KernelCI itself, and to differentiate it from other CI systems.
> 
> The code doing that is likely here:
> https://github.com/kernelci/kernelci-backend/blob/main/app/utils/kcidb.py

To clarify further: the submitting CI system is solely responsible for keeping 
the IDs of the objects it sends to KCIDB unique among themselves. You can 
generate them however you wish, as long as they don't clash between 
themselves. E.g. you could concatenate the name of the host where the test 
executed with a timestamp, and the test process PID, to get a unique-enough 
ID. Or get them from the database you're storing your test results in, if you 
have any.

You're required to format them as a string and prepend them with "ti:" before 
submitting to KCIDB, though. The latter ensures that your IDs don't clash with 
IDs from others.

Nick

Re: kcidb vs KernelCI APIs

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 648 bytes --]

On Thu, Apr 06, 2023 at 11:56:00AM +0300, Nikolai Kondrashov wrote:

> To clarify further: the submitting CI system is solely responsible for
> keeping the IDs of the objects it sends to KCIDB unique among themselves.
> You can generate them however you wish, as long as they don't clash between
> themselves. E.g. you could concatenate the name of the host where the test
> executed with a timestamp, and the test process PID, to get a unique-enough
> ID. Or get them from the database you're storing your test results in, if
> you have any.

For one practical example, I'm using git commit hashes for commits and
a UUID library for test results.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 484 bytes --]