From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:organization:references:date:in-reply-to :message-id:user-agent:mime-version; bh=9FyldufiwN8fcDEZnELe9StVoslwnckPDC47JDxC5Jo=; b=Zw5FSk6Ek3IvY0lbRVtQjs9wnQBOrbOVmvl1fGmkXPXLp6CWG9+pgs/LXR+QMJ9I+J H/GQiDZP6G0+pCkWyQbAawDz2AcbOn4Lo+rJ/hILn//RjHwr7Rm3ETgarkl2ChB423vw Mjx5i7s+KsKJkhZ/25ZfmCNkvH85aQP3zFNCgWsrMSYqm6qY4sFdJXnM8rZiZwWK7PNV xx//s9dhfUc+ymCLjdLIn3LoxQ3ayh1ypUTgwWz7Eb4OUjBuRDqlvi/F39sMXLKSAy39 xoMNMv4OHKG7IK3Rs+Zjbyd69MX8Avrn9Jr+PtJ33mK7XqNSdg6KKf/oJ47LPER/48za tUlw== From: Kevin Hilman References: Date: Thu, 20 Apr 2017 11:13:28 -0700 In-Reply-To: (Timothy Bird's message of "Thu, 13 Apr 2017 19:19:22 +0000") Message-ID: MIME-Version: 1.0 Content-Type: text/plain Subject: Re: [Fuego] Discussion about Fuego unified results format List-Id: Mailing list for the Fuego test framework List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Bird, Timothy" Cc: "fuego@lists.linuxfoundation.org" , "automotive-discussions@lists.linuxfoundation.org" "Bird, Timothy" writes: > Based on discussion in our call yesterday, I have a few notes I'd like to make. > First - for those not involved in the call, here is some introduction. We are > working, for the 1.2 release, on unifying the results format produced by Fuego > tests, to make report generation easier, and support (ultimately) multiple > report output formats (e.g. HTML, XML, PDF, Excel). The intent is to support > all current AGL-JTA reports, previous Fuego reports, and new ones envisioned > for the system (in the 1.2 release). > > Note from the call and about this work are at: > http://bird.org/fuego/Unified_Results_Format_Project Has anyone looked at the JSON schema we designed for kernelci.org[1]? I wasn't involved in the meetings or call, but from what I can see, one important thing missing from your current proposal is how to group related tests together. In the schema we did for kCI, you can have test cases grouped into test sets, which in turn can be grouped into a test suite. IMO, this is crucial since most tests come as part of a larger suite. The kernelci.org project has prototyped this for several testsuites (kselftest, hackbench, lmbench, LTP, etc.) and were pushing JSON results using this /test API to our backend for awhile. But nobody got around to writing the parsing, reporting stuff yet. All of that to say the kCI JSON schema has been through through quite a bit already, and acually used for several test suites, so I think it would be a good idea to start with that and extend it, so combined efforts on this JSON test schema could benefit Fuego as well as other projects. Kevin [1] https://api.kernelci.org/schema-test.html https://api.kernelci.org/collection-test.html