kernelci.org update - 2020-09-23 #minutes

["Guillaume Tucker" <guillaume.tucker@collabora.com>]

Summary of changes going into production
========================================

* fix branch names with slash characters "/"
* enable stable-rc queue/* branches
* enable stable 5.8 branches
* enable soc arm/fixes branch
* build linux-next with clang-10, drop clang-9
* use Linaro test-definitions for kselftest
* enable kselftest to run on a few initial devices
* add direct links to regressions on web dashboard
* improve log filtering to remove more LAVA messages


Technical Steering Committee minutes - 2020-09-08
=================================================

* LAVA test-definitions repository and version control

  * No new tags upstream since “2019.11”, should we ask Linaro about restarting
    that?
  * Should we make a fork in kernelci GitHub with kernelci.org branch and
    kernelci tags?
  * Show we accept to use the head of upstream master branch, even if this
    means hard-to-reproduce issues and unexpected change in results if new
    commits are pushed at any time?

  -> create fork for staging initially, and try to use upstream in production

* Finally fixed support for slashes in branch names, useful for stable-rc queue
  branches in particular

* LLVM/CLang: still waiting for v11 to be released

* LAVA log filtering: tested on staging - seems to work fine

* Login test case fix: implemented more thorough testing needed

* preempt-rt: Linaro test-definitions repo getting updates from Daniel Wagner

* KCIDB

  JSON stream parsing is implemented and tested in jq.py, upstreaming is in
  progress. Still takes four times as long and uses twice the memory as the
  stock JSON parser, but does streams and should be good enough for now.
  Starting implementing report stream parsing in KCIDB.

* Notes: rework test email reports to show number of regressions in table
  -> https://github.com/kernelci/kernelci-backend/issues/257


Technical Steering Committee minutes - 2020-09-15
=================================================

* LLVM/Clang

  LLVM people not super interested in older clang versions or even clang-10.
  Once clang-9 is out, when Mark updates to clang-11 he’ll send something also
  dropping the older clang versions once clang-11 is live. Can reevaluate this
  policy once there are stable distros with clang.

  Nick maintains a clang-latest docker image, we should into integrating that
  for potential inclusion in linux-next coverage - will need some evaluation to
  see for example how noisy it is and if we need to do something about
  segregating results for the bleeding edge compiler.

  LLVM 11 release:
  * Status tracked at https://bugs.llvm.org/show_bug.cgi?id=46725
  * Several pending bugs need fixing, some look relevant

  Android LLVM versions: Mark to ping Todd and ask him about using those for
  Android branches.

  Testing with clang-12: could take a short cut and deploy on staging rather
  than sorting out fancier reporting

* KCIDB

  Half of JSON stream parsing is merged into jq.py. Another half remains,
  pending on the maintainer's attention.

  KCIDB input stream parsing implemented locally, to be tested, and waiting on
  the jq.py PR above.

  KCIDB output splitting next.

* Web dashboard

  Improving web UI to differentiate skips and tests that have always failed
  See on https://staging.kernelci.org
  (pie charts and small things left to tweak)

* Using Linaro test-definitions for LAVA jobs

  Created test-definitions fork for kernelci.org and staging.kernelci.org
  kernelci.org branch updated weekly with prod update
  staging.kernelci.org branch updated with open PRs like other projects

* kselftests: build errors mixed with main kernel build

  Long-term solution would be to have separate stages (kernel build, kselftest
  build, runtime) with dependencies and pass/fail status

  Short-term solution and necessary step is to build kselftests as a separate
  “make” command and keep the output in a separate log file (like tuxmake does)

* login test case false-positives:

  On LAVA 2020.08 kernel panic is detected, test case status marked as failed
  LAVA job Incomplete


Technical Steering Committee minutes - 2020-09-22
=================================================

* KCIDB

  The jq.py maintainer is slow to respond and keeps making requests, official
  streaming support doesn't seem close anymore, so I made a release for
  temporary use in my personal fork.

  JSON streaming support is merged into KCIDB - integration work can be
  started!  You can now write reports one after another into any tool accepting
  reports.  Submitting 1000 reports containing 8000 objects through
  kcidb-submit takes about 35 seconds now.

  Kcidb-query, kcidb-db-query, and kcidb-db-dump now accept the
  '-o/--objects-per-report' option specifying how many objects should be put
  into each output JSON report. When that option is used, they can output
  multiple reports.

  All tools outputting JSON can be asked to not pretty-print it (with
  '--indent=0'), outputting single-line, or to prepend each report with the RS
  character (with '--seq'), complying with RFC 7464, either of which could be
  easier to process with command-line tools.

  Next KCIDB release soon. Theme: JSON streaming support.

  Cristian Marussi from ARM is setting up sending their CI results, and managed
  to push a bunch to the playground setup - looks great!

* Plumbers 2020 KernelCI blog post: ready to get published this week

* KernelCI TSC plan: Starting to make plan for 2020-Q4

* E-mail regression reports: fixing formatting when more than one regression

* User experience:

  KernelCI LF board starting to brainstorm around next-gen dashboard /
  visualization / analytics to potentially fund a 3rd party to develop web
  tooling

* "unknown" failures on web UI

  https://groups.io/g/kernelci/topic/user_interface_issues_with/76927781?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,76927781

  -> Yes this looks very much like the web frontend issue described before,
     being fixed right now (separating regressions, always-fail and skips)
     https://github.com/kernelci/kernelci-frontend/pull/125

* EFI on QEMU

  u-boot with EFI extensions on arm/arm64 to get coverage for the EFI boot
  paths in the kernel - Might just do this via the FVP.

* clang-11

  Can merge the docker update, will need to rebuild when the final LLVM 11
  release lands (hopefully RSN, the tracking bug looks to have mostly no
  dependencies).


Advisory Board minutes - 2020-09-16
===================================

* LPC recorded talks are now on YouTube, time to tweet about them

* Discussing priorities for 2020-Q4 TSC plan:

  Native tests: kselftest, LTP, KUnit, device tree validation
  KCIDB with production data from kernelci.org
  “Polishing”: Improving docs, fixing long-standing bugs, LF project PDFs…
  Metrics: discussed many times, now we should start implementing it
  CIP KernelCI instance: already discussed, also needs some action now

* Discussing next important topics, for 2021

  Define some work packages / RFQs
  Testing patches from mailing lists
  Improve regressions tracking for native KernelCI tests


Best wishes,
Guillaume