[rt-tests v4 00/12] Generate machine-readable output

[Daniel Wagner <dwagner@suse.de>]

changes v4:
  - rebased on top of '[rt-tests v3 00/16] rt-numa.h cleanups' series
  - dropped applied patches
  - dropped RFC label

changes v3:
  - A number of bug fixes added at the beginning of
    the series.
  - Add --output option to all tests which have a
    numeric results, not just failed/passed

changes v2:
  - Moved the common JSON parts into rt-util.[ch]
  - Add --output option to signaltest

The current output of cyclictest is optimized for humans to read. This
is all good when working directly with the tools. But for CI
integration it's a bit of pain. Furthermore, not all rt-tests use the
same output format.

By using some easy to parse existing machine-readable format we can use
standard libraries to parse the data. For example in jitterdebug there
is a short Python program[1] to visualize either the histogram[2] or
all samples[3].

The implementation for JSON output for this is very simple. The last
patch adds a version of jitterdebugs's JSON output, which looks like

{
  "file_version": 1,
  "version:": "cyclictest V 1.90",
  "num_threads": 2,
  "resolution_in_ns": 0,
  "cmdline:": "./cyclictest --affinity=1-2 --duration=1s --output=dump.json -h 1000 -p 80",
  "sysinfo": {
    "sysname": "Linux",
    "nodename": "beryllium",
    "release": "5.9.14-1-default",
    "version": "#1 SMP Sat Dec 12 06:57:32 UTC 2020 (c648a46)",
    "machine": "x86_64"
  },
  "thread": {
    "0": {
      "histogram": {
        "0": 16,
        "1": 853,
        "2": 80,
        "3": 50,
        "4": 1
      },
      "cycles": 1000,
      "min": 0,
      "max": 4,
      "avg": 1.17,
      "cpu": 1,
      "node": 0
    },
    "1": {
      "histogram": {
        "0": 14,
        "1": 833,
        "2": 93,
        "3": 56,
        "4": 4
      },
      "cycles": 1000,
      "min": 0,
      "max": 4,
      "avg": 1.20,
      "cpu": 2,
      "node": 0
    }
  }
}

It's just a rough version. I didn't try to make it generic for the
other rt-tests or make it as plugin as John was suggesting. I'd think
we could make this feature as compile option, if you want to keep the
program small. Obviously, we could also make the terminal output a
compile option, to keep it small.

Anyway, what do you think about it?

Thanks,
Daniel



*** BLURB HERE ***

Daniel Wagner (12):
  cyclictest: Move thread data to struct thread_param
  signaltest: Move thread data to struct thread_param
  rt-utils: Add JSON common header output helper
  cyclictest: Add JSON output feature
  signaltest: Add JSON output feature
  cyclicdeadline: Add JSON output feature
  pmqtest: Add JSON output feature
  ptsematest: Add JSON output feature
  sigwaittest: Add JSON output feature
  svsematest: Add JSON output feature
  oslat: Add JSON output feature
  rt-migrate-test: Add JSON output feature

 src/cyclictest/cyclictest.c           |  80 +++++++++++---
 src/include/rt-utils.h                |   4 +
 src/lib/rt-utils.c                    |  70 ++++++++++++
 src/oslat/oslat.c                     |  97 +++++++++++++---
 src/pmqtest/pmqtest.c                 | 137 +++++++++++++++++++----
 src/ptsematest/ptsematest.c           | 120 ++++++++++++++++----
 src/rt-migrate-test/rt-migrate-test.c | 118 ++++++++++++++++----
 src/sched_deadline/cyclicdeadline.c   |  72 +++++++++---
 src/signaltest/signaltest.c           | 153 ++++++++++++++++++--------
 src/sigwaittest/sigwaittest.c         | 117 +++++++++++++++++---
 src/svsematest/svsematest.c           | 124 +++++++++++++++++----
 11 files changed, 901 insertions(+), 191 deletions(-)

-- 
2.30.0