[PATCH v2 0/7] Add perf testsuite into perf-test

[Michael Petlan <mpetlan@redhat.com>]

Hello all,

as Arnaldo said, this is a first patchset of an effort to upstream
the external perftool-testsuite [1] and merge it into perf-test.

The perftool-testsuite originates back to 2015, when there were no
shell-based testcases for perf. At that point, perf-test contained
rather "unit-tests" and there was a need to test perf as a tool too.

Nowadays, as perf-test contains shell tests too, it might make sense
to merge it in...

Apart from ~ 70 test scripts covering quite a lot of perf functionality,
it also brings design patterns that I consider advanced and useful:

  1) A possibility to keep logs and temp files for later fail
     investigation. This is necessary when the tests are running
     on machines that are immediately reinstalled and reused after
     the testrun. With the logs, many fails are easily narrowed
     down even when not having the machine online. Having variable
     PERFTEST_KEEP_LOGS=y keeps the logs in /tmp/.

  2) High level of configurability allows it to be run in deeper,
     thorough mode, as well as an "smoke testing" CI mode. Also,
     switching to different loads measured by perf is easy, just
     by setting CMD_SIMPLE (defaults to command `true`) to a
     user-defined application.

  3) Multi-level structure of the tests and subtests -- the tests
     are organized by perf-subcommands, test scripts and subtests
     within the scripts. Everything is reported in the results,
     when verbose output is enabled, so it is clear that e.g.
     perf-report failed due to preceding perf-record hadn't get
     samples, etc...
 
  4) Shared/unified result checking helpers and regular expressions

The perftool-testsuite is widely used at Red Hat for testing perf,
as far as I know, there are some users also outside of Red Hat,
but merging it into perf-test will help it spread and increase
the test coverage where it's still not known.

My plan is to move the whole testsuite under perf-test, respecting
both structures, as one perf-test "entry" per the perftool-testsuite's
"base_*" directory. Later possibly to align other shell tests with
it. 

[1] https://github.com/rfmvh/perftool-testsuite

========================

This is a v2 of the patchset, addressing the following issues:
  - shellcheck warnings
  - fixed regular expressions RE_LINE_TRACE_*, although they
    are not used yet
  - missing cover-letter
  
========================

Michael Petlan (1):
  perf testsuite: Install kprobe tests and common files

Veronika Molnarova (6):
  perf testsuite: Add common regex patters
  perf testsuite: Add common setting for shell tests
  perf testsuite: Add initialization script for shell tests
  perf testsuite: Add test case for perf probe
  perf testsuite: Add common output checking helpers
  perf testsuite: Add test for kprobe handling

 tools/perf/Makefile.perf                      |   5 +
 tools/perf/tests/shell/base_probe/settings.sh |  48 +++
 .../shell/base_probe/test_adding_kernel.sh    | 278 ++++++++++++++++++
 .../shell/common/check_all_lines_matched.pl   |  39 +++
 .../shell/common/check_all_patterns_found.pl  |  34 +++
 .../shell/common/check_no_patterns_found.pl   |  34 +++
 tools/perf/tests/shell/common/init.sh         | 117 ++++++++
 tools/perf/tests/shell/common/patterns.sh     | 268 +++++++++++++++++
 tools/perf/tests/shell/common/settings.sh     |  79 +++++
 .../tests/shell/perftool-testsuite_probe.sh   |  23 ++
 10 files changed, 925 insertions(+)
 create mode 100644 tools/perf/tests/shell/base_probe/settings.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_adding_kernel.sh
 create mode 100755 tools/perf/tests/shell/common/check_all_lines_matched.pl
 create mode 100755 tools/perf/tests/shell/common/check_all_patterns_found.pl
 create mode 100755 tools/perf/tests/shell/common/check_no_patterns_found.pl
 create mode 100644 tools/perf/tests/shell/common/init.sh
 create mode 100644 tools/perf/tests/shell/common/patterns.sh
 create mode 100644 tools/perf/tests/shell/common/settings.sh
 create mode 100755 tools/perf/tests/shell/perftool-testsuite_probe.sh

-- 
2.43.0