[PATCH 0/9] tweak results organization and reporting

[PATCH 0/9] tweak results organization and reporting

[Scott Mayhew]

I do most of my kdevops runs under buildbot.  Currently I don't have an
easy way to view test results in buildbot's web interface. 

The first four patches implement a "last-run" directory and
"last-kernel.txt" file similar to what the fstests workflow already has.
This is mainly to facilitate the remaining patches, but I think it will
also make it easier for me to feed my test results into other tooling in
the future.

The remaining patches add a "<workflow>-show-results" makefile target,
which basically print the test results to the standard output.  The
results for the most recent run are printed by default, but that can be
changed by overriding the LAST_KERNEL variable.  The results printed
varies from workflow to workflow, and can be overidden via the PATTERN
variable.

Scott Mayhew (9):
  pynfs: add last-run directory and last-kernel.txt to the results
  nfstest: add last-run directory and last-kernel.txt to the results
  gitr: add last-run directory and last-kernel.txt to the results
  ltp: add last-run directory and last-kernel.txt to the results
  fstests: add makefile target to show test results
  pynfs: add makefile target to show test results
  nfstest: add makefile target to show test results
  gitr: add makefile target to show test results
  ltp: add makefile target to show test results

 playbooks/roles/gitr/tasks/main.yml    | 59 +++++++++++++++++++++----
 playbooks/roles/ltp/tasks/main.yml     | 60 ++++++++++++++++++++++++--
 playbooks/roles/nfstest/tasks/main.yml | 60 ++++++++++++++++++++++----
 playbooks/roles/pynfs/tasks/main.yml   | 38 +++++++++++++++-
 workflows/fstests/Makefile             | 16 +++++++
 workflows/gitr/Makefile                | 21 +++++++++
 workflows/ltp/Makefile                 | 21 +++++++++
 workflows/nfstest/Makefile             | 17 ++++++++
 workflows/pynfs/Makefile               | 12 ++++++
 9 files changed, 282 insertions(+), 22 deletions(-)

-- 
2.48.1

[PATCH 0/9] tweak results organization and reporting

[Scott Mayhew]

I do most of my kdevops runs under buildbot.  Currently I don't have an
easy way to view test results in buildbot's web interface. 

The first four patches implement a "last-run" directory and
"last-kernel.txt" file similar to what the fstests workflow already has.
This is mainly to facilitate the remaining patches, but I think it will
also make it easier for me to feed my test results into other tooling in
the future.

The remaining patches add a "<workflow>-show-results" makefile target,
which basically print the test results to the standard output.  The
results for the most recent run are printed by default, but that can be
changed by overriding the LAST_KERNEL variable.  The results printed
varies from workflow to workflow, and can be overidden via the PATTERN
variable.

Scott Mayhew (9):
  pynfs: add last-run directory and last-kernel.txt to the results
  nfstest: add last-run directory and last-kernel.txt to the results
  gitr: add last-run directory and last-kernel.txt to the results
  ltp: add last-run directory and last-kernel.txt to the results
  fstests: add makefile target to show test results
  pynfs: add makefile target to show test results
  nfstest: add makefile target to show test results
  gitr: add makefile target to show test results
  ltp: add makefile target to show test results

 playbooks/roles/gitr/tasks/main.yml    | 59 +++++++++++++++++++++----
 playbooks/roles/ltp/tasks/main.yml     | 60 ++++++++++++++++++++++++--
 playbooks/roles/nfstest/tasks/main.yml | 60 ++++++++++++++++++++++----
 playbooks/roles/pynfs/tasks/main.yml   | 38 +++++++++++++++-
 workflows/fstests/Makefile             | 16 +++++++
 workflows/gitr/Makefile                | 21 +++++++++
 workflows/ltp/Makefile                 | 21 +++++++++
 workflows/nfstest/Makefile             | 17 ++++++++
 workflows/pynfs/Makefile               | 12 ++++++
 9 files changed, 282 insertions(+), 22 deletions(-)

-- 
2.48.1

[PATCH] Don't include the codereadyrepo role if we're using a custom yum repofile

[Scott Mayhew]

If we're using a custom yum repofile, there's no guarantee the CRB repo
follows the expected naming convention or that a CRB repo even exists
for our setup.

Signed-off-by: Scott Mayhew <smayhew@redhat.com>
---
 playbooks/roles/codereadyrepo/tasks/main.yml                | 5 -----
 playbooks/roles/fstests/tasks/install-deps/redhat/main.yml  | 1 +
 playbooks/roles/ltp/tasks/main.yml                          | 1 +
 playbooks/roles/pynfs/tasks/install-deps/redhat/main.yml    | 3 ++-
 playbooks/roles/sysbench/tasks/install-deps/redhat/main.yml | 1 +
 5 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/playbooks/roles/codereadyrepo/tasks/main.yml b/playbooks/roles/codereadyrepo/tasks/main.yml
index 1b8bad0..45e6f7a 100644
--- a/playbooks/roles/codereadyrepo/tasks/main.yml
+++ b/playbooks/roles/codereadyrepo/tasks/main.yml
@@ -4,7 +4,6 @@
     codeready_repo: "codeready-builder-for-rhel-{{ ansible_distribution_major_version }}-{{ ansible_architecture }}-rpms"
   when:
     - ansible_distribution == 'RedHat'
-    - not devconfig_custom_yum_repofile
     - kdevops_enable_guestfs
 
 - name: Select the Oracle Linux CodeReady Builder repo
@@ -12,14 +11,12 @@
     codeready_repo: "ol9_codeready_builder"
   when:
     - ansible_distribution == 'OracleLinux'
-    - not devconfig_custom_yum_repofile
 
 - name: Select the CentOS CodeReady Builder repo
   ansible.builtin.set_fact:
     codeready_repo: "crb"
   when:
     - ansible_distribution == 'CentOS'
-    - not devconfig_custom_yum_repofile
     - kdevops_enable_guestfs
 
 - name: Select the AWS RHEL CodeReady Builder repo
@@ -27,7 +24,6 @@
     codeready_repo: "codeready-builder-for-rhel-{{ ansible_distribution_major_version }}-rhui-rpms"
   when:
     - ansible_distribution == 'RedHat'
-    - not devconfig_custom_yum_repofile
     - kdevops_enable_terraform
     - kdevops_terraform_provider == "aws"
 
@@ -36,7 +32,6 @@
     codeready_repo: "codeready-builder-for-rhel-{{ ansible_distribution_major_version }}-{{ ansible_architecture }}-rhui-rpms"
   when:
     - ansible_distribution == 'RedHat'
-    - not devconfig_custom_yum_repofile
     - kdevops_enable_terraform
     - kdevops_terraform_provider == "azure"
 
diff --git a/playbooks/roles/fstests/tasks/install-deps/redhat/main.yml b/playbooks/roles/fstests/tasks/install-deps/redhat/main.yml
index 52b4bce..c8df399 100644
--- a/playbooks/roles/fstests/tasks/install-deps/redhat/main.yml
+++ b/playbooks/roles/fstests/tasks/install-deps/redhat/main.yml
@@ -4,6 +4,7 @@
     name: codereadyrepo
   when:
     - ansible_distribution != "Fedora"
+    - not devconfig_custom_yum_repofile
 
 - name: Enable installation of packages from EPEL
   ansible.builtin.include_role:
diff --git a/playbooks/roles/ltp/tasks/main.yml b/playbooks/roles/ltp/tasks/main.yml
index b15f87d..f6cf97a 100644
--- a/playbooks/roles/ltp/tasks/main.yml
+++ b/playbooks/roles/ltp/tasks/main.yml
@@ -48,6 +48,7 @@
   when:
     - ansible_os_family == "RedHat"
     - ansible_distribution != "Fedora"
+    - not devconfig_custom_yum_repofile
 
 - name: Install dependencies for ltp
   tags: ['build', 'ltp']
diff --git a/playbooks/roles/pynfs/tasks/install-deps/redhat/main.yml b/playbooks/roles/pynfs/tasks/install-deps/redhat/main.yml
index b98ff67..2375e93 100644
--- a/playbooks/roles/pynfs/tasks/install-deps/redhat/main.yml
+++ b/playbooks/roles/pynfs/tasks/install-deps/redhat/main.yml
@@ -3,7 +3,8 @@
   ansible.builtin.include_role:
     name: codereadyrepo
   when:
-    ansible_distribution != "Fedora"
+    - ansible_distribution != "Fedora"
+    - not devconfig_custom_yum_repofile
 
 - name: Install build dependencies for pynfs
   become: yes
diff --git a/playbooks/roles/sysbench/tasks/install-deps/redhat/main.yml b/playbooks/roles/sysbench/tasks/install-deps/redhat/main.yml
index d5021e0..47832b2 100644
--- a/playbooks/roles/sysbench/tasks/install-deps/redhat/main.yml
+++ b/playbooks/roles/sysbench/tasks/install-deps/redhat/main.yml
@@ -4,6 +4,7 @@
     name: codereadyrepo
   when:
     - ansible_distribution != "Fedora"
+    - not devconfig_custom_yum_repofile
 
 - name: Enable installation of packages from EPEL
   ansible.builtin.include_role:
-- 
2.48.1

[PATCH 1/9] pynfs: add last-run directory and last-kernel.txt to the results

[Scott Mayhew]

Create the last-run directory and last-kernel.txt file in a manner
similar to the way the fstests workflow does it.

This will make it easier to get at the most recent test results for
reporting purposes.

Signed-off-by: Scott Mayhew <smayhew@redhat.com>
---
 playbooks/roles/pynfs/tasks/main.yml | 38 ++++++++++++++++++++++++++--
 1 file changed, 36 insertions(+), 2 deletions(-)

diff --git a/playbooks/roles/pynfs/tasks/main.yml b/playbooks/roles/pynfs/tasks/main.yml
index c6add8e..882dd1c 100644
--- a/playbooks/roles/pynfs/tasks/main.yml
+++ b/playbooks/roles/pynfs/tasks/main.yml
@@ -12,6 +12,8 @@
 - name: Set the path where we collect our local pynfs results
   set_fact:
     pynfs_workflow_dir: "../workflows/pynfs"
+    pynfs_results_full_path: "{{ topdir_path }}/workflows/pynfs/results"
+    pynfs_results_target: "../workflows/pynfs/results/last-run/"
   tags: [ 'vars' ]
 
 - name: Clean up our localhost results directory and files
@@ -127,6 +129,16 @@
   delay: 1
   until: result.rc == 0
 
+- name: Clean up our localhost results/last-run directory
+  local_action: file path="{{ pynfs_results_target }}/" state=absent
+  run_once: true
+  tags: [ 'run_tests', 'copy_results', 'clean_local_results' ]
+
+- name: Create empty last-run directory
+  local_action: file path="{{ pynfs_results_target }}/" state=directory
+  run_once: true
+  tags: [ 'run_tests', 'copy_results', 'clean_local_results' ]
+
 - name: Run pynfs
   tags: [ 'run_tests' ]
   script: "{{ pynfs_workflow_dir }}/scripts/run_pynfs.sh"
@@ -156,6 +168,11 @@
     kernel_rev: "{{ uname_cmd.stdout_lines | regex_replace('\\]') | regex_replace('\\[') | replace(\"'\",'') }}"
   run_once: true
 
+- name: Document used target kernel version
+  local_action: "shell echo {{ kernel_rev }} > {{ pynfs_results_target }}/../last-kernel.txt"
+  tags: [ 'run_tests', 'copy_results', 'print_results' ]
+  run_once: true
+
 - name: Copy the version-specific results to the control node
   tags: [ 'copy_results' ]
   become: yes
@@ -163,7 +180,7 @@
   become_method: sudo
   fetch:
     src: "{{ pynfs_data }}/pynfs-{{ item }}-results.json"
-    dest: "{{ pynfs_workflow_dir }}/results/{{ kernel_rev }}-v{{ item }}.json"
+    dest: "{{ pynfs_results_full_path }}/last-run/{{ kernel_rev}}/{{ kernel_rev }}-v{{ item }}.json"
     flat: yes
   with_items:
     - 4.0
@@ -176,9 +193,26 @@
   become_method: ansible.builtin.sudo
   fetch:
     src: "{{ pynfs_data }}/pynfs-{{ item }}-results.json"
-    dest: "{{ pynfs_workflow_dir }}/results/{{ kernel_rev }}-v{{ item }}.json"
+    dest: "{{ pynfs_results_full_path }}/last-run/{{ kernel_rev}}/{{ kernel_rev }}-v{{ item }}.json"
     flat: true
   with_items:
     - block
   when:
     - pynfs_pnfs_block|bool
+
+- name: Verify last-run kernel directory exists
+  tags: [ 'copy_results' ]
+  delegate_to: localhost
+  stat:
+    path: "{{ pynfs_results_full_path }}/last-run/{{ kernel_rev }}"
+  register: last_run_kernel_dir
+
+- name: Copy over the last-run into the results directory for archiving into kdevops
+  copy:
+    src: "{{ pynfs_results_full_path }}/last-run/{{ kernel_rev }}"
+    dest: "{{ pynfs_results_full_path }}/"
+  run_once: true
+  tags: [ 'copy_results' ]
+  delegate_to: localhost
+  when:
+    - last_run_kernel_dir.stat.exists
-- 
2.48.1

[PATCH 2/9] nfstest: add last-run directory and last-kernel.txt to the results

[Scott Mayhew]

Create the last-run directory and last-kernel.txt file in a manner
similar to the way the fstests workflow does it.

This will make it easier to get at the most recent test results for
reporting purposes.

Signed-off-by: Scott Mayhew <smayhew@redhat.com>
---
 playbooks/roles/nfstest/tasks/main.yml | 60 ++++++++++++++++++++++----
 1 file changed, 52 insertions(+), 8 deletions(-)

diff --git a/playbooks/roles/nfstest/tasks/main.yml b/playbooks/roles/nfstest/tasks/main.yml
index 8c8db02..bf5cd9f 100644
--- a/playbooks/roles/nfstest/tasks/main.yml
+++ b/playbooks/roles/nfstest/tasks/main.yml
@@ -131,6 +131,24 @@
   tags:
     - run_tests
 
+- name: Set the pathname of the results directory
+  ansible.builtin.set_fact:
+    nfstest_results_full_path: "{{ topdir_path }}/workflows/nfstest/results"
+    nfstest_results_target: "../workflows/nfstest/results/last-run"
+  when:
+    - kdevops_workflows_dedicated_workflow|bool
+  tags: copy_results
+
+- name: Clean up our localhost results/last-run directory
+  local_action: file path="{{ nfstest_results_target }}/" state=absent
+  run_once: true
+  tags: [ 'run_tests', 'copy_results', 'clean_local_results' ]
+
+- name: Create empty last-run directory
+  local_action: file path="{{ nfstest_results_target }}/" state=directory
+  run_once: true
+  tags: [ 'run_tests', 'copy_results', 'clean_local_results' ]
+
 - name: Run the test group script on the target node
   ansible.builtin.command:
     cmd: "/tmp/runtest.sh"
@@ -138,16 +156,25 @@
     - kdevops_workflows_dedicated_workflow|bool
   tags: run_tests
 
-- name: Set the pathname of the results directory
-  ansible.builtin.set_fact:
-    nfstest_results_dir: "../workflows/nfstest/results/{{ ansible_kernel }}"
-  when:
-    - kdevops_workflows_dedicated_workflow|bool
-  tags: copy_results
+- name: Get used target kernel version
+  tags: [ 'copy_results' ]
+  command: "uname -r"
+  register: uname_cmd
+
+- name: Store last kernel variable
+  tags: [ 'copy_results' ]
+  set_fact:
+    last_kernel: "{{ uname_cmd.stdout_lines | regex_replace('\\]') | regex_replace('\\[') | replace(\"'\",'') }}"
+  run_once: true
+
+- name: Document used target kernel version
+  local_action: "shell echo {{ last_kernel }} > {{ nfstest_results_target }}/../last-kernel.txt"
+  tags: [ 'run_tests', 'copy_results', 'print_results' ]
+  run_once: true
 
 - name: Create the results directory on the control node
   ansible.builtin.file:
-    path: "{{ nfstest_results_dir }}"
+    path: "{{ nfstest_results_full_path }}/last-run/{{ last_kernel }}"
     state: directory
     mode: "u=rwx,g=rx,o=rx"
   delegate_to: localhost
@@ -168,7 +195,7 @@
 - name: Copy the output logs to the control node
   ansible.builtin.fetch:
     src: "{{ item.path }}"
-    dest: "{{ nfstest_results_dir }}/{{ nfstest_test_group }}/"
+    dest: "{{ nfstest_results_full_path }}/last-run/{{ last_kernel }}/{{ nfstest_test_group }}/"
     flat: true
   with_items: "{{ nfstest_logs.files }}"
   loop_control:
@@ -178,3 +205,20 @@
     - kdevops_workflows_dedicated_workflow|bool
   changed_when: false
   tags: copy_results
+
+- name: Verify last-run kernel directory exists
+  tags: [ 'copy_results' ]
+  delegate_to: localhost
+  stat:
+    path: "{{ nfstest_results_full_path }}/last-run/{{ last_kernel }}"
+  register: last_run_kernel_dir
+
+- name: Copy over the last-run into the results directory for archiving into kdevops
+  copy:
+    src: "{{ nfstest_results_full_path }}/last-run/{{ last_kernel }}"
+    dest: "{{ nfstest_results_full_path }}/"
+  run_once: true
+  tags: [ 'copy_results' ]
+  delegate_to: localhost
+  when:
+    - last_run_kernel_dir.stat.exists
-- 
2.48.1

[PATCH 3/9] gitr: add last-run directory and last-kernel.txt to the results

[Scott Mayhew]

Create the last-run directory and last-kernel.txt file in a manner
similar to the way the fstests workflow does it.

This will make it easier to get at the most recent test results for
reporting purposes.

Signed-off-by: Scott Mayhew <smayhew@redhat.com>
---
 playbooks/roles/gitr/tasks/main.yml | 59 +++++++++++++++++++++++++----
 1 file changed, 51 insertions(+), 8 deletions(-)

diff --git a/playbooks/roles/gitr/tasks/main.yml b/playbooks/roles/gitr/tasks/main.yml
index 42005d2..f1e7b6c 100644
--- a/playbooks/roles/gitr/tasks/main.yml
+++ b/playbooks/roles/gitr/tasks/main.yml
@@ -24,17 +24,43 @@
 - name: Set the pathname of the local results directory
   tags: vars
   ansible.builtin.set_fact:
-    gitr_results_dir: "../workflows/gitr/results/{{ ansible_kernel }}/{{ gitr_test_group }}/"
+    gitr_results_full_path: "{{ topdir_path }}/workflows/gitr/results"
+    gitr_results_target: "../workflows/gitr/results/last-run"
     gitr_run_uniqifier: "{{ ansible_date_time.iso8601_basic_short }}"
 
+- name: Clean up our localhost results/last-run directory
+  local_action: file path="{{ gitr_results_target }}/" state=absent
+  run_once: true
+  tags: [ 'run_tests', 'copy_results', 'clean_local_results' ]
+
+- name: Create empty last-run directory
+  local_action: file path="{{ gitr_results_target }}/" state=directory
+  run_once: true
+  tags: [ 'run_tests', 'copy_results', 'clean_local_results' ]
+
+- name: Get used target kernel version
+  tags: [ 'copy_results' ]
+  command: "uname -r"
+  register: uname_cmd
+
+- name: Store last kernel variable
+  tags: [ 'copy_results' ]
+  set_fact:
+    last_kernel: "{{ uname_cmd.stdout_lines | regex_replace('\\]') | regex_replace('\\[') | replace(\"'\",'') }}"
+  run_once: true
+
+- name: Document used target kernel version
+  local_action: "shell echo {{ last_kernel }} > {{ gitr_results_target }}/../last-kernel.txt"
+  tags: [ 'run_tests', 'copy_results', 'print_results' ]
+  run_once: true
+
 - name: Ensure the local results directory exists
   tags: copy_results
   ansible.builtin.file:
-    path: "{{ gitr_results_dir }}"
+    path: "{{ gitr_results_full_path }}/last-run/{{ last_kernel }}/{{ gitr_test_group }}"
     state: directory
     mode: "u=rwx,g=rx,o=rx"
   delegate_to: localhost
-  run_once: true
 
 - name: Install dependencies for the git regression tests
   tags: gitr
@@ -303,7 +329,7 @@
   tags: copy_results
   ansible.builtin.fetch:
     src: "/tmp/{{ gitr_run_uniqifier }}.tgz"
-    dest: "{{ gitr_results_dir }}"
+    dest: "{{ gitr_results_full_path }}/last-run/{{ last_kernel }}/{{ gitr_test_group }}/"
     flat: true
     validate_checksum: true
 
@@ -311,7 +337,7 @@
   tags: copy_results
   ansible.builtin.fetch:
     src: "{{ gitr_mnt }}/{{ gitr_run_uniqifier }}.summary"
-    dest: "{{ gitr_results_dir }}"
+    dest: "{{ gitr_results_full_path }}/last-run/{{ last_kernel }}/{{ gitr_test_group }}/"
     flat: true
     validate_checksum: false
 
@@ -319,7 +345,7 @@
   tags: copy_results
   ansible.builtin.fetch:
     src: "{{ gitr_mnt }}/{{ gitr_run_uniqifier }}.stderr"
-    dest: "{{ gitr_results_dir }}"
+    dest: "{{ gitr_results_full_path }}/last-run/{{ last_kernel }}/{{ gitr_test_group }}/"
     flat: true
     validate_checksum: false
   when: gitr_results.rc != 0
@@ -328,7 +354,7 @@
   tags: copy_results
   ansible.builtin.fetch:
     src: "{{ gitr_mnt }}/{{ gitr_run_uniqifier }}.rpc"
-    dest: "{{ gitr_results_dir }}"
+    dest: "{{ gitr_results_full_path }}/last-run/{{ last_kernel }}/{{ gitr_test_group }}/"
     flat: true
     validate_checksum: false
   when:
@@ -339,9 +365,26 @@
   tags: copy_results
   ansible.builtin.fetch:
     src: "{{ gitr_mnt }}/{{ gitr_run_uniqifier }}.xprt"
-    dest: "{{ gitr_results_dir }}"
+    dest: "{{ gitr_results_full_path }}/last-run/{{ last_kernel }}/{{ gitr_test_group }}/"
     flat: true
     validate_checksum: false
   when:
     - gitr_fstype == "nfs"
     - ansible_os_family == 'RedHat'
+
+- name: Verify last-run kernel directory exists
+  tags: [ 'copy_results' ]
+  delegate_to: localhost
+  stat:
+    path: "{{ gitr_results_full_path }}/last-run/{{ last_kernel }}"
+  register: last_run_kernel_dir
+
+- name: Copy over the last-run into the results directory for archiving into kdevops
+  copy:
+    src: "{{ gitr_results_full_path }}/last-run/{{ last_kernel }}"
+    dest: "{{ gitr_results_full_path }}/"
+  run_once: true
+  tags: [ 'copy_results' ]
+  delegate_to: localhost
+  when:
+    - last_run_kernel_dir.stat.exists
-- 
2.48.1

[PATCH 4/9] ltp: add last-run directory and last-kernel.txt to the results

[Scott Mayhew]

Create the last-run directory and last-kernel.txt file in a manner
similar to the way the fstests workflow does it.

This will make it easier to get at the most recent test results for
reporting purposes.

Signed-off-by: Scott Mayhew <smayhew@redhat.com>
---
 playbooks/roles/ltp/tasks/main.yml | 60 ++++++++++++++++++++++++++++--
 1 file changed, 56 insertions(+), 4 deletions(-)

diff --git a/playbooks/roles/ltp/tasks/main.yml b/playbooks/roles/ltp/tasks/main.yml
index 34cae23..71e1541 100644
--- a/playbooks/roles/ltp/tasks/main.yml
+++ b/playbooks/roles/ltp/tasks/main.yml
@@ -20,17 +20,52 @@
   tags: ['vars']
   ansible.builtin.set_fact:
     ltp_test_group: "{{ ansible_host | regex_replace(kdevops_host_prefix + '-') | regex_replace('-dev') }}"
-    ltp_results_dir: "../workflows/ltp/results/{{ ansible_kernel }}"
+    ltp_results_full_path: "{{ topdir_path }}/workflows/ltp/results"
+    ltp_results_target: "../workflows/ltp/results/last-run"
 
 - name: Create the local results directory
   tags: ['first_run']
   ansible.builtin.file:
-    path: "{{ ltp_results_dir }}"
+    path: "{{ ltp_results_full_path }}"
     state: directory
     mode: "u=rwx,g=rx,o=rx"
   delegate_to: localhost
   run_once: true
 
+- name: Clean up our localhost results/last-run directory
+  local_action: file path="{{ ltp_results_target }}/" state=absent
+  run_once: true
+  tags: [ 'run_tests', 'copy_results', 'clean_local_results' ]
+
+- name: Create empty last-run directory
+  local_action: file path="{{ ltp_results_target }}/" state=directory
+  run_once: true
+  tags: [ 'run_tests', 'copy_results', 'clean_local_results' ]
+
+- name: Get used target kernel version
+  tags: [ 'copy_results' ]
+  command: "uname -r"
+  register: uname_cmd
+
+- name: Store last kernel variable
+  tags: [ 'copy_results' ]
+  set_fact:
+    last_kernel: "{{ uname_cmd.stdout_lines | regex_replace('\\]') | regex_replace('\\[') | replace(\"'\",'') }}"
+  run_once: true
+
+- name: Document used target kernel version
+  local_action: "shell echo {{ last_kernel }} > {{ ltp_results_target }}/../last-kernel.txt"
+  tags: [ 'run_tests', 'copy_results', 'print_results' ]
+  run_once: true
+
+- name: Ensure the local results directory exists
+  tags: copy_results
+  ansible.builtin.file:
+    path: "{{ ltp_results_full_path }}/last-run/{{ last_kernel }}/{{ ltp_test_group }}"
+    state: directory
+    mode: "u=rwx,g=rx,o=rx"
+  delegate_to: localhost
+
 - name: Set OS-specific variables
   ansible.builtin.include_vars: "{{ lookup('ansible.builtin.first_found', params) }}"
   vars:
@@ -194,7 +229,7 @@
   tags: ['copy_results']
   ansible.builtin.fetch:
     src: "{{ item.path }}"
-    dest: "{{ ltp_results_dir }}/{{ ltp_test_group }}/"
+    dest: "{{ ltp_results_full_path }}/last-run/{{ last_kernel }}/{{ ltp_test_group }}/"
     flat: true
     validate_checksum: false
   with_items: "{{ results_files.files }}"
@@ -216,7 +251,7 @@
   tags: ['copy_results']
   ansible.builtin.fetch:
     src: "{{ item.path }}"
-    dest: "{{ ltp_results_dir }}/{{ ltp_test_group }}/"
+    dest: "{{ ltp_results_full_path }}/last-run/{{ last_kernel }}/{{ ltp_test_group }}/"
     flat: true
     validate_checksum: false
   with_items: "{{ output_files.files }}"
@@ -224,3 +259,20 @@
     - output_files.matched > 0
   loop_control:
     label: "Copying {{ item.path | basename }} ..."
+
+- name: Verify last-run kernel directory exists
+  tags: [ 'copy_results' ]
+  delegate_to: localhost
+  stat:
+    path: "{{ ltp_results_full_path }}/last-run/{{ last_kernel }}"
+  register: last_run_kernel_dir
+
+- name: Copy over the last-run into the results directory for archiving into kdevops
+  copy:
+    src: "{{ ltp_results_full_path }}/last-run/{{ last_kernel }}"
+    dest: "{{ ltp_results_full_path }}/"
+  run_once: true
+  tags: [ 'copy_results' ]
+  delegate_to: localhost
+  when:
+    - last_run_kernel_dir.stat.exists
-- 
2.48.1

[PATCH 5/9] fstests: add makefile target to show test results

[Scott Mayhew]

Add 'fstests-show-results' makefile target to show test results.

Under the hood, it more or less just does 'find ... | xargs cat'.

By default, the result.xml files will be shown for the most recent
kernel run.  You can show the results for a different kernel by
overriding the LAST_KERNEL variable, e.g.
$ LAST_KERNEL=6.13.4-300.fc41.x86_64 make fstests-show-results

You can change the files being shown by overriding the PATTERN variable.
For example, to just see the summary:
$ PATTERN="\( -name xunit_results.txt \)" make fstests-show-results

or to see the summary and the bad results:
$ PATTERN="\( -name xunit_results.txt -o -name \"*.bad\" \)" make fstests-show-results

or you can do any combination thereof, e.g.
$ LAST_KERNEL=6.13.4-300.fc41.x86_64 PATTERN="\( -name xunit_results.txt \)" make fstests-show-results

Signed-off-by: Scott Mayhew <smayhew@redhat.com>
---
 workflows/fstests/Makefile | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/workflows/fstests/Makefile b/workflows/fstests/Makefile
index 344a1f8..ef9c0fa 100644
--- a/workflows/fstests/Makefile
+++ b/workflows/fstests/Makefile
@@ -115,6 +115,14 @@ ifneq (,$(COUNT))
 FSTESTS_DYNAMIC_RUNTIME_VARS += , "oscheck_extra_args": "-I $(COUNT)"
 endif
 
+ifndef LAST_KERNEL
+LAST_KERNEL := $(shell cat workflows/fstests/results/last-kernel.txt 2>/dev/null)
+endif
+
+ifndef PATTERN
+PATTERN := -name result.xml
+endif
+
 fstests: $(FSTESTS_BASELINE_EXTRA)
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) -l localhost,baseline,dev \
 		-f 30 -i hosts playbooks/fstests.yml --skip-tags run_tests,copy_results $(LIMIT_HOSTS)
@@ -218,6 +226,14 @@ fstests-dev-results: $(KDEVOPS_EXTRA_VARS)
 		--extra-vars=@./extra_vars.yaml \
 		$(LIMIT_HOSTS)
 
+fstests-show-results:
+ifdef LAST_KERNEL
+	@find workflows/fstests/results/$(LAST_KERNEL) -type f $(PATTERN) \
+		| xargs -I {} bash -c 'echo "{}:"; cat {}; echo;'
+else
+	@echo "No results."
+endif
+
 fstests-help-menu:
 	@echo "fstests options:"
 	@echo "fstests                                 - Git clones fstests, builds and install it"
-- 
2.48.1

[PATCH 6/9] pynfs: add makefile target to show test results

[Scott Mayhew]

Add 'pynfs-show-results' makefile target to show test results.

Under the hood, it more or less just does 'find ... | xargs cat'.

It shows the json results files for the most recent kernel run.  You can
show the results for a differnt kernel by overriding the LAST_KERNEL
variable, e.g.

$ LAST_KERNEL=6.13.4-300.fc41.x86_64 make pynfs-show-results

Signed-off-by: Scott Mayhew <smayhew@redhat.com>
---
 workflows/pynfs/Makefile | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/workflows/pynfs/Makefile b/workflows/pynfs/Makefile
index 75a4b6a..253a222 100644
--- a/workflows/pynfs/Makefile
+++ b/workflows/pynfs/Makefile
@@ -24,6 +24,10 @@ PYNFS_KERNEL_CI_LOOP      := scripts/workflows/pynfs/run_kernel_ci.sh
 PYNFS_KERNEL_CI_LOOP_KOTD := scripts/workflows/pynfs/run_kernel_ci_kotd.sh
 endif # CONFIG_KERNEL_CI
 
+ifndef LAST_KERNEL
+LAST_KERNEL := $(shell cat workflows/pynfs/results/last-kernel.txt 2>/dev/null)
+endif
+
 # Makefile for pynfs targets
 pynfs:
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) -l baseline,dev \
@@ -63,6 +67,14 @@ pynfs-dev-reset:
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
 		-f 30 -i hosts -l dev playbooks/pynfs.yml --tags vars,reset --extra-vars=@./extra_vars.yaml
 
+pynfs-show-results:
+ifdef LAST_KERNEL
+	@find workflows/pynfs/results/$(LAST_KERNEL) -type f \
+		| xargs -I {} bash -c 'echo "{}:"; cat {}; echo -e "\n";'
+else
+	@echo "No results."
+endif
+
 pynfs-help-menu:
 	@echo "pynfs options:"
 	@echo "pynfs                             - Git clone pynfs, build and install it"
-- 
2.48.1

[PATCH 7/9] nfstest: add makefile target to show test results

[Scott Mayhew]

Under the hood, it more or less just does 'find ... | xargs cat'.

It shows the text results files for the most recent kernel run.  You can
show the results for a different kernel by overriding the LAST_KERNEL
variable, e.g.

$ LAST_KERNEL=6.13.4-300.fc41.x86_64 make nfstest-show-results

Note that since the nfstest log files have a date and timestamp in the
filename, it's possible to accumulate logs from multiple test runs.  If
you're showing the results from the most recent kernel, then we'll only
show the logs from the most recent run (i.e. logs that are newer than
the timestamp of the last-run directory). But if you're showing the
results from an older kernel, then we'll show all the logs in that
directory (but you can change that by overriding the PATTERN variable).

Signed-off-by: Scott Mayhew <smayhew@redhat.com>
---
 workflows/nfstest/Makefile | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/workflows/nfstest/Makefile b/workflows/nfstest/Makefile
index 638ec1f..2e645ce 100644
--- a/workflows/nfstest/Makefile
+++ b/workflows/nfstest/Makefile
@@ -53,6 +53,10 @@ NFSTEST_KERNEL_CI_LOOP      := scripts/workflows/nfstest/run_kernel_ci.sh
 NFSTEST_KERNEL_CI_LOOP_KOTD := scripts/workflows/nfstest/run_kernel_ci_kotd.sh
 endif # CONFIG_KERNEL_CI
 
+ifndef LAST_KERNEL
+LAST_KERNEL := $(shell cat workflows/nfstest/results/last-kernel.txt 2>/dev/null)
+endif
+
 nfstest:
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) -l baseline,dev \
 		-f 30 -i hosts playbooks/nfstest.yml \
@@ -88,6 +92,19 @@ nfstest-dev-reset:
 		--tags vars,reset \
 		--extra-vars=@./extra_vars.yaml
 
+nfstest-show-results:
+ifdef LAST_KERNEL
+ifeq ($(LAST_KERNEL), $(shell cat workflows/nfstest/results/last-kernel.txt 2>/dev/null))
+	@find workflows/nfstest/results/$(LAST_KERNEL) -type f $(PATTERN) -newer workflows/nfstest/results/last-run \
+		| xargs -I {} bash -c 'echo "{}:"; cat {}; echo;'
+else
+	@find workflows/nfstest/results/$(LAST_KERNEL) -type f $(PATTERN) \
+		| xargs -I {} bash -c 'echo "{}:"; cat {}; echo;'
+endif
+else
+	@echo "No results."
+endif
+
 nfstest-help-menu:
 	@echo "nfstest options:"
 	@echo "nfstest                              - Git clone nfstest and install it"
-- 
2.48.1

[PATCH 8/9] gitr: add makefile target to show test results

[Scott Mayhew]

Under the hood, it more or less just does 'find ... | xargs cat'.

By default, the summary files will be shown for the most recent kernel
run.  You can show the results for a different kernel by overriding the
LAST_KERNEL variable, e.g.

$ LAST_KERNEL=6.13.4-100.fc40.x86_64 make gitr-show-results

You can change the files being shown by overriding the PATTERN variable.
For example, to just see the rpc and xprt statisics:

$ PATTERN="\( -name \"*.rpc\" -o -name \"*.xprt\" \)" make gitr-show-results

Note that since the gitr log files have a date and timestamp in the
filename, it's possible to accumulate logs from multiple test runs.  If
you're showing the results from the most recent kernel, then we'll only
show the logs from the most recent run (i.e. logs that are newer than
the timestamp of the last-run directory). But if you're showing the
results from an older kernel, then we'll show all the logs in that
directory (but you can override that too if you get creative with the
PATTERN variable).

Signed-off-by: Scott Mayhew <smayhew@redhat.com>
---
 workflows/gitr/Makefile | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/workflows/gitr/Makefile b/workflows/gitr/Makefile
index 5432f42..8d64767 100644
--- a/workflows/gitr/Makefile
+++ b/workflows/gitr/Makefile
@@ -70,6 +70,14 @@ GITR_KERNEL_CI_LOOP      := scripts/workflows/gitr/run_kernel_ci.sh
 GITR_KERNEL_CI_LOOP_KOTD := scripts/workflows/gitr/run_kernel_ci_kotd.sh
 endif # CONFIG_KERNEL_CI
 
+ifndef LAST_KERNEL
+LAST_KERNEL := $(shell cat workflows/gitr/results/last-kernel.txt 2>/dev/null)
+endif
+
+ifndef PATTERN
+PATTERN := -name "*.summary"
+endif
+
 gitr:
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) -l baseline,dev \
 		-f 30 -i hosts playbooks/gitr.yml \
@@ -103,6 +111,19 @@ gitr-dev-reset:
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
 		-f 30 -i hosts -l dev playbooks/gitr.yml --tags vars,reset --extra-vars=@./extra_vars.yaml
 
+gitr-show-results:
+ifdef LAST_KERNEL
+ifeq ($(LAST_KERNEL), $(shell cat workflows/gitr/results/last-kernel.txt 2>/dev/null))
+	@find workflows/gitr/results/$(LAST_KERNEL) -type f $(PATTERN) -newer workflows/gitr/results/last-run \
+		| xargs -I {} bash -c 'echo "{}:"; cat {}; echo;'
+else
+	@find workflows/gitr/results/$(LAST_KERNEL) -type f $(PATTERN) \
+		| xargs -I {} bash -c 'echo "{}:"; cat {}; echo;'
+endif
+else
+	@echo "No results."
+endif
+
 gitr-help-menu:
 	@echo "gitr options:"
 	@echo "gitr                              - Git clone git, build and install it"
-- 
2.48.1

[PATCH 9/9] ltp: add makefile target to show test results

[Scott Mayhew]

Under the hood, it more or less just does 'find ... | xargs cat'.

By default, the "*.failed" files will be shown for the most recent kernel
run.  You can show the results for a different kernel by overriding the
LAST_KERNEL variable, e.g.

$ LAST_KERNEL=6.11.9-100.fc39.x86_64 make ltp-show-results

You can change the files being shown by overriding the PATTERN variable.
For example, to just see the full test logs:

$ PATTERN="-name \"*.log\"" make ltp-show-results

Note that since the gitr log files have a date and timestamp in the
filename, it's possible to accumulate logs from multiple test runs.  If
you're showing the results from the most recent kernel, then we'll only
show the logs from the most recent run (i.e. logs that are newer than
the timestamp of the last-run directory). But if you're showing the
results from an older kernel, then we'll show all the logs in that
directory (but you can override that too if you get creative with the
PATTERN variable).

Signed-off-by: Scott Mayhew <smayhew@redhat.com>
---
 workflows/ltp/Makefile | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/workflows/ltp/Makefile b/workflows/ltp/Makefile
index 4e439da..8de76a0 100644
--- a/workflows/ltp/Makefile
+++ b/workflows/ltp/Makefile
@@ -96,6 +96,14 @@ LTP_KERNEL_CI_LOOP      := scripts/workflows/ltp/run_kernel_ci.sh
 LTP_KERNEL_CI_LOOP_KOTD := scripts/workflows/ltp/run_kernel_ci_kotd.sh
 endif # CONFIG_KERNEL_CI
 
+ifndef LAST_KERNEL
+LAST_KERNEL := $(shell cat workflows/ltp/results/last-kernel.txt 2>/dev/null)
+endif
+
+ifndef PATTERN
+PATTERN := -name "*.failed"
+endif
+
 ltp:
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) -l baseline,dev \
 		-f 30 -i hosts playbooks/ltp.yml --skip-tags run_tests,copy_results
@@ -128,6 +136,19 @@ ltp-dev-reset:
 		-f 30 -i hosts -l dev playbooks/ltp.yml \
 		--tags vars,reset --extra-vars=@./extra_vars.yaml
 
+ltp-show-results:
+ifdef LAST_KERNEL
+ifeq ($(LAST_KERNEL), $(shell cat workflows/ltp/results/last-kernel.txt 2>/dev/null))
+	find workflows/ltp/results/$(LAST_KERNEL) -type f $(PATTERN) -newer workflows/ltp/results/last-run \
+		| xargs -I {} bash -c 'echo "{}:"; cat {}; echo;'
+else
+	@find workflows/ltp/results/$(LAST_KERNEL) -type f $(PATTERN) \
+		| xargs -I {} bash -c 'echo "{}:"; cat {}; echo;'
+endif
+else
+	@echo "No results."
+endif
+
 ltp-help-menu:
 	@echo "ltp options:"
 	@echo "ltp                              - Git clone ltp, build and install it"
-- 
2.48.1

[PATCH] Don't include the codereadyrepo role if we're using a custom yum repofile

[Scott Mayhew]

If we're using a custom yum repofile, there's no guarantee the CRB repo
follows the expected naming convention or that a CRB repo even exists
for our setup.

Signed-off-by: Scott Mayhew <smayhew@redhat.com>
---
 playbooks/roles/codereadyrepo/tasks/main.yml                | 5 -----
 playbooks/roles/fstests/tasks/install-deps/redhat/main.yml  | 1 +
 playbooks/roles/ltp/tasks/main.yml                          | 1 +
 playbooks/roles/pynfs/tasks/install-deps/redhat/main.yml    | 3 ++-
 playbooks/roles/sysbench/tasks/install-deps/redhat/main.yml | 1 +
 5 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/playbooks/roles/codereadyrepo/tasks/main.yml b/playbooks/roles/codereadyrepo/tasks/main.yml
index 1b8bad0..45e6f7a 100644
--- a/playbooks/roles/codereadyrepo/tasks/main.yml
+++ b/playbooks/roles/codereadyrepo/tasks/main.yml
@@ -4,7 +4,6 @@
     codeready_repo: "codeready-builder-for-rhel-{{ ansible_distribution_major_version }}-{{ ansible_architecture }}-rpms"
   when:
     - ansible_distribution == 'RedHat'
-    - not devconfig_custom_yum_repofile
     - kdevops_enable_guestfs
 
 - name: Select the Oracle Linux CodeReady Builder repo
@@ -12,14 +11,12 @@
     codeready_repo: "ol9_codeready_builder"
   when:
     - ansible_distribution == 'OracleLinux'
-    - not devconfig_custom_yum_repofile
 
 - name: Select the CentOS CodeReady Builder repo
   ansible.builtin.set_fact:
     codeready_repo: "crb"
   when:
     - ansible_distribution == 'CentOS'
-    - not devconfig_custom_yum_repofile
     - kdevops_enable_guestfs
 
 - name: Select the AWS RHEL CodeReady Builder repo
@@ -27,7 +24,6 @@
     codeready_repo: "codeready-builder-for-rhel-{{ ansible_distribution_major_version }}-rhui-rpms"
   when:
     - ansible_distribution == 'RedHat'
-    - not devconfig_custom_yum_repofile
     - kdevops_enable_terraform
     - kdevops_terraform_provider == "aws"
 
@@ -36,7 +32,6 @@
     codeready_repo: "codeready-builder-for-rhel-{{ ansible_distribution_major_version }}-{{ ansible_architecture }}-rhui-rpms"
   when:
     - ansible_distribution == 'RedHat'
-    - not devconfig_custom_yum_repofile
     - kdevops_enable_terraform
     - kdevops_terraform_provider == "azure"
 
diff --git a/playbooks/roles/fstests/tasks/install-deps/redhat/main.yml b/playbooks/roles/fstests/tasks/install-deps/redhat/main.yml
index 52b4bce..c8df399 100644
--- a/playbooks/roles/fstests/tasks/install-deps/redhat/main.yml
+++ b/playbooks/roles/fstests/tasks/install-deps/redhat/main.yml
@@ -4,6 +4,7 @@
     name: codereadyrepo
   when:
     - ansible_distribution != "Fedora"
+    - not devconfig_custom_yum_repofile
 
 - name: Enable installation of packages from EPEL
   ansible.builtin.include_role:
diff --git a/playbooks/roles/ltp/tasks/main.yml b/playbooks/roles/ltp/tasks/main.yml
index b15f87d..f6cf97a 100644
--- a/playbooks/roles/ltp/tasks/main.yml
+++ b/playbooks/roles/ltp/tasks/main.yml
@@ -48,6 +48,7 @@
   when:
     - ansible_os_family == "RedHat"
     - ansible_distribution != "Fedora"
+    - not devconfig_custom_yum_repofile
 
 - name: Install dependencies for ltp
   tags: ['build', 'ltp']
diff --git a/playbooks/roles/pynfs/tasks/install-deps/redhat/main.yml b/playbooks/roles/pynfs/tasks/install-deps/redhat/main.yml
index b98ff67..2375e93 100644
--- a/playbooks/roles/pynfs/tasks/install-deps/redhat/main.yml
+++ b/playbooks/roles/pynfs/tasks/install-deps/redhat/main.yml
@@ -3,7 +3,8 @@
   ansible.builtin.include_role:
     name: codereadyrepo
   when:
-    ansible_distribution != "Fedora"
+    - ansible_distribution != "Fedora"
+    - not devconfig_custom_yum_repofile
 
 - name: Install build dependencies for pynfs
   become: yes
diff --git a/playbooks/roles/sysbench/tasks/install-deps/redhat/main.yml b/playbooks/roles/sysbench/tasks/install-deps/redhat/main.yml
index d5021e0..47832b2 100644
--- a/playbooks/roles/sysbench/tasks/install-deps/redhat/main.yml
+++ b/playbooks/roles/sysbench/tasks/install-deps/redhat/main.yml
@@ -4,6 +4,7 @@
     name: codereadyrepo
   when:
     - ansible_distribution != "Fedora"
+    - not devconfig_custom_yum_repofile
 
 - name: Enable installation of packages from EPEL
   ansible.builtin.include_role:
-- 
2.48.1

Re: [PATCH 0/9] tweak results organization and reporting

[Scott Mayhew]

Whoops, I accidentally re-sent this series while I was sending the
"Don't include the codereadyrepo role if we're using a custom yum repofile"
patch.

Sorry for the noise.

-Scott

On Tue, 04 Mar 2025, Scott Mayhew wrote:

> I do most of my kdevops runs under buildbot.  Currently I don't have an
> easy way to view test results in buildbot's web interface. 
> 
> The first four patches implement a "last-run" directory and
> "last-kernel.txt" file similar to what the fstests workflow already has.
> This is mainly to facilitate the remaining patches, but I think it will
> also make it easier for me to feed my test results into other tooling in
> the future.
> 
> The remaining patches add a "<workflow>-show-results" makefile target,
> which basically print the test results to the standard output.  The
> results for the most recent run are printed by default, but that can be
> changed by overriding the LAST_KERNEL variable.  The results printed
> varies from workflow to workflow, and can be overidden via the PATTERN
> variable.
> 
> Scott Mayhew (9):
>   pynfs: add last-run directory and last-kernel.txt to the results
>   nfstest: add last-run directory and last-kernel.txt to the results
>   gitr: add last-run directory and last-kernel.txt to the results
>   ltp: add last-run directory and last-kernel.txt to the results
>   fstests: add makefile target to show test results
>   pynfs: add makefile target to show test results
>   nfstest: add makefile target to show test results
>   gitr: add makefile target to show test results
>   ltp: add makefile target to show test results
> 
>  playbooks/roles/gitr/tasks/main.yml    | 59 +++++++++++++++++++++----
>  playbooks/roles/ltp/tasks/main.yml     | 60 ++++++++++++++++++++++++--
>  playbooks/roles/nfstest/tasks/main.yml | 60 ++++++++++++++++++++++----
>  playbooks/roles/pynfs/tasks/main.yml   | 38 +++++++++++++++-
>  workflows/fstests/Makefile             | 16 +++++++
>  workflows/gitr/Makefile                | 21 +++++++++
>  workflows/ltp/Makefile                 | 21 +++++++++
>  workflows/nfstest/Makefile             | 17 ++++++++
>  workflows/pynfs/Makefile               | 12 ++++++
>  9 files changed, 282 insertions(+), 22 deletions(-)
> 
> -- 
> 2.48.1
> 
> 

Re: [PATCH 0/9] tweak results organization and reporting

[Chuck Lever]

On 3/3/25 8:09 AM, Scott Mayhew wrote:
> I do most of my kdevops runs under buildbot.  Currently I don't have an
> easy way to view test results in buildbot's web interface. 
> 
> The first four patches implement a "last-run" directory and
> "last-kernel.txt" file similar to what the fstests workflow already has.
> This is mainly to facilitate the remaining patches, but I think it will
> also make it easier for me to feed my test results into other tooling in
> the future.
> 
> The remaining patches add a "<workflow>-show-results" makefile target,
> which basically print the test results to the standard output.  The
> results for the most recent run are printed by default, but that can be
> changed by overriding the LAST_KERNEL variable.  The results printed
> varies from workflow to workflow, and can be overidden via the PATTERN
> variable.
> 
> Scott Mayhew (9):
>   pynfs: add last-run directory and last-kernel.txt to the results
>   nfstest: add last-run directory and last-kernel.txt to the results
>   gitr: add last-run directory and last-kernel.txt to the results
>   ltp: add last-run directory and last-kernel.txt to the results
>   fstests: add makefile target to show test results
>   pynfs: add makefile target to show test results
>   nfstest: add makefile target to show test results
>   gitr: add makefile target to show test results
>   ltp: add makefile target to show test results
> 
>  playbooks/roles/gitr/tasks/main.yml    | 59 +++++++++++++++++++++----
>  playbooks/roles/ltp/tasks/main.yml     | 60 ++++++++++++++++++++++++--
>  playbooks/roles/nfstest/tasks/main.yml | 60 ++++++++++++++++++++++----
>  playbooks/roles/pynfs/tasks/main.yml   | 38 +++++++++++++++-
>  workflows/fstests/Makefile             | 16 +++++++
>  workflows/gitr/Makefile                | 21 +++++++++
>  workflows/ltp/Makefile                 | 21 +++++++++
>  workflows/nfstest/Makefile             | 17 ++++++++
>  workflows/pynfs/Makefile               | 12 ++++++
>  9 files changed, 282 insertions(+), 22 deletions(-)
> 

Thanks for tackling this.

Adding a "last-run" directory for the newer workflows makes sense to me.

I think we can refine the report format over time, if needed (I haven't
looked to closely at those). The main ingredients there are brevity and
human-readability. IMO, displaying XML/JSON/YAML in those reports is
probably the opposite of brief and readable. ;-) Since the xxML format
reports are already available, I don't think we need to be concerned
about "backwards compatibility" for the human-consumed reports, so it
shouldn't be forbidden to alter the reports fairly freely.


-- 
Chuck Lever

Re: [PATCH 0/9] tweak results organization and reporting

[Luis Chamberlain]

On Wed, Mar 12, 2025 at 04:09:02PM -0400, Chuck Lever wrote:
> On 3/3/25 8:09 AM, Scott Mayhew wrote:
> > I do most of my kdevops runs under buildbot.  Currently I don't have an
> > easy way to view test results in buildbot's web interface. 
> > 
> > The first four patches implement a "last-run" directory and
> > "last-kernel.txt" file similar to what the fstests workflow already has.
> > This is mainly to facilitate the remaining patches, but I think it will
> > also make it easier for me to feed my test results into other tooling in
> > the future.
> > 
> > The remaining patches add a "<workflow>-show-results" makefile target,
> > which basically print the test results to the standard output.  The
> > results for the most recent run are printed by default, but that can be
> > changed by overriding the LAST_KERNEL variable.  The results printed
> > varies from workflow to workflow, and can be overidden via the PATTERN
> > variable.
> > 
> > Scott Mayhew (9):
> >   pynfs: add last-run directory and last-kernel.txt to the results
> >   nfstest: add last-run directory and last-kernel.txt to the results
> >   gitr: add last-run directory and last-kernel.txt to the results
> >   ltp: add last-run directory and last-kernel.txt to the results
> >   fstests: add makefile target to show test results
> >   pynfs: add makefile target to show test results
> >   nfstest: add makefile target to show test results
> >   gitr: add makefile target to show test results
> >   ltp: add makefile target to show test results
> > 
> >  playbooks/roles/gitr/tasks/main.yml    | 59 +++++++++++++++++++++----
> >  playbooks/roles/ltp/tasks/main.yml     | 60 ++++++++++++++++++++++++--
> >  playbooks/roles/nfstest/tasks/main.yml | 60 ++++++++++++++++++++++----
> >  playbooks/roles/pynfs/tasks/main.yml   | 38 +++++++++++++++-
> >  workflows/fstests/Makefile             | 16 +++++++
> >  workflows/gitr/Makefile                | 21 +++++++++
> >  workflows/ltp/Makefile                 | 21 +++++++++
> >  workflows/nfstest/Makefile             | 17 ++++++++
> >  workflows/pynfs/Makefile               | 12 ++++++
> >  9 files changed, 282 insertions(+), 22 deletions(-)
> > 
> 
> Thanks for tackling this.
> 
> Adding a "last-run" directory for the newer workflows makes sense to me.
> 
> I think we can refine the report format over time, if needed (I haven't
> looked to closely at those). The main ingredients there are brevity and
> human-readability. IMO, displaying XML/JSON/YAML in those reports is
> probably the opposite of brief and readable. ;-) Since the xxML format
> reports are already available, I don't think we need to be concerned
> about "backwards compatibility" for the human-consumed reports, so it
> shouldn't be forbidden to alter the reports fairly freely.

Then ... wait for it... later in the future we just parse all this to
kidcb and boom, kernel-ci integration.

https://docs.kernelci.org/kcidb/submitter_guide/

  Luis