[PATCH 1/2] devconfig/redhat: Fix up a few issues with the install-deps playbook

[PATCH 1/2] devconfig/redhat: Fix up a few issues with the install-deps playbook

[Scott Mayhew]

1. Replace the "Install btrfs-progs" task with a "Add btrfs-progs to
   install package list" task.  Due to a collision with the "packages"
   variable, the "Install btrfs-progs" task was actually installing
   everything on the previously declared list, regardless of whether
   devconfig_try_install_kdevtools was enabled or not.

2. Add a when clause so that the following tasks only run when
   devconfig_try_install_kdevtools is enabled (no point in messing with
   the packages list if we're not going to ultimately run the install
   task):
        * Build install package list
        * Add btrfs-progs to install package list
        * Add GNU screen to install package list
        * Add Tmux to install package list

3. Remove the "Add systemd-journal-remote to install package list" task.
   It's redundant because there's a task later in the playbook that
   will install systemd-journal-remote when
   devconfig_enable_systemd_journal_remote is enabled.

Signed-off-by: Scott Mayhew <smayhew@redhat.com>
---
 .../tasks/install-deps/redhat/main.yml        | 34 ++++++-------------
 1 file changed, 11 insertions(+), 23 deletions(-)

diff --git a/playbooks/roles/devconfig/tasks/install-deps/redhat/main.yml b/playbooks/roles/devconfig/tasks/install-deps/redhat/main.yml
index 5a0f1e2..7078443 100644
--- a/playbooks/roles/devconfig/tasks/install-deps/redhat/main.yml
+++ b/playbooks/roles/devconfig/tasks/install-deps/redhat/main.yml
@@ -111,41 +111,29 @@
       - net-snmp
       - trace-cmd
       - perf
+  when:
+    - devconfig_try_install_kdevtools|bool
 
-- name: Install btrfs-progs
-  become: yes
-  become_method: sudo
-  yum:
-    update_cache: yes
-    name: "{{ packages }}"
-  retries: 3
-  delay: 5
-  register: result
-  until: result.rc == 0
-  vars:
-    packages:
-      - btrfs-progs
-  when: ansible_distribution == 'Fedora'
+- name: Add btrfs-progs to install package list
+  set_fact:
+    packages: "{{ packages + [ 'btrfs-progs' ] }}"
+  when:
+    - devconfig_try_install_kdevtools|bool
+    - ansible_distribution == 'Fedora'
 
 - name: Add GNU screen to install package list
   set_fact:
     packages: "{{ packages + [ 'screen' ] }}"
   when:
-    ansible_facts['os_family']|lower != 'redhat' or ansible_facts['distribution_major_version'] | int < 8
+    - devconfig_try_install_kdevtools|bool
+    - ansible_facts['os_family']|lower != 'redhat' or ansible_facts['distribution_major_version'] | int < 8
 
 - name: Add Tmux to install package list
   set_fact:
     packages: "{{ packages + [ 'tmux' ] }}"
   when:
-    ansible_facts['os_family']|lower == 'redhat' or ansible_facts['distribution_major_version'] | int >= 8
-
-- name: Add systemd-journal-remote to install package list
-  set_fact:
-    packages: "{{ packages + [ 'systemd-journal-remote' ] }}"
-  when:
-    - devconfig_enable_systemd_journal_remote|bool
+    - devconfig_try_install_kdevtools|bool
     - ansible_facts['os_family']|lower == 'redhat' or ansible_facts['distribution_major_version'] | int >= 8
-  tags: journal-upload
 
 - name: Install packages we typically care about
   become: yes
-- 
2.44.0

[PATCH 2/2] gitr: Make gitr work on RHEL/CentOS

[Scott Mayhew]

RHEL and CentOS do not ship cvsps or perl-TAP-Harness-Archive.  For the
former, we'll have to build it from git.  For the latter, we can install
it from CPAN, but we'll also have to install the perl-App-cpanminus
package to do so.

Also add 'make' and 'tar' to the dependencies list.  These were
previously getting installed inadvertently by the "Install btrfs-progs"
task in playbooks/roles/devconfig/tasks/install-deps/redhat/main.yml,
which was installing more than just btrfs-progs.

Signed-off-by: Scott Mayhew <smayhew@redhat.com>
---
 playbooks/roles/gitr/defaults/main.yml |  3 ++
 playbooks/roles/gitr/tasks/main.yml    | 62 ++++++++++++++++++++++++++
 playbooks/roles/gitr/vars/RedHat.yml   |  7 ++-
 3 files changed, 70 insertions(+), 2 deletions(-)

diff --git a/playbooks/roles/gitr/defaults/main.yml b/playbooks/roles/gitr/defaults/main.yml
index bcfbaef..ff737ca 100644
--- a/playbooks/roles/gitr/defaults/main.yml
+++ b/playbooks/roles/gitr/defaults/main.yml
@@ -18,3 +18,6 @@ gitr_thread_single: false
 gitr_thread_fast: false
 gitr_thread_stress: false
 gitr_thread_custom: false
+
+cvsps_git: https://github.com/andreyvit/cvsps.git
+cvsps_data: "{{ data_path }}/cvsps"
diff --git a/playbooks/roles/gitr/tasks/main.yml b/playbooks/roles/gitr/tasks/main.yml
index 330350c..2eafeaa 100644
--- a/playbooks/roles/gitr/tasks/main.yml
+++ b/playbooks/roles/gitr/tasks/main.yml
@@ -11,6 +11,11 @@
   failed_when: false
   tags: vars
 
+- name: Set up the /data mount point
+  tags: ['data_partition']
+  ansible.builtin.include_role:
+    name: create_data_partition
+
 - name: Set the name of the test group
   tags: vars
   ansible.builtin.set_fact:
@@ -42,6 +47,20 @@
       paths:
         - 'vars'
 
+- name: Update gitr depencies for RHEL/Centos
+  tags: gitr
+  ansible.builtin.set_fact:
+    gitr_packages: "{{ gitr_packages + ['perl-App-cpanminus'] }}"
+  when:
+    - ansible_distribution != "Fedora"
+
+- name: Update gitr dependencies for Fedora
+  tags: gitr
+  ansible.builtin.set_fact:
+    gitr_packages: "{{ gitr_packages + ['cvsps', 'perl-TAP-Harness-Archive'] }}"
+  when:
+    - ansible_distribution == "Fedora"
+
 - name: Install dependencies for gitr
   tags: gitr
   become: true
@@ -51,6 +70,49 @@
     name: "{{ gitr_packages }}"
     state: present
 
+- name: Install CPAN modules for gitr
+  tags: gitr
+  become: true
+  become_flags: 'su - -c'
+  become_method: ansible.builtin.sudo
+  community.general.cpanm:
+    name: "{{ item }}"
+  with_items: "{{ gitr_cpan_modules }}"
+  when:
+    - ansible_distribution != "Fedora"
+
+- name: git clone cvsps
+  tags: gitr
+  ansible.builtin.git:
+    repo: "{{ cvsps_git }}"
+    dest: "{{ cvsps_data }}"
+    update: true
+  retries: 3
+  delay: 5
+  register: result
+  until: not result.failed
+  when:
+    - ansible_distribution != "Fedora"
+
+- name: Build cvsps
+  tags: gitr
+  community.general.make:
+    chdir: "{{ cvsps_data }}"
+    jobs: "{{ ansible_processor_nproc }}"
+  when:
+    - ansible_distribution != "Fedora"
+
+- name: Install cvsps
+  tags: gitr
+  become: true
+  become_flags: 'su - -c'
+  become_method: ansible.builtin.sudo
+  community.general.make:
+    target: install
+    chdir: "{{ cvsps_data }}"
+  when:
+    - ansible_distribution != "Fedora"
+
 - name: Set up an iSCSI initiator on the target node
   ansible.builtin.include_role:
     name: iscsi
diff --git a/playbooks/roles/gitr/vars/RedHat.yml b/playbooks/roles/gitr/vars/RedHat.yml
index 57f2f35..caf9840 100644
--- a/playbooks/roles/gitr/vars/RedHat.yml
+++ b/playbooks/roles/gitr/vars/RedHat.yml
@@ -1,7 +1,6 @@
 ---
 gitr_packages:
   - cvs
-  - cvsps
   - expat-devel
   - gcc
   - gettext
@@ -9,11 +8,15 @@ gitr_packages:
   - git-email
   - glibc-utils
   - libcurl-devel
+  - make
   - openssl-devel
   - perl-Memoize
-  - perl-TAP-Harness-Archive
   - perl-Test-Harness
   - perl-Test-Simple
   - subversion
   - subversion-perl
+  - tar
   - zlib-devel
+
+gitr_cpan_modules:
+  - TAP::Harness::Archive
-- 
2.44.0

Re: [PATCH 2/2] gitr: Make gitr work on RHEL/CentOS

[Chuck Lever]

On Tue, Sep 03, 2024 at 10:48:47AM -0400, Scott Mayhew wrote:
> RHEL and CentOS do not ship cvsps or perl-TAP-Harness-Archive.  For the
> former, we'll have to build it from git.  For the latter, we can install
> it from CPAN, but we'll also have to install the perl-App-cpanminus
> package to do so.
> 
> Also add 'make' and 'tar' to the dependencies list.  These were
> previously getting installed inadvertently by the "Install btrfs-progs"
> task in playbooks/roles/devconfig/tasks/install-deps/redhat/main.yml,
> which was installing more than just btrfs-progs.
> 
> Signed-off-by: Scott Mayhew <smayhew@redhat.com>
> ---
>  playbooks/roles/gitr/defaults/main.yml |  3 ++
>  playbooks/roles/gitr/tasks/main.yml    | 62 ++++++++++++++++++++++++++
>  playbooks/roles/gitr/vars/RedHat.yml   |  7 ++-
>  3 files changed, 70 insertions(+), 2 deletions(-)
> 
> diff --git a/playbooks/roles/gitr/defaults/main.yml b/playbooks/roles/gitr/defaults/main.yml
> index bcfbaef..ff737ca 100644
> --- a/playbooks/roles/gitr/defaults/main.yml
> +++ b/playbooks/roles/gitr/defaults/main.yml
> @@ -18,3 +18,6 @@ gitr_thread_single: false
>  gitr_thread_fast: false
>  gitr_thread_stress: false
>  gitr_thread_custom: false
> +
> +cvsps_git: https://github.com/andreyvit/cvsps.git
> +cvsps_data: "{{ data_path }}/cvsps"
> diff --git a/playbooks/roles/gitr/tasks/main.yml b/playbooks/roles/gitr/tasks/main.yml
> index 330350c..2eafeaa 100644
> --- a/playbooks/roles/gitr/tasks/main.yml
> +++ b/playbooks/roles/gitr/tasks/main.yml
> @@ -11,6 +11,11 @@
>    failed_when: false
>    tags: vars
>  
> +- name: Set up the /data mount point
> +  tags: ['data_partition']
> +  ansible.builtin.include_role:
> +    name: create_data_partition
> +
>  - name: Set the name of the test group
>    tags: vars
>    ansible.builtin.set_fact:
> @@ -42,6 +47,20 @@
>        paths:
>          - 'vars'
>  
> +- name: Update gitr depencies for RHEL/Centos
> +  tags: gitr
> +  ansible.builtin.set_fact:
> +    gitr_packages: "{{ gitr_packages + ['perl-App-cpanminus'] }}"
> +  when:
> +    - ansible_distribution != "Fedora"
> +
> +- name: Update gitr dependencies for Fedora
> +  tags: gitr
> +  ansible.builtin.set_fact:
> +    gitr_packages: "{{ gitr_packages + ['cvsps', 'perl-TAP-Harness-Archive'] }}"
> +  when:
> +    - ansible_distribution == "Fedora"
> +
>  - name: Install dependencies for gitr
>    tags: gitr
>    become: true
> @@ -51,6 +70,49 @@
>      name: "{{ gitr_packages }}"
>      state: present
>  
> +- name: Install CPAN modules for gitr
> +  tags: gitr
> +  become: true
> +  become_flags: 'su - -c'
> +  become_method: ansible.builtin.sudo
> +  community.general.cpanm:
> +    name: "{{ item }}"
> +  with_items: "{{ gitr_cpan_modules }}"
> +  when:
> +    - ansible_distribution != "Fedora"
> +
> +- name: git clone cvsps
> +  tags: gitr
> +  ansible.builtin.git:
> +    repo: "{{ cvsps_git }}"
> +    dest: "{{ cvsps_data }}"
> +    update: true
> +  retries: 3
> +  delay: 5
> +  register: result
> +  until: not result.failed
> +  when:
> +    - ansible_distribution != "Fedora"
> +
> +- name: Build cvsps
> +  tags: gitr
> +  community.general.make:
> +    chdir: "{{ cvsps_data }}"
> +    jobs: "{{ ansible_processor_nproc }}"
> +  when:
> +    - ansible_distribution != "Fedora"
> +
> +- name: Install cvsps
> +  tags: gitr
> +  become: true
> +  become_flags: 'su - -c'
> +  become_method: ansible.builtin.sudo
> +  community.general.make:
> +    target: install
> +    chdir: "{{ cvsps_data }}"
> +  when:
> +    - ansible_distribution != "Fedora"
> +
>  - name: Set up an iSCSI initiator on the target node
>    ansible.builtin.include_role:
>      name: iscsi
> diff --git a/playbooks/roles/gitr/vars/RedHat.yml b/playbooks/roles/gitr/vars/RedHat.yml
> index 57f2f35..caf9840 100644
> --- a/playbooks/roles/gitr/vars/RedHat.yml
> +++ b/playbooks/roles/gitr/vars/RedHat.yml
> @@ -1,7 +1,6 @@
>  ---
>  gitr_packages:
>    - cvs
> -  - cvsps
>    - expat-devel
>    - gcc
>    - gettext
> @@ -9,11 +8,15 @@ gitr_packages:
>    - git-email
>    - glibc-utils
>    - libcurl-devel
> +  - make
>    - openssl-devel
>    - perl-Memoize
> -  - perl-TAP-Harness-Archive
>    - perl-Test-Harness
>    - perl-Test-Simple
>    - subversion
>    - subversion-perl
> +  - tar
>    - zlib-devel
> +
> +gitr_cpan_modules:
> +  - TAP::Harness::Archive
> -- 
> 2.44.0
> 
> 

Reviewed-by: Chuck Lever <chuck.lever@oracle.com>

With stock kdevops, I'm seeing some gitr failures (eg, t6300) on
Fedora 40 that were not there on Fedora 39. Wondering if this series
might help with that.

-- 
Chuck Lever

Re: [PATCH 2/2] gitr: Make gitr work on RHEL/CentOS

[Scott Mayhew]

On Wed, 04 Sep 2024, Chuck Lever wrote:

> On Tue, Sep 03, 2024 at 10:48:47AM -0400, Scott Mayhew wrote:
> > RHEL and CentOS do not ship cvsps or perl-TAP-Harness-Archive.  For the
> > former, we'll have to build it from git.  For the latter, we can install
> > it from CPAN, but we'll also have to install the perl-App-cpanminus
> > package to do so.
> > 
> > Also add 'make' and 'tar' to the dependencies list.  These were
> > previously getting installed inadvertently by the "Install btrfs-progs"
> > task in playbooks/roles/devconfig/tasks/install-deps/redhat/main.yml,
> > which was installing more than just btrfs-progs.
> > 
> > Signed-off-by: Scott Mayhew <smayhew@redhat.com>
> > ---
> >  playbooks/roles/gitr/defaults/main.yml |  3 ++
> >  playbooks/roles/gitr/tasks/main.yml    | 62 ++++++++++++++++++++++++++
> >  playbooks/roles/gitr/vars/RedHat.yml   |  7 ++-
> >  3 files changed, 70 insertions(+), 2 deletions(-)
> > 
> > diff --git a/playbooks/roles/gitr/defaults/main.yml b/playbooks/roles/gitr/defaults/main.yml
> > index bcfbaef..ff737ca 100644
> > --- a/playbooks/roles/gitr/defaults/main.yml
> > +++ b/playbooks/roles/gitr/defaults/main.yml
> > @@ -18,3 +18,6 @@ gitr_thread_single: false
> >  gitr_thread_fast: false
> >  gitr_thread_stress: false
> >  gitr_thread_custom: false
> > +
> > +cvsps_git: https://github.com/andreyvit/cvsps.git
> > +cvsps_data: "{{ data_path }}/cvsps"
> > diff --git a/playbooks/roles/gitr/tasks/main.yml b/playbooks/roles/gitr/tasks/main.yml
> > index 330350c..2eafeaa 100644
> > --- a/playbooks/roles/gitr/tasks/main.yml
> > +++ b/playbooks/roles/gitr/tasks/main.yml
> > @@ -11,6 +11,11 @@
> >    failed_when: false
> >    tags: vars
> >  
> > +- name: Set up the /data mount point
> > +  tags: ['data_partition']
> > +  ansible.builtin.include_role:
> > +    name: create_data_partition
> > +
> >  - name: Set the name of the test group
> >    tags: vars
> >    ansible.builtin.set_fact:
> > @@ -42,6 +47,20 @@
> >        paths:
> >          - 'vars'
> >  
> > +- name: Update gitr depencies for RHEL/Centos
> > +  tags: gitr
> > +  ansible.builtin.set_fact:
> > +    gitr_packages: "{{ gitr_packages + ['perl-App-cpanminus'] }}"
> > +  when:
> > +    - ansible_distribution != "Fedora"
> > +
> > +- name: Update gitr dependencies for Fedora
> > +  tags: gitr
> > +  ansible.builtin.set_fact:
> > +    gitr_packages: "{{ gitr_packages + ['cvsps', 'perl-TAP-Harness-Archive'] }}"
> > +  when:
> > +    - ansible_distribution == "Fedora"
> > +
> >  - name: Install dependencies for gitr
> >    tags: gitr
> >    become: true
> > @@ -51,6 +70,49 @@
> >      name: "{{ gitr_packages }}"
> >      state: present
> >  
> > +- name: Install CPAN modules for gitr
> > +  tags: gitr
> > +  become: true
> > +  become_flags: 'su - -c'
> > +  become_method: ansible.builtin.sudo
> > +  community.general.cpanm:
> > +    name: "{{ item }}"
> > +  with_items: "{{ gitr_cpan_modules }}"
> > +  when:
> > +    - ansible_distribution != "Fedora"
> > +
> > +- name: git clone cvsps
> > +  tags: gitr
> > +  ansible.builtin.git:
> > +    repo: "{{ cvsps_git }}"
> > +    dest: "{{ cvsps_data }}"
> > +    update: true
> > +  retries: 3
> > +  delay: 5
> > +  register: result
> > +  until: not result.failed
> > +  when:
> > +    - ansible_distribution != "Fedora"
> > +
> > +- name: Build cvsps
> > +  tags: gitr
> > +  community.general.make:
> > +    chdir: "{{ cvsps_data }}"
> > +    jobs: "{{ ansible_processor_nproc }}"
> > +  when:
> > +    - ansible_distribution != "Fedora"
> > +
> > +- name: Install cvsps
> > +  tags: gitr
> > +  become: true
> > +  become_flags: 'su - -c'
> > +  become_method: ansible.builtin.sudo
> > +  community.general.make:
> > +    target: install
> > +    chdir: "{{ cvsps_data }}"
> > +  when:
> > +    - ansible_distribution != "Fedora"
> > +
> >  - name: Set up an iSCSI initiator on the target node
> >    ansible.builtin.include_role:
> >      name: iscsi
> > diff --git a/playbooks/roles/gitr/vars/RedHat.yml b/playbooks/roles/gitr/vars/RedHat.yml
> > index 57f2f35..caf9840 100644
> > --- a/playbooks/roles/gitr/vars/RedHat.yml
> > +++ b/playbooks/roles/gitr/vars/RedHat.yml
> > @@ -1,7 +1,6 @@
> >  ---
> >  gitr_packages:
> >    - cvs
> > -  - cvsps
> >    - expat-devel
> >    - gcc
> >    - gettext
> > @@ -9,11 +8,15 @@ gitr_packages:
> >    - git-email
> >    - glibc-utils
> >    - libcurl-devel
> > +  - make
> >    - openssl-devel
> >    - perl-Memoize
> > -  - perl-TAP-Harness-Archive
> >    - perl-Test-Harness
> >    - perl-Test-Simple
> >    - subversion
> >    - subversion-perl
> > +  - tar
> >    - zlib-devel
> > +
> > +gitr_cpan_modules:
> > +  - TAP::Harness::Archive
> > -- 
> > 2.44.0
> > 
> > 
> 
> Reviewed-by: Chuck Lever <chuck.lever@oracle.com>
> 
> With stock kdevops, I'm seeing some gitr failures (eg, t6300) on
> Fedora 40 that were not there on Fedora 39. Wondering if this series
> might help with that.

Unfortunately no - the only effect that these patches should have on
Fedora is that the devconfig playbook is no longer accidentally
installing extra packages.

I'm seeing this failures on 6.10.6-200.fc40.x86_64:

Test Summary Report
-------------------
./t6300-for-each-ref.sh                            (Wstat: 256 (exited 1) Tests: 418 Failed: 3)
  Failed tests:  35, 107-108
  Non-zero exit status: 1
./t9001-send-email.sh                              (Wstat: 256 (exited 1) Tests: 199 Failed: 4)
  Failed tests:  37, 48-50
  Non-zero exit status: 1
Files=999, Tests=29254, 1711 wallclock secs (17.16 usr  4.92 sys + 1259.25 cusr 2402.60 csys = 3683.93 CPU)
Result: FAIL

and the *.stderr files have no info in them at all.

Everything passed on 6.10.6-100.fc39.x86_64.

> 
> -- 
> Chuck Lever
> 

Re: [PATCH 2/2] gitr: Make gitr work on RHEL/CentOS

[Chuck Lever]

On Wed, Sep 04, 2024 at 02:07:33PM -0400, Scott Mayhew wrote:
> On Wed, 04 Sep 2024, Chuck Lever wrote:
> > With stock kdevops, I'm seeing some gitr failures (eg, t6300) on
> > Fedora 40 that were not there on Fedora 39. Wondering if this series
> > might help with that.
> 
> Unfortunately no - the only effect that these patches should have on
> Fedora is that the devconfig playbook is no longer accidentally
> installing extra packages.
> 
> I'm seeing this failures on 6.10.6-200.fc40.x86_64:
> 
> Test Summary Report
> -------------------
> ./t6300-for-each-ref.sh                            (Wstat: 256 (exited 1) Tests: 418 Failed: 3)
>   Failed tests:  35, 107-108
>   Non-zero exit status: 1
> ./t9001-send-email.sh                              (Wstat: 256 (exited 1) Tests: 199 Failed: 4)
>   Failed tests:  37, 48-50
>   Non-zero exit status: 1
> Files=999, Tests=29254, 1711 wallclock secs (17.16 usr  4.92 sys + 1259.25 cusr 2402.60 csys = 3683.93 CPU)
> Result: FAIL
> 
> and the *.stderr files have no info in them at all.

Aye, that's what I see on F40.


-- 
Chuck Lever