[PATCH 0/5] scripts/ci: update build environment playbooks

qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed

* [PATCH 0/5] scripts/ci: update build environment playbooks
@ 2023-01-19 15:41 Paolo Bonzini
  2023-01-19 15:41 ` [PATCH 1/5] scripts/ci: remove unnecessary checks from CentOS playbook Paolo Bonzini
                   ` (4 more replies)
  0 siblings, 5 replies; 6+ messages in thread
From: Paolo Bonzini @ 2023-01-19 15:41 UTC (permalink / raw)
  To: qemu-devel; +Cc: crosa

This series cleans up and updates the Ansible playbooks in scripts/ci,
with the aim of enabling Python 3.8 in the custom CentOS runner.
This is the first step towards bumping the minimum required version
of Python past 3.6, because the custom runner has to be updated manually
by Cleber unlike the containers in tests/docker/dockerfiles.

The patches comprise other changes than just updating Python 3.8:
- small cleanups (patch 1)
- support CentOS Stream 8 (patch 2)
- make the playbook actually work (patch 3)
- do not force me to test two different sets of packages :) (patch 4)

Paolo Bonzini (5):
  scripts/ci: remove unnecessary checks from CentOS playbook
  scripts/ci: support CentOS Stream 8 in build-environment.yaml
  scripts/ci: add capstone development packages
  scripts/ci: unify package lists for CentOS in build-environment files
  scripts/ci: bump CentOS Python to 3.8

 .../org.centos/stream/8/build-environment.yml | 38 ++++++++++--
 .../ci/org.centos/stream/8/x86_64/configure   |  1 +
 scripts/ci/setup/build-environment.yml        | 59 +++++++++++++++++--
 3 files changed, 90 insertions(+), 8 deletions(-)

-- 
2.38.1



^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH 1/5] scripts/ci: remove unnecessary checks from CentOS playbook
  2023-01-19 15:41 [PATCH 0/5] scripts/ci: update build environment playbooks Paolo Bonzini
@ 2023-01-19 15:41 ` Paolo Bonzini
  2023-01-19 15:41 ` [PATCH 2/5] scripts/ci: support CentOS Stream 8 in build-environment.yaml Paolo Bonzini
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Paolo Bonzini @ 2023-01-19 15:41 UTC (permalink / raw)
  To: qemu-devel; +Cc: crosa

The playbook is meant for a CentOS 8 install and is already checking
/etc/redhat-release.  Simplify it by checking only the result of the
"extra check" rather than the Ansible facts dictionary.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 scripts/ci/org.centos/stream/8/build-environment.yml | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/scripts/ci/org.centos/stream/8/build-environment.yml b/scripts/ci/org.centos/stream/8/build-environment.yml
index 42b0471634c2..fe8807b7b074 100644
--- a/scripts/ci/org.centos/stream/8/build-environment.yml
+++ b/scripts/ci/org.centos/stream/8/build-environment.yml
@@ -17,8 +17,6 @@
         option: enabled
         value: "1"
       when:
-        - ansible_facts['distribution'] == 'CentOS'
-        - ansible_facts['distribution_major_version'] == '8'
         - centos_stream_8
 
     - name: Install basic packages to build QEMU on CentOS Stream 8
@@ -46,6 +44,4 @@
           - systemd-devel
         state: present
       when:
-        - ansible_facts['distribution'] == 'CentOS'
-        - ansible_facts['distribution_major_version'] == '8'
         - centos_stream_8
-- 
2.38.1



^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [PATCH 2/5] scripts/ci: support CentOS Stream 8 in build-environment.yaml
  2023-01-19 15:41 [PATCH 0/5] scripts/ci: update build environment playbooks Paolo Bonzini
  2023-01-19 15:41 ` [PATCH 1/5] scripts/ci: remove unnecessary checks from CentOS playbook Paolo Bonzini
@ 2023-01-19 15:41 ` Paolo Bonzini
  2023-01-19 15:41 ` [PATCH 3/5] scripts/ci: add capstone development packages Paolo Bonzini
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Paolo Bonzini @ 2023-01-19 15:41 UTC (permalink / raw)
  To: qemu-devel; +Cc: crosa

Update the CI playbook so that it is able to prepare a system with a
fresh CentOS Stream 8 install, rather than just support RHEL.  The
ninja-build package is only available if the CentOS Powertools are
enabled, which indeed is what the org.centos/stream/8 playbook is
already doing.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 scripts/ci/setup/build-environment.yml | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/build-environment.yml
index b04c2b7ceee9..ad63bbb3ccf7 100644
--- a/scripts/ci/setup/build-environment.yml
+++ b/scripts/ci/setup/build-environment.yml
@@ -130,6 +130,16 @@
         - ansible_facts['distribution_version'] == '20.04'
         - ansible_facts['architecture'] == 'aarch64'
 
+    - name: Enable PowerTools repo on CentOS 8
+      ini_file:
+        path: /etc/yum.repos.d/CentOS-Stream-PowerTools.repo
+        section: powertools
+        option: enabled
+        value: "1"
+      when:
+        - ansible_facts['distribution_file_variety'] == 'CentOS'
+        - ansible_facts['distribution_major_version'] == '8'
+
     - name: Install basic packages to build QEMU on EL8
       dnf:
         # This list of packages start with tests/docker/dockerfiles/centos8.docker
@@ -165,7 +175,7 @@
           - zlib-devel
         state: present
       when:
-        - ansible_facts['distribution_file_variety'] == 'RedHat'
+        - ansible_facts['distribution_file_variety'] in ['RedHat', 'CentOS']
         - ansible_facts['distribution_version'] == '8'
 
     - name: Install packages only available on x86 and aarch64
@@ -175,6 +185,6 @@
           - spice-server
         state: present
       when:
-        - ansible_facts['distribution_file_variety'] == 'RedHat'
+        - ansible_facts['distribution_file_variety'] in ['RedHat', 'CentOS']
         - ansible_facts['distribution_version'] == '8'
         - ansible_facts['architecture'] == 'aarch64' or ansible_facts['architecture'] == 'x86_64'
-- 
2.38.1



^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [PATCH 3/5] scripts/ci: add capstone development packages
  2023-01-19 15:41 [PATCH 0/5] scripts/ci: update build environment playbooks Paolo Bonzini
  2023-01-19 15:41 ` [PATCH 1/5] scripts/ci: remove unnecessary checks from CentOS playbook Paolo Bonzini
  2023-01-19 15:41 ` [PATCH 2/5] scripts/ci: support CentOS Stream 8 in build-environment.yaml Paolo Bonzini
@ 2023-01-19 15:41 ` Paolo Bonzini
  2023-01-19 15:41 ` [PATCH 4/5] scripts/ci: unify package lists for CentOS in build-environment files Paolo Bonzini
  2023-01-19 15:41 ` [PATCH 5/5] scripts/ci: bump CentOS Python to 3.8 Paolo Bonzini
  4 siblings, 0 replies; 6+ messages in thread
From: Paolo Bonzini @ 2023-01-19 15:41 UTC (permalink / raw)
  To: qemu-devel; +Cc: crosa

Capstone is not provided anymore by QEMU as a submodule since
commit 83602083b4ad ("capstone: Remove the capstone submodule", 2022-05-18).
For the org.centos/stream/8 playbook to actually provide a usable
build environment, it needs to include the capstone-devel package.

Include it in the "basic" build packages as well so that the resulting
QEMU has full disassembling functionality.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 scripts/ci/org.centos/stream/8/build-environment.yml |  9 +++++++++
 scripts/ci/setup/build-environment.yml               | 11 +++++++++++
 2 files changed, 20 insertions(+)

diff --git a/scripts/ci/org.centos/stream/8/build-environment.yml b/scripts/ci/org.centos/stream/8/build-environment.yml
index fe8807b7b074..95474ad034dc 100644
--- a/scripts/ci/org.centos/stream/8/build-environment.yml
+++ b/scripts/ci/org.centos/stream/8/build-environment.yml
@@ -10,6 +10,14 @@
       check_mode: yes
       register: centos_stream_8
 
+    - name: Enable EPEL repo on CentOS Stream 8
+      dnf:
+        name:
+          - epel-release
+        state: present
+      when:
+        - centos_stream_8
+
     - name: Enable PowerTools repo on CentOS Stream 8
       ini_file:
         path: /etc/yum.repos.d/CentOS-Stream-PowerTools.repo
@@ -22,6 +30,7 @@
     - name: Install basic packages to build QEMU on CentOS Stream 8
       dnf:
         name:
+          - capstone-devel
           - device-mapper-multipath-devel
           - glusterfs-api-devel
           - gnutls-devel
diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/build-environment.yml
index ad63bbb3ccf7..2e5f3b06028a 100644
--- a/scripts/ci/setup/build-environment.yml
+++ b/scripts/ci/setup/build-environment.yml
@@ -46,6 +46,7 @@
           - libbrlapi-dev
           - libbz2-dev
           - libcacard-dev
+          - libcapstone-dev
           - libcap-ng-dev
           - libcurl4-gnutls-dev
           - libdrm-dev
@@ -130,6 +131,15 @@
         - ansible_facts['distribution_version'] == '20.04'
         - ansible_facts['architecture'] == 'aarch64'
 
+    - name: Enable EPEL repo on EL8
+      dnf:
+        name:
+          - epel-release
+        state: present
+      when:
+        - ansible_facts['distribution_file_variety'] in ['RedHat', 'CentOS']
+        - ansible_facts['distribution_major_version'] == '8'
+
     - name: Enable PowerTools repo on CentOS 8
       ini_file:
         path: /etc/yum.repos.d/CentOS-Stream-PowerTools.repo
@@ -148,6 +158,7 @@
         name:
           - bzip2
           - bzip2-devel
+          - capstone-devel
           - dbus-daemon
           - diffutils
           - gcc
-- 
2.38.1



^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [PATCH 4/5] scripts/ci: unify package lists for CentOS in build-environment files
  2023-01-19 15:41 [PATCH 0/5] scripts/ci: update build environment playbooks Paolo Bonzini
                   ` (2 preceding siblings ...)
  2023-01-19 15:41 ` [PATCH 3/5] scripts/ci: add capstone development packages Paolo Bonzini
@ 2023-01-19 15:41 ` Paolo Bonzini
  2023-01-19 15:41 ` [PATCH 5/5] scripts/ci: bump CentOS Python to 3.8 Paolo Bonzini
  4 siblings, 0 replies; 6+ messages in thread
From: Paolo Bonzini @ 2023-01-19 15:41 UTC (permalink / raw)
  To: qemu-devel; +Cc: crosa

scripts/ci/org.centos/stream/8/build-environment.yml has a slightly different
list of packages compared to scripts/ci/setup/build-environment.yaml.  Make
them the same for simplicity.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 .../org.centos/stream/8/build-environment.yml | 25 +++++++++++++++++++
 scripts/ci/setup/build-environment.yml        | 20 ++++++++++++++-
 2 files changed, 44 insertions(+), 1 deletion(-)

diff --git a/scripts/ci/org.centos/stream/8/build-environment.yml b/scripts/ci/org.centos/stream/8/build-environment.yml
index 95474ad034dc..b1768d18afac 100644
--- a/scripts/ci/org.centos/stream/8/build-environment.yml
+++ b/scripts/ci/org.centos/stream/8/build-environment.yml
@@ -30,13 +30,26 @@
     - name: Install basic packages to build QEMU on CentOS Stream 8
       dnf:
         name:
+          - bzip2
+          - bzip2-devel
           - capstone-devel
+          - dbus-daemon
           - device-mapper-multipath-devel
+          - diffutils
+          - gcc
+          - gcc-c++
+          - genisoimage
+          - gettext
+          - git
+          - glib2-devel
           - glusterfs-api-devel
           - gnutls-devel
+          - libaio-devel
           - libcap-ng-devel
           - libcurl-devel
+          - libepoxy-devel
           - libfdt-devel
+          - libgcrypt-devel
           - libiscsi-devel
           - libpmem-devel
           - librados-devel
@@ -44,13 +57,25 @@
           - libseccomp-devel
           - libssh-devel
           - libxkbcommon-devel
+          - lzo-devel
+          - make
+          - mesa-libEGL-devel
+          - nettle-devel
           - ninja-build
+          - nmap-ncat
           - numactl-devel
+          - pixman-devel
+          - python36
           - python3-sphinx
+          - rdma-core-devel
           - redhat-rpm-config
           - snappy-devel
+          - spice-glib-devel
           - spice-server-devel
           - systemd-devel
+          - systemtap-sdt-devel
+          - tar
+          - zlib-devel
         state: present
       when:
         - centos_stream_8
diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/build-environment.yml
index 2e5f3b06028a..2274f736f7de 100644
--- a/scripts/ci/setup/build-environment.yml
+++ b/scripts/ci/setup/build-environment.yml
@@ -160,6 +160,7 @@
           - bzip2-devel
           - capstone-devel
           - dbus-daemon
+          - device-mapper-multipath-devel
           - diffutils
           - gcc
           - gcc-c++
@@ -167,20 +168,36 @@
           - gettext
           - git
           - glib2-devel
+          - glusterfs-api-devel
+          - gnutls-devel
           - libaio-devel
+          - libcap-ng-devel
+          - libcurl-devel
           - libepoxy-devel
+          - libfdt-devel
           - libgcrypt-devel
+          - libiscsi-devel
+          - libpmem-devel
+          - librados-devel
+          - librbd-devel
+          - libseccomp-devel
+          - libssh-devel
+          - libxkbcommon-devel
           - lzo-devel
           - make
           - mesa-libEGL-devel
           - nettle-devel
           - ninja-build
           - nmap-ncat
-          - perl-Test-Harness
+          - numactl-devel
           - pixman-devel
           - python36
+          - python3-sphinx
           - rdma-core-devel
+          - redhat-rpm-config
+          - snappy-devel
           - spice-glib-devel
+          - systemd-devel
           - systemtap-sdt-devel
           - tar
           - zlib-devel
@@ -194,6 +211,7 @@
         # Spice server not available in ppc64le
         name:
           - spice-server
+          - spice-server-devel
         state: present
       when:
         - ansible_facts['distribution_file_variety'] in ['RedHat', 'CentOS']
-- 
2.38.1



^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [PATCH 5/5] scripts/ci: bump CentOS Python to 3.8
  2023-01-19 15:41 [PATCH 0/5] scripts/ci: update build environment playbooks Paolo Bonzini
                   ` (3 preceding siblings ...)
  2023-01-19 15:41 ` [PATCH 4/5] scripts/ci: unify package lists for CentOS in build-environment files Paolo Bonzini
@ 2023-01-19 15:41 ` Paolo Bonzini
  4 siblings, 0 replies; 6+ messages in thread
From: Paolo Bonzini @ 2023-01-19 15:41 UTC (permalink / raw)
  To: qemu-devel; +Cc: crosa

In preparation for bumping the minimum requirement for the Python
runtime in QEMU 8.0, use the Python 3.8 package in the CentOS Stream 8
build environments.  To cover all bases, the "generic" playbook sets up
alternatives so that /usr/bin/python3 points to Python 3.8, while the
custom runner uses the --python option to configure.  The latter is
probably more suitable for the task of building an RPM file.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 .../ci/org.centos/stream/8/build-environment.yml   |  2 +-
 scripts/ci/org.centos/stream/8/x86_64/configure    |  1 +
 scripts/ci/setup/build-environment.yml             | 14 +++++++++++++-
 3 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/scripts/ci/org.centos/stream/8/build-environment.yml b/scripts/ci/org.centos/stream/8/build-environment.yml
index b1768d18afac..0d094d70c3b3 100644
--- a/scripts/ci/org.centos/stream/8/build-environment.yml
+++ b/scripts/ci/org.centos/stream/8/build-environment.yml
@@ -65,7 +65,7 @@
           - nmap-ncat
           - numactl-devel
           - pixman-devel
-          - python36
+          - python38
           - python3-sphinx
           - rdma-core-devel
           - redhat-rpm-config
diff --git a/scripts/ci/org.centos/stream/8/x86_64/configure b/scripts/ci/org.centos/stream/8/x86_64/configure
index 75882faa9ca5..65eacf3c56bb 100755
--- a/scripts/ci/org.centos/stream/8/x86_64/configure
+++ b/scripts/ci/org.centos/stream/8/x86_64/configure
@@ -16,6 +16,7 @@
 # that patches adding downstream specific devices are not available.
 #
 ../configure \
+--python=/usr/bin/python3.8 \
 --prefix="/usr" \
 --libdir="/usr/lib64" \
 --datadir="/usr/share" \
diff --git a/scripts/ci/setup/build-environment.yml b/scripts/ci/setup/build-environment.yml
index 2274f736f7de..78b1021cd420 100644
--- a/scripts/ci/setup/build-environment.yml
+++ b/scripts/ci/setup/build-environment.yml
@@ -191,7 +191,7 @@
           - nmap-ncat
           - numactl-devel
           - pixman-devel
-          - python36
+          - python38
           - python3-sphinx
           - rdma-core-devel
           - redhat-rpm-config
@@ -217,3 +217,15 @@
         - ansible_facts['distribution_file_variety'] in ['RedHat', 'CentOS']
         - ansible_facts['distribution_version'] == '8'
         - ansible_facts['architecture'] == 'aarch64' or ansible_facts['architecture'] == 'x86_64'
+
+    - name: Check whether the Python runtime version is managed by alternatives
+      stat:
+        path: /etc/alternatives/python3
+      register: python3
+
+    - name: Set default Python runtime to 3.8 on EL8
+      command: alternatives --set python3 /usr/bin/python3.8
+      when:
+        - ansible_facts['distribution_file_variety'] in ['RedHat', 'CentOS']
+        - ansible_facts['distribution_version'] == '8'
+        - python3.stat.islnk and python3.stat.lnk_target != '/usr/bin/python3.8'
-- 
2.38.1



^ permalink raw reply related	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2023-01-19 15:45 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-01-19 15:41 [PATCH 0/5] scripts/ci: update build environment playbooks Paolo Bonzini
2023-01-19 15:41 ` [PATCH 1/5] scripts/ci: remove unnecessary checks from CentOS playbook Paolo Bonzini
2023-01-19 15:41 ` [PATCH 2/5] scripts/ci: support CentOS Stream 8 in build-environment.yaml Paolo Bonzini
2023-01-19 15:41 ` [PATCH 3/5] scripts/ci: add capstone development packages Paolo Bonzini
2023-01-19 15:41 ` [PATCH 4/5] scripts/ci: unify package lists for CentOS in build-environment files Paolo Bonzini
2023-01-19 15:41 ` [PATCH 5/5] scripts/ci: bump CentOS Python to 3.8 Paolo Bonzini

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).