[RFC PATCH 2/3] playbooks: Add a build_linux role

[cel@kernel.org]

From: Chuck Lever <chuck.lever@oracle.com>

Currently, for cloud configurations or when 9p is disabled, kdevops
builds the test kernel on each test runner. This is inefficient,
and gets worse as the test matrix for a single kernel version scales
out.

Instead we want to build the test kernel once and make the build
artifacts available for test runners to install. This would be
similar to what KOTD does now, except it does not require setting up
a separate yum repo.

For this experiment, I've created a stripped down version of the
bootlinux role that has only the steps needed to build the kernel on
a target node. It adds some steps to create kernel packages and then
fetch them to the control host.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 .gitignore                                    |   2 +
 playbooks/build_linux.yml                     |   4 +
 playbooks/roles/build_linux/README.md         |  74 +++++
 playbooks/roles/build_linux/defaults/main.yml |  38 +++
 .../tasks/install-deps/debian/main.yml        |  46 +++
 .../tasks/install-deps/redhat/main.yml        | 102 ++++++
 .../tasks/install-deps/suse/main.yml          |  31 ++
 playbooks/roles/build_linux/tasks/main.yml    | 295 ++++++++++++++++++
 workflows/linux/Makefile                      |   7 +
 9 files changed, 599 insertions(+)
 create mode 100644 playbooks/build_linux.yml
 create mode 100644 playbooks/roles/build_linux/README.md
 create mode 100644 playbooks/roles/build_linux/defaults/main.yml
 create mode 100644 playbooks/roles/build_linux/tasks/install-deps/debian/main.yml
 create mode 100644 playbooks/roles/build_linux/tasks/install-deps/redhat/main.yml
 create mode 100644 playbooks/roles/build_linux/tasks/install-deps/suse/main.yml
 create mode 100644 playbooks/roles/build_linux/tasks/main.yml

diff --git a/.gitignore b/.gitignore
index 45113a669390..f51213a59ad9 100644
--- a/.gitignore
+++ b/.gitignore
@@ -86,6 +86,8 @@ playbooks/roles/linux-mirror/linux-mirror-systemd/mirrors.yaml
 #   yet.
 workflows/selftests/results/
 
+workflows/linux/artifacts/
+
 workflows/linux/refs/default/Kconfig.linus
 workflows/linux/refs/default/Kconfig.next
 workflows/linux/refs/default/Kconfig.stable
diff --git a/playbooks/build_linux.yml b/playbooks/build_linux.yml
new file mode 100644
index 000000000000..094441d1befd
--- /dev/null
+++ b/playbooks/build_linux.yml
@@ -0,0 +1,4 @@
+---
+- hosts: all
+  roles:
+    - role: build_linux
diff --git a/playbooks/roles/build_linux/README.md b/playbooks/roles/build_linux/README.md
new file mode 100644
index 000000000000..cbf9755ce5e0
--- /dev/null
+++ b/playbooks/roles/build_linux/README.md
@@ -0,0 +1,74 @@
+build_linux
+===========
+
+The build_linux role downloads and builds the Linux kernel.  It also
+lets you apply custom patches, remove kernels, etc.; anything you
+have to do with regards to generic kernel development.
+
+By default, it tracks one of the latest stable kernels that are
+still supported using the linux stable git tree.
+
+Requirements
+------------
+
+A separate block device is required on which to create the file
+system where the test kernel is built.
+
+Role Variables
+--------------
+
+  * infer_uid_and_group: defaults to False, if set to True, then we will ignore
+    the passed on data_user and data_group and instead try to infer this by
+    inspecting the `whoami` and getent on the logged in target system we are
+    provisioning. So if user sam is running able on a host, targetting a system
+    called foofighter and logging into that system using username pincho,
+    then the data_user will be set overwritten and set to pincho. We will then
+    also lookup for pincho's default group id and use that for data_group.
+    This is useful if you are targetting a slew of systems and don't really
+    want to deal with the complexities of the username and group, and the
+    default target username you use to ssh into a system suffices to use as
+    a base. This is set to False to remain compatible with old users of
+    this role.
+  * data_path: where to place the git trees we clone under
+  * data_user: the user to assign permissions to
+  * data_group: the group to assign permissions to
+
+  * data_device: the target device to use for the data partition
+  * data_fstype: the filesystem to store the data parition under
+  * data_label: the label to use
+  * data_fs_opts: the filesystem options to use, you want to ensure to add the
+    label
+
+  * target_linux_admin_name: your developer name
+  * target_linux_admin_email: your email
+  * target_linux_git: the git tree to clone, by default this is the linux-stable
+    tree
+  * target_linux_tree: the name of the tree
+  * target_linux_dir_path: where to place the tree on the target system
+
+  * target_linux_ref : the actual tag as used on linux, so v4.19.62
+  * target_linux_extra_patch: if defined an extra patch to apply with git
+     am prior to compilation
+  * target_linux_config: the configuration file to use
+
+Dependencies
+------------
+
+None.
+
+Example Playbook
+----------------
+
+Below is an example playbook, say a build_linux.yml file:
+
+```
+---
+- hosts: all
+  roles:
+    - role: build_linux
+```
+
+License
+-------
+
+copyleft-next-0.3.1
diff --git a/playbooks/roles/build_linux/defaults/main.yml b/playbooks/roles/build_linux/defaults/main.yml
new file mode 100644
index 000000000000..5ff5ece28b1e
--- /dev/null
+++ b/playbooks/roles/build_linux/defaults/main.yml
@@ -0,0 +1,38 @@
+# SPDX-License-Identifier copyleft-next-0.3.1
+---
+kdevops_bootlinux: false
+infer_uid_and_group: false
+
+data_path: "/data"
+data_user: "kdevops"
+data_group: "kdevops"
+data_device: "/dev/disk/by-id/nvme-QEMU_NVMe_Ctrl_kdevops0"
+data_fstype: "xfs"
+data_label: "data"
+data_fs_opts: "-L {{ disk_setup_label }}"
+
+target_linux_admin_name: "Hacker Amanda"
+target_linux_admin_email: "devnull@kernel.org"
+target_linux_git: "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git"
+target_linux_shallow_depth: 0
+target_linux_tree: "linux-stable"
+target_linux_dir_path: "{{ data_path }}/{{ target_linux_tree }}"
+kdevops_baseline_and_dev: false
+
+target_linux_ref: "v4.19.133"
+target_linux_delta_file:
+target_linux_config: "config-{{ target_linux_ref }}"
+make: "make"
+target_linux_make_cmd: "{{ make }} -j{{ ansible_processor_nproc }}"
+target_linux_make_install_cmd: "{{ target_linux_make_cmd }} modules_install install"
+
+build_artifacts_dir: "{{ topdir_path }}/workflows/linux/artifacts/"
+
+uninstall_kernel_enable: false
+
+bootlinux_b4_am_this_host: false
+
+kdevops_workflow_enable_cxl: false
+bootlinux_cxl_test: false
+
+bootlinux_tree_set_by_cli: false
diff --git a/playbooks/roles/build_linux/tasks/install-deps/debian/main.yml b/playbooks/roles/build_linux/tasks/install-deps/debian/main.yml
new file mode 100644
index 000000000000..ef57dca756ec
--- /dev/null
+++ b/playbooks/roles/build_linux/tasks/install-deps/debian/main.yml
@@ -0,0 +1,46 @@
+---
+# Install dependencies for building linux on Debian
+
+- name: Update apt cache
+  become: true
+  become_method: ansible.builtin.sudo
+  ansible.builtin.apt:
+    update_cache: true
+  tags:
+    - linux
+
+# apt-get build-dep does not capture all requirements
+- name: Install Linux kernel build dependencies
+  become: true
+  become_method: ansible.builtin.sudo
+  ansible.builtin.apt:
+    name:
+      - bison
+      - flex
+      - git
+      - gcc
+      - make
+      - gawk
+      - bc
+      - dump
+      - indent
+      - sed
+      - libssl-dev
+      - libelf-dev
+      - liburcu-dev
+      - xfsprogs
+      - e2fsprogs
+      - btrfs-progs
+      - ntfs-3g
+      - mdadm
+      - rpcbind
+      - portmap
+      - hwinfo
+      - open-iscsi
+      - python3-pip
+      - zstd
+      - libncurses-dev
+      - b4
+    state: present
+  tags:
+    - linux
diff --git a/playbooks/roles/build_linux/tasks/install-deps/redhat/main.yml b/playbooks/roles/build_linux/tasks/install-deps/redhat/main.yml
new file mode 100644
index 000000000000..371e7f7f11ee
--- /dev/null
+++ b/playbooks/roles/build_linux/tasks/install-deps/redhat/main.yml
@@ -0,0 +1,102 @@
+---
+- name: Enable installation of packages from EPEL
+  ansible.builtin.include_role:
+    name: epel-release
+  when:
+    - ansible_distribution != "Fedora"
+
+- name: Install packages for building the Linux kernel
+  become: true
+  become_method: ansible.builtin.sudo
+  ansible.builtin.dnf:
+    name: "{{ packages }}"
+    state: present
+    update_cache: true
+  retries: 3
+  delay: 5
+  register: result
+  until: result is succeeded
+  vars:
+    packages:
+      - bison
+      - flex
+      - git-core
+      - e2fsprogs
+      - xfsprogs
+      - xfsdump
+      - lvm2
+      - gcc
+      - make
+      - gawk
+      - bc
+      - dump
+      - libtool
+      - psmisc
+      - sed
+      - vim
+      - fio
+      - libaio-devel
+      - diffutils
+      - net-tools
+      - ncurses-devel
+      - xfsprogs
+      - e2fsprogs
+      - elfutils-libelf-devel
+      - ntfs-3g
+      - mdadm
+      - rpcbind
+      - portmap
+      - hwinfo
+      - iscsi-initiator-utils
+      - openssl
+      - openssl-devel
+      - dwarves
+      - userspace-rcu
+      - zstd
+
+- name: Install btrfs-progs
+  become: true
+  become_method: ansible.builtin.sudo
+  ansible.builtin.dnf:
+    name:
+      - btrfs-progs
+    state: present
+    update_cache: true
+  retries: 3
+  delay: 5
+  register: btrfs_result
+  until: btrfs_result is succeeded
+  when:
+    - ansible_distribution == "Fedora"
+
+- name: Install rpmbuild
+  become: true
+  become_method: ansible.builtin.sudo
+  ansible.builtin.dnf:
+    name:
+      - rpmbuild
+    state: present
+    update_cache: true
+  retries: 3
+  delay: 5
+  register: rpmbuild_result
+  until: rpmbuild_result is succeeded
+  when:
+    - ansible_distribution != "Fedora"
+
+- name: Install rpmbuild
+  become: true
+  become_method: ansible.builtin.sudo
+  ansible.builtin.dnf:
+    name:
+      - perl-core
+      - rpm-build
+      - rsync
+    state: present
+    update_cache: true
+  retries: 3
+  delay: 5
+  register: rpmbuild_result
+  until: rpmbuild_result is succeeded
+  when:
+    - ansible_distribution == "Fedora"
diff --git a/playbooks/roles/build_linux/tasks/install-deps/suse/main.yml b/playbooks/roles/build_linux/tasks/install-deps/suse/main.yml
new file mode 100644
index 000000000000..0260da5a912d
--- /dev/null
+++ b/playbooks/roles/build_linux/tasks/install-deps/suse/main.yml
@@ -0,0 +1,31 @@
+---
+- name: Install Linux kernel build dependencies for SUSE
+  become: true
+  become_method: ansible.builtin.sudo
+  community.general.zypper:
+    name:
+      - bison
+      - flex
+      - git-core
+      - gcc
+      - make
+      - gawk
+      - bc
+      - dump
+      - sed
+      - libopenssl-devel
+      - libelf-devel
+      - liburcu8
+      - diffutils
+      - net-tools
+      - ncurses-devel
+      - xfsprogs
+      - e2fsprogs
+      - btrfsprogs
+      - ntfs-3g
+      - mdadm
+      - rpcbind
+      - portmap
+      - hwinfo
+      - open-iscsi
+    disable_recommends: false
diff --git a/playbooks/roles/build_linux/tasks/main.yml b/playbooks/roles/build_linux/tasks/main.yml
new file mode 100644
index 000000000000..a95b5ee98bb3
--- /dev/null
+++ b/playbooks/roles/build_linux/tasks/main.yml
@@ -0,0 +1,295 @@
+---
+- name: Include optional extra_vars
+  ansible.builtin.include_vars:
+    file: "{{ item }}"
+  with_first_found:
+    - files:
+        - "../extra_vars.yml"
+        - "../extra_vars.yaml"
+        - "../extra_vars.json"
+      skip: true
+  failed_when: false
+
+- name: Debian-specific set up
+  ansible.builtin.import_tasks: install-deps/debian/main.yml
+  when:
+    - ansible_os_family == "Debian"
+
+- name: Red Hat-specific set up
+  ansible.builtin.import_tasks: install-deps/redhat/main.yml
+  when:
+    - ansible_os_family == "RedHat"
+
+- name: Suse-specific set up
+  ansible.builtin.import_tasks: install-deps/suse/main.yml
+  when:
+    - ansible_os_family == "Suse"
+
+- name: Install b4
+  become: true
+  become_method: ansible.builtin.sudo
+  ansible.builtin.pip:
+    name:
+      - b4
+  when:
+    - target_linux_install_b4 is defined
+    - target_linux_install_b4
+    - ansible_os_family == "Debian"
+
+- name: Create the /data partition
+  ansible.builtin.include_role:
+    name: create_data_partition
+
+- name: Wipe the build directory
+  ansible.builtin.file:
+    path: "{{ target_linux_dir_path }}"
+    state: absent
+
+- name: Clone {{ target_linux_tree }}
+  ansible.builtin.git:
+    repo: "{{ target_linux_git }}"
+    dest: "{{ target_linux_dir_path }}"
+    update: true
+    depth: "{{ target_linux_shallow_depth }}"
+    version: "{{ target_linux_ref }}"
+  retries: 3
+  delay: 5
+  register: git_result
+  until: not git_result.failed
+
+- name: Copy the kernel delta to the builder
+  ansible.builtin.template:
+    src: "{{ target_linux_extra_patch }}"
+    dest: "{{ target_linux_dir_path }}/{{ target_linux_extra_patch }}"
+    owner: "{{ data_user }}"
+    group: "{{ data_group }}"
+    mode: "u=rw,g=r,o=r"
+  when:
+    - target_linux_extra_patch is defined
+
+- name: Apply the kernel delta on the builder
+  ansible.builtin.command:
+    cmd: "git am {{ target_linux_extra_patch }}"  # noqa: command-instead-of-module
+    chdir: "{{ target_linux_dir_path }}"
+  register: git_am
+  changed_when: not git_am.failed
+  when:
+    - target_linux_extra_patch is defined
+
+- name: Set git user name and email
+  ansible.builtin.shell: |
+    if ! $(git config --get user.email) ; then
+      git config --global user.email user@example.com
+    fi
+    if ! $(git config --get user.name) ; then
+      git config --global user.name user
+    fi
+  register: git_config
+  changed_when: not git_config.failed
+  when:
+    - target_linux_apply_patch_message_id is defined
+    - target_linux_apply_patch_message_id | length > 0
+    - bootlinux_b4_am_this_host|bool
+
+- name: Apply a message patch set
+  ansible.builtin.shell:
+    cmd: |
+      set -o pipefail
+      b4 am -o - {{ target_linux_apply_patch_message_id }} | git am
+    chdir: "{{ target_linux_dir_path }}"
+  register: b4_am
+  changed_when: not b4_am.failed
+  when:
+    - target_linux_apply_patch_message_id is defined
+    - target_linux_apply_patch_message_id | length > 0
+    - bootlinux_b4_am_this_host|bool
+
+- name: Set the pathname of the .config templates directory
+  ansible.builtin.set_fact:
+    template_path: "{{ topdir_path }}/playbooks/roles/bootlinux/templates"
+
+- name: Check whether config-kdevops exists
+  delegate_to: localhost
+  ansible.builtin.stat:
+    path: "{{ template_path }}/config-kdevops"
+  register: config_kdevops
+
+- name: Found config-kdevops, using it for template
+  ansible.builtin.set_fact:
+    linux_config: "config-kdevops"
+  when: config_kdevops.stat.exists
+
+- name: No config-kdevops, looking for {{ target_linux_config }}
+  ansible.builtin.set_fact:
+    linux_config: "{{ target_linux_config }}"
+  when: not config_kdevops.stat.exists
+
+- name: Check whether specific kernel config exists for {{ target_linux_ref }}
+  delegate_to: localhost
+  ansible.builtin.stat:
+    path: "{{ template_path }}/{{ target_linux_config }}"
+  register: kernel_config
+
+- name: Find all linux-next configs
+  delegate_to: localhost
+  ansible.builtin.find:
+    paths: "{{ template_path }}"
+    patterns: "config-next*"
+    file_type: file
+    recurse: false
+  register: found_configs
+  when:
+    - not config_kdevops.stat.exists
+    - not kernel_config.stat.exists
+
+- name: Extract the date from the filenames
+  ansible.builtin.set_fact:
+    configs_with_dates: "{{ configs_with_dates | default([]) + [{'file': item.path, 'date': (item.path | regex_search('config-next-(\\d{8})')).split('-')[-1]}] }}"
+  loop: "{{ found_configs.files }}"
+  no_log: true
+  when:
+    - not config_kdevops.stat.exists
+    - not kernel_config.stat.exists
+    - item.path is search('config-next-(\\d{8})')
+
+- name: Sort configs based on date extracted from filename
+  ansible.builtin.set_fact:
+    sorted_configs: "{{ configs_with_dates | selectattr('date', 'defined') | sort(attribute='date', reverse=True) | map(attribute='file') | list }}"
+  when:
+    - not config_kdevops.stat.exists
+    - not kernel_config.stat.exists
+    - configs_with_dates | length > 0
+
+- name: Set latest linux-next config
+  ansible.builtin.set_fact:
+    latest_linux_next_config: "{{ sorted_configs[0] }}"
+  when:
+    - not config_kdevops.stat.exists and not kernel_config.stat.exists
+    - sorted_configs | length > 0
+
+- name: Use the specific kernel config or fallback to the latest linux-next
+  ansible.builtin.set_fact:
+    linux_config: "{{ target_linux_config | default('') if kernel_config.stat.exists else (latest_linux_next_config | default('') | basename) }}"
+  when:
+    - not config_kdevops.stat.exists
+    - not kernel_config.stat.exists
+    - latest_linux_next_config is defined
+
+- name: Verify that the Linux configuration file exists
+  delegate_to: localhost
+  ansible.builtin.stat:
+    path: "{{ template_path }}/{{ linux_config }}"
+  register: config_stat
+  when: linux_config is defined
+
+- name: Fail if the configuration file does not exist
+  ansible.builtin.fail:
+    msg: "The configuration file {{ template_path }}/{{ linux_config }} does not exist."
+  when: not config_stat.stat.exists
+
+- name: Copy configuration for Linux {{ target_linux_tree }}
+  ansible.builtin.template:
+    src: "{{ template_path }}/{{ linux_config }}"
+    dest: "{{ target_linux_dir_path }}/.config"
+    owner: "{{ data_user }}"
+    group: "{{ data_group }}"
+    mode: "u=rw,g=r,o=r"
+
+- name: Set the kernel localversion
+  ansible.builtin.lineinfile:
+    path: "{{ target_linux_dir_path }}/localversion"
+    line: "{{ target_linux_localversion }}"
+    mode: "u=rw,g=r,o=r"
+    create: true
+  when:
+    - target_linux_localversion is defined and target_linux_localversion != ""
+
+- name: Configure Linux {{ target_linux_tree }}
+  community.general.make:
+    chdir: "{{ target_linux_dir_path }}"
+    target: "olddefconfig"
+
+- name: Build {{ target_linux_tree }}
+  community.general.make:
+    chdir: "{{ target_linux_dir_path }}"
+    target: "all"
+    jobs: "{{ ansible_processor_nproc }}"
+
+- name: Remove the artifacts directory
+  delegate_to: localhost
+  ansible.builtin.file:
+    path: "{{ build_artifacts_dir }}"
+    state: absent
+
+- name: Create an empty artifacts directory
+  delegate_to: localhost
+  ansible.builtin.file:
+    path: "{{ build_artifacts_dir }}"
+    state: directory
+    mode: "u=rwx,g=rx,o=rx"
+
+- name: Build kernel .deb packages
+  when:
+    - ansible_os_family == "Debian"
+  block:
+    - name: Make the bindeb-pkg target
+      community.general.make:
+        chdir: "{{ target_linux_dir_path }}"
+        target: "bindeb-pkg"
+
+    - name: Find the build artifacts
+      ansible.builtin.find:
+        paths: "{{ target_linux_dir_path }}"
+        patterns: "*.deb"
+        file_type: file
+        recurse: true
+      register: found_debs
+
+    - name: Fetch the build artifacts to the control host
+      ansible.builtin.fetch:
+        src: "{{ item.path }}"
+        dest: "{{ build_artifacts_dir }}"
+        flat: true
+      loop: "{{ found_debs.files }}"
+      loop_control:
+        label: "Fetching {{ item.path }}"
+
+- name: Build kernel .rpm packages
+  when:
+    - ansible_os_family == "RedHat"
+  block:
+    - name: Make the binrpm-pkg target
+      community.general.make:
+        chdir: "{{ target_linux_dir_path }}"
+        target: "binrpm-pkg"
+
+    - name: Find the build artifacts
+      ansible.builtin.find:
+        paths: "{{ target_linux_dir_path }}/rpmbuild/RPMS"
+        patterns: "*.rpm"
+        file_type: file
+        recurse: true
+      register: found_rpms
+
+    - name: Fetch the build artifacts to the control host
+      ansible.builtin.fetch:
+        src: "{{ item.path }}"
+        dest: "{{ build_artifacts_dir }}"
+        flat: true
+      loop: "{{ found_rpms.files }}"
+      loop_control:
+        label: "Fetching {{ item.path }}"
+
+- name: Extract the release information of the built kernel
+  community.general.make:
+    chdir: "{{ target_linux_dir_path }}"
+    target: "kernelrelease"
+  register: kernelrelease
+
+- name: Store the kernel release information with the build artifacts
+  delegate_to: localhost
+  ansible.builtin.lineinfile:
+    create: true
+    line: "{{ kernelrelease.stdout }}"
+    mode: "u=rw,g=r,o=r"
+    path: "{{ build_artifacts_dir }}/kernel.release"
diff --git a/workflows/linux/Makefile b/workflows/linux/Makefile
index 65bbb8ae9a90..bb7441e71fda 100644
--- a/workflows/linux/Makefile
+++ b/workflows/linux/Makefile
@@ -84,6 +84,7 @@ linux-help-menu:
 	@echo "linux-clone        - Only clones Linux"
 	@echo "linux-grub-setup   - Ensures the appropriate target kernel is set to boot"
 	@echo "linux-reboot       - Reboot guests"
+	@echo "linux-packages     - Clones, builds, and packages a Linux kernel"
 	@echo "uname              - Prints current running kernel"
 
 PHONY += linux-help-end
@@ -159,6 +160,12 @@ linux-reboot:
 		$(KDEVOPS_HOSTFILE) $(KDEVOPS_PLAYBOOKS_DIR)/bootlinux.yml \
 		--extra-vars="$(BOOTLINUX_ARGS)" $(LIMIT_HOSTS) --tags vars,reboot
 
+PHONY += linux-packages
+linux-packages:
+	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) -i $(KDEVOPS_HOSTFILE) \
+		$(KDEVOPS_PLAYBOOKS_DIR)/build_linux.yml \
+		--extra-vars="$(BOOTLINUX_ARGS)" $(LIMIT_HOSTS)
+
 PHONY += uname
 uname:
 	$(Q)ansible all -i hosts -b -m command -a "uname -r" -o \
-- 
2.49.0