From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 472CA30CB49 for ; Fri, 19 Sep 2025 12:25:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758284742; cv=none; b=VpngBxHQCAawjNM2ByM+xLf05B0fs5p7h757HX6S4F1CPaI2tuminVwnpQYcAM9Ll4FK8+OVH35F6skmPXlkT+UeSH0+nqekN1QwZWocNs73jsMZV9P/+jOzPxP3FhSEHgcP5z1UNkB/3dztv6garBEXxT8IISImL14oZCtwXnw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758284742; c=relaxed/simple; bh=in/Firqn1IZ/YFxy3VW0OUdE9Iqx0r2fqewtGBHCoPI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=i8LtIwQZhPxRtoxOhn35rll9GQd2MKmWkIYrMlDGOYIVgvuR++jpiV+MDU3uJ1Eqtg1BzIOUjPu+hneO/E0Hlb7vaz7k/S5HyxnH2psdPuR8hGsIHdA/kk03cZyT7ZxqJZ72qTA/Hw+ifyx0YkF+qaFxxmmgB8pUtJ1P8XdbRpE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jCp6vFNo; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jCp6vFNo" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F06D0C4CEF9; Fri, 19 Sep 2025 12:25:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1758284741; bh=in/Firqn1IZ/YFxy3VW0OUdE9Iqx0r2fqewtGBHCoPI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=jCp6vFNoFFJ7bdlEIiS/unOT/gYa57Auihz3R+liBO+KN7E6Hzhwm5LY/N67KcxDF GiEnNnuziEDbybLD2YeP2AT30AWt2vElVg2ohz5ojbul4fWopcV3ZpYnnxjWzwz6Wo GH8+wb7OpN3e7yTgcBfVFQh4xwhJdK/NJKgmlvIw8JDI9yBx291AN0RLOH/v+zner6 QZ+LtFbpdKZFe+9sFK/GwFAcTm3g9aDkycb0P6yz/QS6IfaoCzasSZNpZ/KBouE8aH OgYagLLjSxowpHkV7QmA4RPyqiLX4GiAHixTdAnFzjVENW+6GScRomagTVUeGEtytF L6kervMDT4QVA== From: Daniel Gomez Date: Fri, 19 Sep 2025 14:25:08 +0200 Subject: [PATCH v2 2/4] workflows: bootlinux: add comprehensive ccache support Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit Message-Id: <20250919-kernel-fragment-support-v2-2-8d2b7b8cb4e4@samsung.com> References: <20250919-kernel-fragment-support-v2-0-8d2b7b8cb4e4@samsung.com> In-Reply-To: <20250919-kernel-fragment-support-v2-0-8d2b7b8cb4e4@samsung.com> To: Luis Chamberlain , Chuck Lever Cc: kdevops@lists.linux.dev, Daniel Gomez X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=openpgp-sha256; l=13246; i=da.gomez@samsung.com; h=from:subject:message-id; bh=NYHUlIWQF5+UIc6ia/9dUYXDwfIvsdw8M9mIRAf9fs4=; b=owEBbQKS/ZANAwAIAUCeo8QfGVH7AcsmYgBozUu4vYbW4a22YnjBIoXgyI4nDjLL/jfVWAtnH v7Nul6tHviJAjMEAAEIAB0WIQTvdRrhHw9z4bnGPFNAnqPEHxlR+wUCaM1LuAAKCRBAnqPEHxlR +w9rEACWkliCFsxq+GcXvW44bs+LVQ711LH2RaoQBKH9yOc86aPto641QRrt9Ey2gJfvk8z2qid a9YuInlZ/lAw2F8ZxxBfbhjH7xrLjdOwfc1uIKWWgPEVEQWMIOEqZ2c4HlqEVmlWyq7BDZWKqsj kH0yR0xdgfVA80+hnlXuWqQJeikxkjkPc93dXEgMczMix5gOGv7XYUY+DufQI6qgjYhy79rnxMz dhgcYUFL7zKYt3VCEkxcsKCQfHrKtjrcOYG7Z3sjEaoBMCLd+hKVXYokt85YBX+UhinOy7pKZh8 NEcQJChsf2M5TlqD9s395vbPzCUv4IXex2d/Yydqtz+ri8eE6wSgQSE1dz9It4WLsQYJOUEer82 vNFsr2M2rD8z7AzCUNQsdpQM7v1IxXE3fZ6QvmM4QjplyDXJZ8eoabQmKgLBO3fRR3WGPg8Q1bx RhlcXLHx+HWkalsGzWX5beO1ZUhZNsJqL5UbGPdgnDuhPuuI3LlDbOY6Q4MuyV0FTkFg1R59dY1 NjAUfmV9+WvFeKWnVPRPoSAMNRmSdOB617xc4iUb6AOeRBE82V9z8M2nzunBzcX4cTiw8dSk6KA +jrl1XO7Fk/zFrqIkfzjL1/h+kZsYl9ahZPpCFuCd/dN9Lk+fLsUeBOrimEstSThFwAkn7nuXqu XA/ZoYJUTDmICpg== X-Developer-Key: i=da.gomez@samsung.com; a=openpgp; fpr=B2A7A9CFDD03B540FF58B27185F56EA4E9E8138F From: Daniel Gomez Add ccache (compiler cache) support to accelerate kernel builds with both GCC and Clang toolchains. Implementation provides flexible configuration options while maintaining simplicity for common use cases. Key features: - System-wide ccache configuration (default) for immediate use - kdevops-managed configuration for project isolation - Automatic package installation across all supported distros - Modular environment variable composition for clean integration - LLVM documentation compliance (CC="ccache clang") - Seamless integration with reproducible builds and existing features Configuration: - Cache size specified in GiB (digits only, e.g., 10 = 10 GiB) - Uses automatic Kconfig→Ansible integration via .extra_vars_auto.yaml - Template-based configuration management with TOPDIR_PATH isolation - Organized in dedicated ccache submenu for clean UI Technical implementation: - Uses modular bootlinux_make_params for community.general.make module - Uses modular bootlinux_build_environment for shell tasks - Automatic compiler detection (ccache gcc vs ccache clang) - Enhanced debug output for build command validation Generated-by: Claude AI Signed-off-by: Daniel Gomez --- playbooks/roles/bootlinux/defaults/main.yml | 6 ++ playbooks/roles/bootlinux/tasks/build/9p.yml | 5 +- playbooks/roles/bootlinux/tasks/ccache.yml | 37 ++++++++++ .../bootlinux/tasks/install-deps/debian/main.yml | 1 + .../bootlinux/tasks/install-deps/redhat/main.yml | 1 + .../bootlinux/tasks/install-deps/suse/main.yml | 1 + .../tasks/install-minimal-deps/debian/main.yml | 1 + .../tasks/install-minimal-deps/redhat/main.yml | 2 + .../tasks/install-minimal-deps/suse/main.yml | 1 + playbooks/roles/bootlinux/tasks/main.yml | 9 +++ playbooks/roles/bootlinux/templates/ccache.conf.j2 | 27 ++++++++ workflows/linux/Kconfig | 78 ++++++++++++++++++++++ 12 files changed, 168 insertions(+), 1 deletion(-) diff --git a/playbooks/roles/bootlinux/defaults/main.yml b/playbooks/roles/bootlinux/defaults/main.yml index 2d2eb798..97adec30 100644 --- a/playbooks/roles/bootlinux/defaults/main.yml +++ b/playbooks/roles/bootlinux/defaults/main.yml @@ -37,6 +37,8 @@ target_linux_make_cmd: "{{ make }} -j{{ ansible_processor_vcpus }}" bootlinux_make_params: >- {{ ({} | combine( + {'CC': 'ccache clang' if bootlinux_compiler_clang|default(false)|bool else 'ccache gcc'} if bootlinux_ccache|default(false)|bool else {} + ) | combine( { 'KBUILD_BUILD_TIMESTAMP': '', 'KBUILD_BUILD_USER': 'kdevops', @@ -49,6 +51,10 @@ bootlinux_make_params: >- bootlinux_build_environment: >- {{ ({} | combine( + {'CCACHE_CONFIGPATH': topdir_path + '/.ccache/ccache.conf'} if (bootlinux_ccache|default(false)|bool and bootlinux_ccache_kdevops_managed|default(false)|bool) else {} + ) | combine( + {'CC': 'ccache clang' if bootlinux_compiler_clang|default(false)|bool else 'ccache gcc'} if bootlinux_ccache|default(false)|bool else {} + ) | combine( { 'KBUILD_BUILD_TIMESTAMP': '', 'KBUILD_BUILD_USER': 'kdevops', diff --git a/playbooks/roles/bootlinux/tasks/build/9p.yml b/playbooks/roles/bootlinux/tasks/build/9p.yml index 17b19cb3..6f03ed9a 100644 --- a/playbooks/roles/bootlinux/tasks/build/9p.yml +++ b/playbooks/roles/bootlinux/tasks/build/9p.yml @@ -164,8 +164,10 @@ build_jobs: {{ nproc_9p.stdout }} bootlinux_make_params: {{ bootlinux_make_params }} bootlinux_build_environment: {{ bootlinux_build_environment }} + bootlinux_ccache: {{ bootlinux_ccache|default(false) }} bootlinux_compiler_clang: {{ bootlinux_compiler_clang|default(false) }} bootlinux_reproducible_builds: {{ bootlinux_reproducible_builds|default(false) }} + bootlinux_ccache_kdevops_managed: {{ bootlinux_ccache_kdevops_managed|default(false) }} run_once: true delegate_to: localhost vars: @@ -178,7 +180,8 @@ echo "CC=${CC:-'not set'}" echo "PATH=${PATH}" which gcc || echo "gcc not found" - env | grep -E '^(CC|PATH)' | sort + which ccache || echo "ccache not found" + env | grep -E '^(CC|CCACHE|PATH)' | sort environment: "{{ bootlinux_build_environment }}" register: cc_test_result run_once: true diff --git a/playbooks/roles/bootlinux/tasks/ccache.yml b/playbooks/roles/bootlinux/tasks/ccache.yml new file mode 100644 index 00000000..38b2c034 --- /dev/null +++ b/playbooks/roles/bootlinux/tasks/ccache.yml @@ -0,0 +1,37 @@ +--- +# ccache configuration and setup + +- name: Create kdevops ccache directory structure + ansible.builtin.file: + path: "{{ item }}" + state: directory + mode: '0755' + loop: + - "{{ topdir_path }}/.ccache" + - "{{ bootlinux_ccache_dir }}" + when: + - bootlinux_ccache|default(false)|bool + - bootlinux_ccache_kdevops_managed|default(false)|bool + +- name: Generate kdevops-managed ccache configuration + ansible.builtin.template: + src: ccache.conf.j2 + dest: "{{ topdir_path }}/.ccache/ccache.conf" + mode: '0644' + when: + - bootlinux_ccache|default(false)|bool + - bootlinux_ccache_kdevops_managed|default(false)|bool + +- name: Display ccache configuration info + ansible.builtin.debug: + msg: | + ccache enabled: {{ bootlinux_ccache|default(false)|bool }} + ccache mode: {{ 'kdevops-managed' if bootlinux_ccache_kdevops_managed|default(false)|bool else 'system-wide' }} + {% if bootlinux_ccache_kdevops_managed|default(false)|bool %} + Config file: {{ topdir_path }}/.ccache/ccache.conf + Cache directory: {{ bootlinux_ccache_dir }} + Max size: {{ bootlinux_ccache_max_size }} GiB + {% endif %} + when: bootlinux_ccache|default(false)|bool + vars: + ansible_callback_diy_runner_on_ok_msg: "{{ ansible_callback_diy.result.output.msg }}" diff --git a/playbooks/roles/bootlinux/tasks/install-deps/debian/main.yml b/playbooks/roles/bootlinux/tasks/install-deps/debian/main.yml index 4edf8e1d..e5911029 100644 --- a/playbooks/roles/bootlinux/tasks/install-deps/debian/main.yml +++ b/playbooks/roles/bootlinux/tasks/install-deps/debian/main.yml @@ -39,4 +39,5 @@ - zstd - libncurses-dev - b4 + - ccache state: present diff --git a/playbooks/roles/bootlinux/tasks/install-deps/redhat/main.yml b/playbooks/roles/bootlinux/tasks/install-deps/redhat/main.yml index c7b18cd5..7f1955bf 100644 --- a/playbooks/roles/bootlinux/tasks/install-deps/redhat/main.yml +++ b/playbooks/roles/bootlinux/tasks/install-deps/redhat/main.yml @@ -52,6 +52,7 @@ - dwarves - userspace-rcu - zstd + - ccache - name: Install btrfs-progs become: true diff --git a/playbooks/roles/bootlinux/tasks/install-deps/suse/main.yml b/playbooks/roles/bootlinux/tasks/install-deps/suse/main.yml index 7f0cf249..5c9f0e3b 100644 --- a/playbooks/roles/bootlinux/tasks/install-deps/suse/main.yml +++ b/playbooks/roles/bootlinux/tasks/install-deps/suse/main.yml @@ -28,4 +28,5 @@ - portmap - hwinfo - open-iscsi + - ccache disable_recommends: false diff --git a/playbooks/roles/bootlinux/tasks/install-minimal-deps/debian/main.yml b/playbooks/roles/bootlinux/tasks/install-minimal-deps/debian/main.yml index fab4998a..cc05a44a 100644 --- a/playbooks/roles/bootlinux/tasks/install-minimal-deps/debian/main.yml +++ b/playbooks/roles/bootlinux/tasks/install-minimal-deps/debian/main.yml @@ -24,4 +24,5 @@ - make - gcc - kmod + - ccache state: present diff --git a/playbooks/roles/bootlinux/tasks/install-minimal-deps/redhat/main.yml b/playbooks/roles/bootlinux/tasks/install-minimal-deps/redhat/main.yml index 50651ab1..950dbfe1 100644 --- a/playbooks/roles/bootlinux/tasks/install-minimal-deps/redhat/main.yml +++ b/playbooks/roles/bootlinux/tasks/install-minimal-deps/redhat/main.yml @@ -10,6 +10,7 @@ - make - gcc - kmod + - ccache state: present when: - ansible_facts['distribution_major_version']|int < 8 @@ -22,6 +23,7 @@ - make - gcc - kmod + - ccache state: present when: - ansible_facts['distribution_major_version']|int >= 8 diff --git a/playbooks/roles/bootlinux/tasks/install-minimal-deps/suse/main.yml b/playbooks/roles/bootlinux/tasks/install-minimal-deps/suse/main.yml index a4cc27bb..d8f75240 100644 --- a/playbooks/roles/bootlinux/tasks/install-minimal-deps/suse/main.yml +++ b/playbooks/roles/bootlinux/tasks/install-minimal-deps/suse/main.yml @@ -10,4 +10,5 @@ - make - gcc - kmod-compat + - ccache state: present diff --git a/playbooks/roles/bootlinux/tasks/main.yml b/playbooks/roles/bootlinux/tasks/main.yml index 4110d596..eff6fb6b 100644 --- a/playbooks/roles/bootlinux/tasks/main.yml +++ b/playbooks/roles/bootlinux/tasks/main.yml @@ -32,6 +32,15 @@ - bootlinux_9p|bool - not workflow_linux_packaged|bool +# Setup ccache configuration +- name: Setup ccache configuration + ansible.builtin.import_tasks: + file: ccache.yml + when: + - bootlinux_ccache|default(false)|bool + - not workflow_linux_packaged|bool + tags: ['ccache'] + # We do this regardless of what distro you use - name: Install b4 become: true diff --git a/playbooks/roles/bootlinux/templates/ccache.conf.j2 b/playbooks/roles/bootlinux/templates/ccache.conf.j2 new file mode 100644 index 00000000..8c0a7bf9 --- /dev/null +++ b/playbooks/roles/bootlinux/templates/ccache.conf.j2 @@ -0,0 +1,27 @@ +# kdevops-managed ccache configuration +# Generated automatically - do not edit manually + +# Cache settings +cache_dir = {{ bootlinux_ccache_dir }} +max_size = {{ bootlinux_ccache_max_size }}.0 GiB + +# Performance settings +compression = {{ bootlinux_ccache_compression | lower }} +stats = true + +# Build reproducibility - ensure deterministic compilation +compiler_check = mtime +direct_mode = true +hash_dir = true + +# Debugging (disabled by default for performance) +debug = false + +# File handling +file_clone = false +hard_link = false +inode_cache = true + +# Security +umask = +read_only = false diff --git a/workflows/linux/Kconfig b/workflows/linux/Kconfig index 897cb00f..cbfa8a3c 100644 --- a/workflows/linux/Kconfig +++ b/workflows/linux/Kconfig @@ -188,6 +188,84 @@ config BOOTLINUX_REPRODUCIBLE_BUILDS Recommended for CI/CD environments and when build reproducibility is required. Compatible with both GCC and Clang toolchains. +menu "ccache configuration" + +config BOOTLINUX_CCACHE + bool "Enable ccache for faster kernel builds" + output yaml + default n + help + Enable ccache (compiler cache) to accelerate repeated kernel + builds by caching compilation results. Particularly effective + for incremental builds and CI/CD environments. + + When enabled with Clang (LLVM=1), uses CC="ccache clang". + When enabled with GCC, uses CC="ccache gcc". + + Requires ccache package to be installed on build nodes. + +choice + prompt "ccache configuration mode" + depends on BOOTLINUX_CCACHE + default BOOTLINUX_CCACHE_SYSTEM_WIDE + help + Choose how ccache configuration is managed. + +config BOOTLINUX_CCACHE_SYSTEM_WIDE + bool "Use system-wide ccache configuration" + output yaml + help + Use the system's existing ccache configuration + (~/.cache/ccache or system defaults). Allows manual + tuning via "ccache --set-config" commands. + +config BOOTLINUX_CCACHE_KDEVOPS_MANAGED + bool "kdevops-managed ccache configuration" + output yaml + help + kdevops generates and manages a project-specific ccache + configuration file using TOPDIR_PATH. This ensures + isolation from system-wide ccache settings and allows + project-specific tuning. + + Configuration location: $(TOPDIR_PATH)/.ccache/ccache.conf + +endchoice + +if BOOTLINUX_CCACHE_KDEVOPS_MANAGED + +config BOOTLINUX_CCACHE_MAX_SIZE + int "Maximum cache size (GiB)" + output yaml + default 10 + help + Maximum size for the ccache in GiB. Common values: + - 5 (default ccache setting) + - 10 (recommended for kernel development) + - 20 (large projects/multiple kernels) + - 50 (high-performance development environments) + +config BOOTLINUX_CCACHE_DIR + string "Cache directory" + output yaml + default "$(TOPDIR_PATH)/.ccache/cache" + help + Directory where ccache stores cached compilation results. + Using TOPDIR_PATH ensures project isolation. + +config BOOTLINUX_CCACHE_COMPRESSION + bool "Enable compression" + output yaml + default y + help + Enable compression of cached files to save disk space. + Slightly increases CPU usage but significantly reduces + storage requirements. + +endif # BOOTLINUX_CCACHE_KDEVOPS_MANAGED + +endmenu + choice prompt "Type of development version of Linux to use" default BOOTLINUX_LINUS if !BOOTLINUX_TREE_SET_BY_CLI && !BOOTLINUX_TREE_REF_SET_BY_CLI -- 2.50.1