From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 67AEB137C31 for ; Thu, 25 Jan 2024 20:35:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706214915; cv=none; b=BYrAujsds0NqOlYvhc6Nt4tJ7QGyeMHeUJUJP5XctB4kYjom9JQygYd+jeQEQECAo1pDnnTIz3Ea8YVIuuTAHBkI9LKB+eNkC+tfGoWKvF/58npWiJOHCGxTzeKsGdeP0eaaKVUnGKG6LHb/oIaAtbnSazbLvH7SecZUAmfiQGs= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706214915; c=relaxed/simple; bh=TJzX/jyWPsLo5Z5SrcJ1wHSMzJmpb/Gzt7ETGc1r2fM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=B8no8mTWkIYikPkuLvHhbKlrXXhEswK3v6aomQLkt5jg8t0K3IFaYTJACsYQJc/vVkeu2uujJsyyxsNAQeD2zmz/7uhR/gDGM9yaH0V+9lrKKj6vIEOIAnphwttUh/se9u8gniZHgHuPEH5bROHGv+XTaa0W2LPR6iI6++m2ixY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=K86N4b1O; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="K86N4b1O" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=RlVU9Etw3NCKhPpXEtby3KtT3KZ+DVnDaMGt2pzC+pI=; b=K86N4b1Om/BJtmZR9/gjLMcCEg y804zymsix9+gGGCtygUE2BmlqHyO0Hh88kd1KKQX+w0vVLrn6MRQ28g18ShFhP8ajoR0qsfMKhcp j0V+4wvXISQclpSsYlTts8qo0EZyXXVcM0+aKLyp0YJ457LuMh6OsHvqT03gw/TEtB2GfL3bINo9F HwcnGCTyPFCMubf+/v7VI4lhYIiqJ1guW9bOC3fOaUq/M6xtM548A88Qtn0AyzyDUxulTKJBmaR4A RHkMsRBQ0RJZ64mdKqKKLsH/57137Z/SHZYDLJU1/UZRodLNxddKqpIS1k1zdvDCzNbDEysJn1zDH rNys34BA==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rT6RK-00000001nth-1t9A; Thu, 25 Jan 2024 20:35:10 +0000 From: Luis Chamberlain To: kdevops@lists.linux.dev, da.gomez@samsung.com, p.raghav@samsung.com Cc: Luis Chamberlain Subject: [PATCH 3/6] bringup: share bringup method targe and agument by two steps Date: Thu, 25 Jan 2024 12:35:04 -0800 Message-ID: <20240125203507.430113-4-mcgrof@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240125203507.430113-1-mcgrof@kernel.org> References: <20240125203507.430113-1-mcgrof@kernel.org> Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: Luis Chamberlain First we formalize now a kdevops provisioner as for one which can bring up nodes for us, can optionally setup ssh for us, and handles destroying nodes. Prior to this patch we defined provisioning as a two step process: 1) ssh update 2) devconfig playbook is run However we implicitly were issuing each provisioning script. So let us augment this first by defining the provisionin method first. The devconfig playbook target is also common, so no need to keep that separate. By sharing the devconfig playbook run as a shared bringup goal we also implicitly fix bare metal bringup by ensuring we run the devconfig playbook once at bringup. We also enhance this with a new step: KDEVOPS_BRING_UP_DEPS_EARLY This variable can be augmented by new features for so to be able to install things using the devconfig playbook before the general devconfig playbook is run. The diff stat reveals sharing is best. It will allow us later to say, for example add ktest support or xfstests-bld, if we want to in a much easier smoother way, as yet another provisioner. This should pretty much be a no-op for most deployments, but enhances bare metal setup so to ensure we run the devconfig playbook. It also allows makes it easier to add new tools which may be optional which we want installed early. Signed-off-by: Luis Chamberlain --- .gitignore | 4 +-- Makefile | 67 ++++++++++++++++++++++++++++++-------- scripts/guestfs.Makefile | 31 +++--------------- scripts/terraform.Makefile | 28 +++------------- scripts/vagrant.Makefile | 31 ++++-------------- 5 files changed, 70 insertions(+), 91 deletions(-) diff --git a/.gitignore b/.gitignore index 1d2653c5b26c..d0499ddb2348 100644 --- a/.gitignore +++ b/.gitignore @@ -3,6 +3,7 @@ *.retry .kdevops\.depcheck +.provisioned_once* guestfs/ @@ -29,11 +30,8 @@ scripts/workflows/lib/__pycache__/ vagrant/kdevops_nodes.yaml vagrant/.Vagrantfile.generated vagrant/Vagrantfile -vagrant/.provisioned_once* .vagrant/ -terraform/.provisioned_once* - include/ # You can override role specific stuff on these diff --git a/Makefile b/Makefile index 8ee02f2a9b6c..be686b279e2c 100644 --- a/Makefile +++ b/Makefile @@ -85,19 +85,34 @@ endif # CONFIG_NEEDS_LOCAL_DEVELOPMENT_PATH ANSIBLE_EXTRA_ARGS += $(LOCAL_DEVELOPMENT_ARGS) -# These should be set as non-empty if you want any generic bring up -# targets to come up. We support 3 bring up methods: -# -# - vagrant: for kvm/virtualbox - will eventually be deprecated -# - libguestfs: for kvm -# - terraform: for any cloud provider -# -# If you are using bare metal, you don't do bring up, or you'd -# likely do this yourself. What you *might* need if working -# with bare metal is provisioning, but our workflows targets -# provide that. The devconfig ansible role can be also augmented -# to support many different custom provisioning preferences outside -# of the scope of workflows. With things like kdump, etc. +# Provisioning methods should set this to their target which will ensure +# systems will be up after this. +KDEVOPS_PROVISION_METHOD := + +# Provisioning methods should set this to their target which will ensure +# the systems will be removed after this +KDEVOPS_PROVISION_DESTROY_METHOD := + +# The default guard for ssh provisioning. Provisioning methods can set the +# KDEVOPS_PROVISIONED_SSH to this if they are OK with the default guard. +KDEVOPS_PROVISIONED_SSH_DEFAULT_GUARD := .provisioned_once_ssh + +# This is empty on purpose so to support terraform which does this for us +# on the terraform provider and bare metal where it is assumed you already +# have ssh setup. Later we can grow this for other bare metal setups or to +# consume / gather existing topologies. To be clear, terraform providers are +# in charge of figuring out ssh configuration updates for you in kdevops. +KDEVOPS_PROVISIONED_SSH := + +# You should augment this with targets which should be after ssh is +# ready to go to each node, and before devconfig playbook is run. +# The devconfig playbook can be used with tags to ensure deps / setup +# for early deps are run. +KDEVOPS_BRING_UP_DEPS_EARLY := + +# This is shared task. +KDEVOPS_PROVISIONED_DEVCONFIG := .provisioned_once_devconfig + KDEVOPS_BRING_UP_DEPS := KDEVOPS_DESTROY_DEPS := @@ -115,6 +130,32 @@ ifeq (y,$(CONFIG_GUESTFS)) include scripts/guestfs.Makefile endif +KDEVOPS_MRPROPER += $(KDEVOPS_PROVISIONED_SSH) +KDEVOPS_MRPROPER += $(KDEVOPS_PROVISIONED_DEVCONFIG) + +$(KDEVOPS_PROVISIONED_DEVCONFIG): + $(Q)if [[ "$(CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK)" != "" ]]; then \ + ansible-playbook $(ANSIBLE_VERBOSE) -i \ + $(KDEVOPS_HOSTFILE) $(KDEVOPS_PLAYBOOKS_DIR)/$(KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK) ;\ + fi + $(Q)touch $(KDEVOPS_PROVISIONED_DEVCONFIG) + +# Provisioning split into 4 steps: +# +# 1) Provisioning method, if one is defined +# 2) Ensuring we can use ansible with ssh, if required +# 3) Installing early dependencies, if any, which independent of any workflow +# 4) Optionally run the devconfig playbook for configuration (which may include +# extra tools) +# +# Anything deps after this is dealt with on each respective workflow. +KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_PROVISION_METHOD) +KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_PROVISIONED_SSH) +KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_BRING_UP_DEPS_EARLY) +KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_PROVISIONED_DEVCONFIG) + +KDEVOPS_DESTROY_DEPS += $(KDEVOPS_PROVISION_DESTROY_METHOD) + ifeq (y,$(CONFIG_WORKFLOWS)) include workflows/Makefile endif # CONFIG_WORKFLOWS diff --git a/scripts/guestfs.Makefile b/scripts/guestfs.Makefile index 17c8cbd354c1..054ce7a7a3ea 100644 --- a/scripts/guestfs.Makefile +++ b/scripts/guestfs.Makefile @@ -5,11 +5,7 @@ GUESTFS_ARGS := KDEVOPS_NODES_TEMPLATE := $(KDEVOPS_NODES_ROLE_TEMPLATE_DIR)/guestfs_nodes.j2 KDEVOPS_NODES := guestfs/kdevops_nodes.yaml -export KDEVOPS_GUESTFS_PROVISIONED_SSH := guestfs/.provisioned_once_ssh -export KDEVOPS_GUESTFS_PROVISIONED_DEVCONFIG := guestfs/.provisioned_once_devconfig - -KDEVOPS_MRPROPER += $(KDEVOPS_GUESTFS_PROVISIONED_SSH) -KDEVOPS_MRPROPER += $(KDEVOPS_GUESTFS_PROVISIONED_DEVCONFIG) +export KDEVOPS_PROVISIONED_SSH := $(KDEVOPS_PROVISIONED_SSH_DEFAULT_GUARD) GUESTFS_ARGS += kdevops_enable_guestfs=True GUESTFS_ARGS += guestfs_path='$(TOPDIR_PATH)/guestfs' @@ -53,18 +49,8 @@ GUESTFS_BRINGUP_DEPS := GUESTFS_BRINGUP_DEPS += $(9P_HOST_CLONE) GUESTFS_BRINGUP_DEPS += $(LIBVIRT_PCIE_PASSTHROUGH) - KDEVOPS_BRING_UP_DEPS := bringup_guestfs -KDEVOPS_DESTROY_DEPS := destroy_guestfs - -# Provisioning goes last -# -# Provisioning split into 2 steps: -# 1) Ensuring we can use ansible with ssh -# 2) Generic devconfig final configuration (which may include extra tools) -# -# Anything deps after this is dealt with on each respective workflow. -KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_GUESTFS_PROVISIONED_SSH) -KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_GUESTFS_PROVISIONED_DEVCONFIG) +KDEVOPS_PROVISION_METHOD := bringup_guestfs +KDEVOPS_PROVISION_DESTROY_METHOD := destroy_guestfs 9p_linux_clone: $(Q)make linux-clone @@ -75,18 +61,11 @@ libvirt_pcie_passthrough_permissions: playbooks/libvirt_pcie_passthrough.yml \ -e 'ansible_python_interpreter=/usr/bin/python3' -$(KDEVOPS_GUESTFS_PROVISIONED_SSH): +$(KDEVOPS_PROVISIONED_SSH): $(Q)if [[ "$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE)" == "y" ]]; then \ LIBVIRT_DEFAULT_URI=$(CONFIG_LIBVIRT_URI) $(TOPDIR)/scripts/update_ssh_config_guestfs.py; \ fi - $(Q)touch $(KDEVOPS_GUESTFS_PROVISIONED_SSH) - -$(KDEVOPS_GUESTFS_PROVISIONED_DEVCONFIG): - $(Q)if [[ "$(CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK)" != "" ]]; then \ - ansible-playbook $(ANSIBLE_VERBOSE) -i \ - $(KDEVOPS_HOSTFILE) $(KDEVOPS_PLAYBOOKS_DIR)/$(KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK) ; \ - fi - $(Q)touch $(KDEVOPS_GUESTFS_PROVISIONED_DEVCONFIG) + $(Q)touch $(KDEVOPS_PROVISIONED_SSH) bringup_guestfs: $(GUESTFS_BRINGUP_DEPS) $(Q)$(TOPDIR)/scripts/bringup_guestfs.sh diff --git a/scripts/terraform.Makefile b/scripts/terraform.Makefile index 8e95e8cd822b..9c04a89d5a83 100644 --- a/scripts/terraform.Makefile +++ b/scripts/terraform.Makefile @@ -2,23 +2,10 @@ TERRAFORM_EXTRA_VARS := -export KDEVOPS_TERRAFORM_PROVISIONED_DEVCONFIG := terraform/.provisioned_once_devconfig -KDEVOPS_MRPROPER += $(KDEVOPS_TERRAFORM_PROVISIONED_DEVCONFIG) - -KDEVOPS_BRING_UP_DEPS := bringup_terraform -# Provisioning goes last -# -# Provisioning split into 2 steps: -# 1) Ensuring we can use ansible with ssh - this is implied by all -# terraform providers. That is, our terraform providers all have a last -# provisioning element task which does this update for us as part of -# the above bringup_terraform. -# 2) Generic devconfig final configuration (which may include extra tools) -# -# Anything deps after this is dealt with on each respective workflow. -KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_TERRAFORM_PROVISIONED_DEVCONFIG) - -KDEVOPS_DESTROY_DEPS := destroy_terraform +export KDEVOPS_PROVISIONED_SSH := $(KDEVOPS_PROVISIONED_SSH_DEFAULT_GUARD) + +KDEVOPS_PROVISION_METHOD := bringup_terraform +KDEVOPS_PROVISION_DESTROY_METHOD := destroy_terraform KDEVOPS_NODES_TEMPLATE := $(KDEVOPS_NODES_ROLE_TEMPLATE_DIR)/terraform_nodes.tf.j2 KDEVOPS_NODES := terraform/nodes.tf @@ -179,13 +166,6 @@ ANSIBLE_EXTRA_ARGS += $(TERRAFORM_EXTRA_VARS) bringup_terraform: $(Q)$(TOPDIR)/scripts/bringup_terraform.sh -$(KDEVOPS_TERRAFORM_PROVISIONED_DEVCONFIG): - $(Q)if [[ "$(CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK)" != "" ]]; then \ - ansible-playbook $(ANSIBLE_VERBOSE) -i \ - $(KDEVOPS_HOSTFILE) $(KDEVOPS_PLAYBOOKS_DIR)/$(KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK) ;\ - fi - $(Q)touch $(KDEVOPS_TERRAFORM_PROVISIONED_DEVCONFIG) - destroy_terraform: $(Q)$(TOPDIR)/scripts/destroy_terraform.sh diff --git a/scripts/vagrant.Makefile b/scripts/vagrant.Makefile index 825cfd210c6b..85e1c57356de 100644 --- a/scripts/vagrant.Makefile +++ b/scripts/vagrant.Makefile @@ -8,13 +8,11 @@ KDEVOPS_NODES := vagrant/kdevops_nodes.yaml KDEVOPS_VAGRANT_TEMPLATE := $(KDEVOPS_NODES_ROLE_TEMPLATE_DIR)/Vagrantfile.j2 KDEVOPS_VAGRANT_GENERATED:= vagrant/.Vagrantfile.generated KDEVOPS_VAGRANT := vagrant/Vagrantfile -export KDEVOPS_VAGRANT_PROVISIONED_SSH := vagrant/.provisioned_once_ssh -export KDEVOPS_VAGRANT_PROVISIONED_DEVCONFIG := vagrant/.provisioned_once_devconfig + +export KDEVOPS_PROVISIONED_SSH := $(KDEVOPS_PROVISIONED_SSH_DEFAULT_GUARD) KDEVOPS_MRPROPER += $(KDEVOPS_VAGRANT_GENERATED) KDEVOPS_MRPROPER += $(KDEVOPS_VAGRANT) -KDEVOPS_MRPROPER += $(KDEVOPS_VAGRANT_PROVISIONED_SSH) -KDEVOPS_MRPROPER += $(KDEVOPS_VAGRANT_PROVISIONED_DEVCONFIG) VAGRANT_ARGS += kdevops_vagrant_template_full_path='$(TOPDIR_PATH)/$(KDEVOPS_VAGRANT_TEMPLATE)' @@ -75,18 +73,8 @@ VAGRANT_BRINGUP_DEPS += $(VAGRANT_PRIVATE_BOX_DEPS) VAGRANT_BRINGUP_DEPS += $(VAGRANT_9P_HOST_CLONE) VAGRANT_BRINGUP_DEPS += $(LIBVIRT_PCIE_PASSTHROUGH) -KDEVOPS_BRING_UP_DEPS := bringup_vagrant -# Provisioning goes last -# -# Provisioning split into 2 steps: -# 1) Ensuring we can use ansible with ssh -# 2) Generic devconfig final configuration (which may include extra tools) -# -# Anything deps after this is dealt with on each respective workflow. -KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_VAGRANT_PROVISIONED_SSH) -KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_VAGRANT_PROVISIONED_DEVCONFIG) - -KDEVOPS_DESTROY_DEPS := destroy_vagrant +KDEVOPS_PROVISION_METHOD := bringup_vagrant +KDEVOPS_PROVISION_DESTROY_METHOD := destroy_vagrant extend-extra-args-vagrant: @if [[ "$(CONFIG_HAVE_VAGRANT_BOX_URL)" == "y" ]]; then \ @@ -108,21 +96,14 @@ libvirt_pcie_passthrough_permissions: playbooks/libvirt_pcie_passthrough.yml \ -e 'ansible_python_interpreter=/usr/bin/python3' -$(KDEVOPS_VAGRANT_PROVISIONED_SSH): +$(KDEVOPS_PROVISIONED_SSH): $(Q)if [[ "$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE)" == "y" ]]; then \ ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \ --inventory localhost, \ playbooks/update_ssh_config_vagrant.yml \ -e 'ansible_python_interpreter=/usr/bin/python3' ;\ fi - $(Q)touch $(KDEVOPS_VAGRANT_PROVISIONED_SSH) - -$(KDEVOPS_VAGRANT_PROVISIONED_DEVCONFIG): - $(Q)if [[ "$(CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK)" != "" ]]; then \ - ansible-playbook $(ANSIBLE_VERBOSE) -i \ - $(KDEVOPS_HOSTFILE) $(KDEVOPS_PLAYBOOKS_DIR)/$(KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK) ;\ - fi - $(Q)touch $(KDEVOPS_VAGRANT_PROVISIONED_DEVCONFIG) + $(Q)touch $(KDEVOPS_PROVISIONED_SSH) bringup_vagrant: $(VAGRANT_BRINGUP_DEPS) $(Q)$(TOPDIR)/scripts/bringup_vagrant.sh -- 2.42.0