From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 67A4E137C2C for ; Thu, 25 Jan 2024 20:35:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706214915; cv=none; b=skQpcDKrVKVy4GOAUEJbgqEyT4uCJHJ2FQB/Ntv6edve0ckIeeqNz347/Tp2AXfrn3eCbX+bK/jpxeaCRSNMouNRXUUqTtTCraR+XpQoJb4VLbaop/7Jc2vcFae+vjyLnyXBvMJFc1JUqIw0ggVHcswBsjMXUdNETAyYzpgVd7k= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706214915; c=relaxed/simple; bh=EYmi//3eIun2MAXtdXRjalS9EGzPpItkzPCiLWg0h+E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Or/FRrvPB6xt/IkbWwhmOGn87bG0qsKkIxLQtTV30qsFmJA2KCwfAgCqXtizT2UQt/10AkAZkTQpEbreOLxsNXhAd3Nx+QHB79K9EHIrcD5pEUSCj6BtVR0b4Xmy+HCUx1h+6AiXV3zzggeE4ZbaKpIsTVP3ZdMLZSxqm8ddTQI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=bZth11ix; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="bZth11ix" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=QpoaIt9QJpJ1fKNdKDndBGCZJXi8mmM0x/Tfii6qpiM=; b=bZth11ixn/w1uKgi1lbEGOBnnO wZ0LUhOD+RsnFSCxrVDwPiev+k4NQD6gbIJDtNm2e3c6jKV+5+lclbYk739cb4KvIXhJnAVCxdlfJ dxSdlGqvBsQd+9IyKQH7iua7yj37pQwMi1qpZTLZeER2N6wGRZHbfmQqBXX2RjMULanmngqW3+ALD C+EeCKxlV0eiStDkXq30CV1V42T1bY8cqrbWh6b4NivFU61ZAratDrn2k0eCgaViAxCrqiiEy+fVj XFHwaceclHgRkjd1ofWxbqxGo3OWjl3ZiBhUz4gMV89bDD1VBvmrPv4A/D6WrulSyvOHD+v4Hwtwx PmgzS+LA==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rT6RK-00000001ntj-25Me; Thu, 25 Jan 2024 20:35:10 +0000 From: Luis Chamberlain To: kdevops@lists.linux.dev, da.gomez@samsung.com, p.raghav@samsung.com Cc: Luis Chamberlain Subject: [PATCH 4/6] provision: move all provisioning things to its own Makefile Date: Thu, 25 Jan 2024 12:35:05 -0800 Message-ID: <20240125203507.430113-5-mcgrof@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240125203507.430113-1-mcgrof@kernel.org> References: <20240125203507.430113-1-mcgrof@kernel.org> Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: Luis Chamberlain The provisioning aspect of kdevops is creating clutter at the top level Makefile. Move it all into its own file so to be able to easily identify related tasks and to easily extend it. We keep augmenting KDEVOPS_BRING_UP_DEPS with KDEVOPS_BRING_UP_DEPS_EARLY in the top level Makefile so we can in between add makefiles for tools which we want to include in between and make it clear these requirements will be met before KDEVOPS_PROVISIONED_DEVCONFIG, ie running the devconfig playbook for general setup. Signed-off-by: Luis Chamberlain --- Makefile | 69 +---------------------------------- scripts/provision.Makefile | 74 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+), 68 deletions(-) create mode 100644 scripts/provision.Makefile diff --git a/Makefile b/Makefile index be686b279e2c..67e13edbf6ff 100644 --- a/Makefile +++ b/Makefile @@ -85,77 +85,10 @@ endif # CONFIG_NEEDS_LOCAL_DEVELOPMENT_PATH ANSIBLE_EXTRA_ARGS += $(LOCAL_DEVELOPMENT_ARGS) -# Provisioning methods should set this to their target which will ensure -# systems will be up after this. -KDEVOPS_PROVISION_METHOD := - -# Provisioning methods should set this to their target which will ensure -# the systems will be removed after this -KDEVOPS_PROVISION_DESTROY_METHOD := - -# The default guard for ssh provisioning. Provisioning methods can set the -# KDEVOPS_PROVISIONED_SSH to this if they are OK with the default guard. -KDEVOPS_PROVISIONED_SSH_DEFAULT_GUARD := .provisioned_once_ssh - -# This is empty on purpose so to support terraform which does this for us -# on the terraform provider and bare metal where it is assumed you already -# have ssh setup. Later we can grow this for other bare metal setups or to -# consume / gather existing topologies. To be clear, terraform providers are -# in charge of figuring out ssh configuration updates for you in kdevops. -KDEVOPS_PROVISIONED_SSH := - -# You should augment this with targets which should be after ssh is -# ready to go to each node, and before devconfig playbook is run. -# The devconfig playbook can be used with tags to ensure deps / setup -# for early deps are run. -KDEVOPS_BRING_UP_DEPS_EARLY := - -# This is shared task. -KDEVOPS_PROVISIONED_DEVCONFIG := .provisioned_once_devconfig - -KDEVOPS_BRING_UP_DEPS := -KDEVOPS_DESTROY_DEPS := - -include scripts/dynamic-kconfig.Makefile - -ifeq (y,$(CONFIG_TERRAFORM)) -include scripts/terraform.Makefile -endif # CONFIG_TERRAFORM - -ifeq (y,$(CONFIG_VAGRANT)) -include scripts/vagrant.Makefile -endif - -ifeq (y,$(CONFIG_GUESTFS)) -include scripts/guestfs.Makefile -endif - -KDEVOPS_MRPROPER += $(KDEVOPS_PROVISIONED_SSH) -KDEVOPS_MRPROPER += $(KDEVOPS_PROVISIONED_DEVCONFIG) - -$(KDEVOPS_PROVISIONED_DEVCONFIG): - $(Q)if [[ "$(CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK)" != "" ]]; then \ - ansible-playbook $(ANSIBLE_VERBOSE) -i \ - $(KDEVOPS_HOSTFILE) $(KDEVOPS_PLAYBOOKS_DIR)/$(KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK) ;\ - fi - $(Q)touch $(KDEVOPS_PROVISIONED_DEVCONFIG) - -# Provisioning split into 4 steps: -# -# 1) Provisioning method, if one is defined -# 2) Ensuring we can use ansible with ssh, if required -# 3) Installing early dependencies, if any, which independent of any workflow -# 4) Optionally run the devconfig playbook for configuration (which may include -# extra tools) -# -# Anything deps after this is dealt with on each respective workflow. -KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_PROVISION_METHOD) -KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_PROVISIONED_SSH) +include scripts/provision.Makefile KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_BRING_UP_DEPS_EARLY) KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_PROVISIONED_DEVCONFIG) -KDEVOPS_DESTROY_DEPS += $(KDEVOPS_PROVISION_DESTROY_METHOD) - ifeq (y,$(CONFIG_WORKFLOWS)) include workflows/Makefile endif # CONFIG_WORKFLOWS diff --git a/scripts/provision.Makefile b/scripts/provision.Makefile new file mode 100644 index 000000000000..3767e3332e44 --- /dev/null +++ b/scripts/provision.Makefile @@ -0,0 +1,74 @@ +# SPDX-License-Identifier: copyleft-next-0.3.1 + +# Provisioning methods should set this to their target which will ensure +# systems will be up after this. +KDEVOPS_PROVISION_METHOD := + +# Provisioning methods should set this to their target which will ensure +# the systems will be removed after this +KDEVOPS_PROVISION_DESTROY_METHOD := + +# The default guard for ssh provisioning. Provisioning methods can set the +# KDEVOPS_PROVISIONED_SSH to this if they are OK with the default guard. +KDEVOPS_PROVISIONED_SSH_DEFAULT_GUARD := .provisioned_once_ssh + +# This is empty on purpose so to support terraform which does this for us +# on the terraform provider and bare metal where it is assumed you already +# have ssh setup. Later we can grow this for other bare metal setups or to +# consume / gather existing topologies. To be clear, terraform providers are +# in charge of figuring out ssh configuration updates for you in kdevops. +KDEVOPS_PROVISIONED_SSH := + +# You should augment this with targets which should be after ssh is +# ready to go to each node, and before devconfig playbook is run. +# The devconfig playbook can be used with tags to ensure deps / setup +# for early deps are run. +KDEVOPS_BRING_UP_DEPS_EARLY := + +# This is shared task. +KDEVOPS_PROVISIONED_DEVCONFIG := .provisioned_once_devconfig + +KDEVOPS_BRING_UP_DEPS := +KDEVOPS_DESTROY_DEPS := + +include scripts/dynamic-kconfig.Makefile + +ifeq (y,$(CONFIG_TERRAFORM)) +include scripts/terraform.Makefile +endif # CONFIG_TERRAFORM + +ifeq (y,$(CONFIG_VAGRANT)) +include scripts/vagrant.Makefile +endif + +ifeq (y,$(CONFIG_GUESTFS)) +include scripts/guestfs.Makefile +endif + +KDEVOPS_MRPROPER += $(KDEVOPS_PROVISIONED_SSH) +KDEVOPS_MRPROPER += $(KDEVOPS_PROVISIONED_DEVCONFIG) + +$(KDEVOPS_PROVISIONED_DEVCONFIG): + $(Q)if [[ "$(CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK)" != "" ]]; then \ + ansible-playbook $(ANSIBLE_VERBOSE) -i \ + $(KDEVOPS_HOSTFILE) $(KDEVOPS_PLAYBOOKS_DIR)/$(KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK) ;\ + fi + $(Q)touch $(KDEVOPS_PROVISIONED_DEVCONFIG) + +# Provisioning split into 4 steps: +# +# 1) Provisioning method, if one is defined +# 2) Ensuring we can use ansible with ssh, if required +# 3) Installing early dependencies, if any, which independent of any workflow +# 4) Optionally run the devconfig playbook for configuration (which may include +# extra tools) +# +# The last two are dealt with in the top level Makefile so to allow us to +# define early dependencies from the top level Makefile and make it clear +# that the devconfig playbook runs last. +# +# Anything deps after this is dealt with on each respective workflow. +KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_PROVISION_METHOD) +KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_PROVISIONED_SSH) + +KDEVOPS_DESTROY_DEPS += $(KDEVOPS_PROVISION_DESTROY_METHOD) -- 2.42.0