From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A7A06170A11 for ; Wed, 17 Sep 2025 00:34:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758069294; cv=none; b=B6BZ6D88rXqN0AqqJOjTb/XGNIzRqWmc0nEP4l4ZcTvWgELDFcCMnmZNd9U4uW1x+92YvlDO+SuVBhfZNCtCtIMrekJWxuEMTSrhafFLb1s4MRsp9hx6kmaFSQSPapK5Skuc4CR0AjClvTuy65WR1nR9HNI3FAr3KLMaEd1QE2Y= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758069294; c=relaxed/simple; bh=3ikYRMSy0p0l+vk+64+w1sLypKJTT9QsZ/Ubi4JeX+8=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=qmvMfal5fZF12ZtSIur8c/L2zW/YHDFz2YzZ/E2Xz/ucQXLzmUVgtN6yBGRAovNMf6JkIQac8e7m9IwEN3BRpSAMNJbIcISBSxy1er9yHtQ2IFptC+BqyvVbxwVh0VcoGu989hlweTzpI+AahzSYTVufBd6IVjoXAeDX+xszzIk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=g6QLRxYm; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="g6QLRxYm" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-Type: Content-ID:Content-Description:In-Reply-To:References; bh=t4YfAvVYxl50GMMTtapl+XYR2F2tQ47SSqNHCcPFQiQ=; b=g6QLRxYm5KqF85M5T4COFMTCyn w/bdshTft3gYq2YqKBT4H7kylHFTI4/VXYx2lSJbGBdX1s0tkBSeTZ6ElTfL9uOqse0rOhJtVAmw0 utxV12LuI2uFrNxo1V1wDxK+uL2r2mYTRpLLD5yaQmCHpYwPD/qDtXGOwsBCA5fy8POGv3r7gNedt 9AUNJS3YbbYuxODv/W7QR7YHjuiXAE0stDrgFnoH6Y2Zl6GSGexAQZgKfvNnzfsOtRg9pw9JHBX/3 0fOrtQcoYcExnykeoe4nsMN1i6Rxp13lg8nIy+sBmOiB/wvwDP/DRMxCFOn5Vvtuztu8PCzL//KFW 3fX1gAQg==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98.2 #2 (Red Hat Linux)) id 1uyg8J-00000009j5E-3kX2; Wed, 17 Sep 2025 00:34:51 +0000 From: Luis Chamberlain To: Chuck Lever , Daniel Gomez , kdevops@lists.linux.dev Cc: Luis Chamberlain Subject: [PATCH v4 0/8] aws: add dynamic kconfig support Date: Tue, 16 Sep 2025 17:34:41 -0700 Message-ID: <20250917003451.2318229-1-mcgrof@kernel.org> X-Mailer: git-send-email 2.51.0 Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: Luis Chamberlain Chuck, in hopes to help move forward faster, I've taken your scripts and tried to base the idea on it instead. Changes on this v4: - Adopt Chuck's script - Make AWS ssh keys unique to avoid clashes - Use a ~/.cache/ for json data - Uber optimize for speed This shaves down generation by *a lot*. If you want to test: https://github.com/linux-kdevops/kdevops/tree/mcgrof/aws-dynamic-cloud-config-v4 If you agree to it, please just merge :) Chuck Lever (1): terraform/aws: Add scripts to gather provider resource information Luis Chamberlain (7): aws: prevent SSH key conflicts across multiple kdevops directories aws: add optimized Kconfig generator using Chuck's scripts aws: integrate dynamic Kconfig generation with make targets aws: add cloud billing support with make cloud-bill aws: replace static Kconfig files with dynamically generated ones aws: add GPU instance defconfigs for AI/ML workloads docs: add documentation for dynamic cloud configuration defconfigs/aws-gpu-g5-xlarge | 36 + defconfigs/aws-gpu-p5-48xlarge | 36 + docs/cloud-dynamic-config.md | 272 ++++++ playbooks/roles/gen_tfvars/defaults/main.yml | 2 + .../templates/aws/terraform.tfvars.j2 | 3 +- scripts/aws-costs.sh | 39 + scripts/aws-parse-costs.py | 98 +++ scripts/aws_ssh_key_name.py | 165 ++++ scripts/dynamic-cloud-kconfig.Makefile | 60 +- scripts/generate_cloud_configs.py | 78 +- terraform/Kconfig.ssh | 14 +- terraform/aws/Kconfig | 5 +- terraform/aws/kconfigs/Kconfig.compute | 118 --- terraform/aws/kconfigs/Kconfig.location | 679 --------------- terraform/aws/kconfigs/Kconfig.ssh | 79 ++ .../aws/kconfigs/instance-types/Kconfig.c7a | 28 - .../aws/kconfigs/instance-types/Kconfig.i4i | 33 - .../aws/kconfigs/instance-types/Kconfig.im4gn | 25 - .../kconfigs/instance-types/Kconfig.is4gen | 25 - .../aws/kconfigs/instance-types/Kconfig.m5 | 48 -- .../aws/kconfigs/instance-types/Kconfig.m7a | 57 -- terraform/aws/scripts/aws_ami_info.py | 771 ++++++++++++++++++ terraform/aws/scripts/aws_regions_info.py | 371 +++++++++ terraform/aws/scripts/ec2_instance_info.py | 540 ++++++++++++ terraform/aws/scripts/generate_aws_kconfig.py | 474 +++++++++++ 25 files changed, 3027 insertions(+), 1029 deletions(-) create mode 100644 defconfigs/aws-gpu-g5-xlarge create mode 100644 defconfigs/aws-gpu-p5-48xlarge create mode 100644 docs/cloud-dynamic-config.md create mode 100755 scripts/aws-costs.sh create mode 100755 scripts/aws-parse-costs.py create mode 100755 scripts/aws_ssh_key_name.py delete mode 100644 terraform/aws/kconfigs/Kconfig.compute delete mode 100644 terraform/aws/kconfigs/Kconfig.location create mode 100644 terraform/aws/kconfigs/Kconfig.ssh delete mode 100644 terraform/aws/kconfigs/instance-types/Kconfig.c7a delete mode 100644 terraform/aws/kconfigs/instance-types/Kconfig.i4i delete mode 100644 terraform/aws/kconfigs/instance-types/Kconfig.im4gn delete mode 100644 terraform/aws/kconfigs/instance-types/Kconfig.is4gen delete mode 100644 terraform/aws/kconfigs/instance-types/Kconfig.m5 delete mode 100644 terraform/aws/kconfigs/instance-types/Kconfig.m7a create mode 100755 terraform/aws/scripts/aws_ami_info.py create mode 100755 terraform/aws/scripts/aws_regions_info.py create mode 100755 terraform/aws/scripts/ec2_instance_info.py create mode 100755 terraform/aws/scripts/generate_aws_kconfig.py -- 2.51.0