From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A670630C62E for ; Wed, 27 Aug 2025 21:29:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756330146; cv=none; b=XnSbj7JdsIXBZRzeGKbrqpr48srJKR/rlaxa/v309L4NGC/UpzQ601TuL6znJZbFU39ftynGV75PHaeugNgMkoJvCKVyXb7Q9OJOU9ERSWzeMJ6aCGZXIaRjCYFMrTPiry9BtIbQJJy6hqfNDYlWvHiAqnRRmKbeRQ9z29oIGnk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756330146; c=relaxed/simple; bh=yFuZMEpU6IOnNAzhCoOIeShzur2lfMVd+kID+H9wf0w=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=OR11y29HQ9IlXDdHp5j/V48fv0ZZ+ygQMotTmY92X+0Mn2p1ToKqCj/JHPMbRGwaEFFsYBe/eV4g5elFQrX6I9ee5FIbGOuWSR4Pd4zRZmOVM9bPqfPOaDeRUfWHPfp+HD6Em0hunoTiuj29V2z+xgK35FwG9Zsm1ad96f8P7/w= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=pMzNrPGo; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="pMzNrPGo" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-Type: Content-ID:Content-Description:In-Reply-To:References; bh=HxTOYFC8RWp9+uZz4jqQb6vaU7Mx7hraiHH10l+V47k=; b=pMzNrPGoEiFDmMEiQpufmzeyYL EKMtjZj1TJXdF+qcv5VvIoJXbi9T21MOVXTnxTcFOvNElJgmHxKz6b/RWFNXe4zVoISiNf0hGYFKx WOeg8zZFFjCEYpNGzxBPEQeTnWAoMJQXo4Bx/4mwmggO1St7EsGSvDsP5vJmDIsutysA2kXUB+qdz roo6Irhs+F/59NdHD691xAYRgUGc4IIK9OqBdUEJ9e/aLaiMLw68osAwQhvytaRDgthMFnfPFhyW0 +FmSQQ7n24+PNlpO8WpphNh1BPH4hx3uNz76ii2yrdOXSTDbalUQ/0EmQt+lj3bKYanFL25GVED/N iZNiHoZg==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98.2 #2 (Red Hat Linux)) id 1urNhY-0000000GsJA-0vsI; Wed, 27 Aug 2025 21:29:04 +0000 From: Luis Chamberlain To: Chuck Lever , Daniel Gomez , kdevops@lists.linux.dev Cc: Luis Chamberlain Subject: [PATCH v2 00/10] terraform: add Lambda Labs cloud provider support with dynamic API-driven configuration Date: Wed, 27 Aug 2025 14:28:51 -0700 Message-ID: <20250827212902.4021990-1-mcgrof@kernel.org> X-Mailer: git-send-email 2.49.0 Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: Luis Chamberlain This v2 splits up the original patch [0] into parts to make it easier to review. [0] https://lkml.kernel.org/r/20250827101648.3581048-1-mcgrof@kernel.org Luis Chamberlain (10): gitignore: add entries for Lambda Labs dynamic configuration scripts: add Lambda Labs Python API library scripts: add Lambda Labs credentials management scripts: add Lambda Labs SSH key management utilities kconfig: add dynamic cloud provider configuration infrastructure terraform/lambdalabs: add Kconfig structure for Lambda Labs terraform/lambdalabs: add terraform provider implementation ansible/terraform: integrate Lambda Labs into build system scripts: add Lambda Labs testing and debugging utilities terraform: enable Lambda Labs cloud provider in menus .gitignore | 9 + PROMPTS.md | 56 ++ defconfigs/lambdalabs | 15 + defconfigs/lambdalabs-gpu-1x-a10 | 9 + defconfigs/lambdalabs-gpu-1x-a100 | 8 + defconfigs/lambdalabs-gpu-1x-h100 | 8 + defconfigs/lambdalabs-gpu-8x-a100 | 8 + defconfigs/lambdalabs-gpu-8x-h100 | 8 + defconfigs/lambdalabs-shared-key | 11 + defconfigs/lambdalabs-smart | 10 + kconfigs/Kconfig.bringup | 5 + playbooks/roles/gen_tfvars/defaults/main.yml | 23 + .../templates/lambdalabs/terraform.tfvars.j2 | 18 + playbooks/roles/terraform/tasks/main.yml | 71 +++ scripts/check_lambdalabs_capacity.py | 172 ++++++ scripts/cloud_list_all.sh | 151 +++++ scripts/debug_lambdalabs_api.sh | 87 +++ scripts/dynamic-cloud-kconfig.Makefile | 44 ++ scripts/dynamic-kconfig.Makefile | 2 + scripts/explore_lambda_api.py | 48 ++ scripts/generate_cloud_configs.py | 223 ++++++++ scripts/lambdalabs_api.py | 538 ++++++++++++++++++ scripts/lambdalabs_credentials.py | 242 ++++++++ scripts/lambdalabs_infer_cheapest.py | 107 ++++ scripts/lambdalabs_infer_region.py | 36 ++ scripts/lambdalabs_list_instances.py | 167 ++++++ scripts/lambdalabs_smart_inference.py | 196 +++++++ scripts/lambdalabs_ssh_key_name.py | 135 +++++ scripts/lambdalabs_ssh_keys.py | 358 ++++++++++++ scripts/ssh_config_file_name.py | 79 +++ scripts/terraform.Makefile | 108 +++- scripts/terraform_list_instances.sh | 79 +++ scripts/test_lambda_ssh.py | 111 ++++ scripts/test_lambdalabs_credentials.py | 50 ++ scripts/test_ssh_keys.py | 97 ++++ scripts/update_lambdalabs_instance.sh | 29 + scripts/update_ssh_config_lambdalabs.py | 145 +++++ scripts/upload_ssh_key_to_lambdalabs.py | 176 ++++++ terraform/Kconfig.providers | 10 + terraform/Kconfig.ssh | 37 +- terraform/lambdalabs/Kconfig | 33 ++ terraform/lambdalabs/README.md | 295 ++++++++++ terraform/lambdalabs/SET_API_KEY.sh | 20 + .../lambdalabs/ansible_provision_cmd.tpl | 1 + terraform/lambdalabs/extract_api_key.py | 40 ++ terraform/lambdalabs/kconfigs/Kconfig.compute | 34 ++ .../lambdalabs/kconfigs/Kconfig.identity | 76 +++ .../lambdalabs/kconfigs/Kconfig.location | 89 +++ .../kconfigs/Kconfig.location.manual | 57 ++ terraform/lambdalabs/kconfigs/Kconfig.smart | 25 + terraform/lambdalabs/kconfigs/Kconfig.storage | 12 + terraform/lambdalabs/main.tf | 154 +++++ terraform/lambdalabs/output.tf | 51 ++ terraform/lambdalabs/provider.tf | 19 + terraform/lambdalabs/shared.tf | 1 + terraform/lambdalabs/vars.tf | 65 +++ terraform/shared.tf | 14 +- 57 files changed, 4660 insertions(+), 12 deletions(-) create mode 100644 defconfigs/lambdalabs create mode 100644 defconfigs/lambdalabs-gpu-1x-a10 create mode 100644 defconfigs/lambdalabs-gpu-1x-a100 create mode 100644 defconfigs/lambdalabs-gpu-1x-h100 create mode 100644 defconfigs/lambdalabs-gpu-8x-a100 create mode 100644 defconfigs/lambdalabs-gpu-8x-h100 create mode 100644 defconfigs/lambdalabs-shared-key create mode 100644 defconfigs/lambdalabs-smart create mode 100644 playbooks/roles/gen_tfvars/templates/lambdalabs/terraform.tfvars.j2 create mode 100755 scripts/check_lambdalabs_capacity.py create mode 100755 scripts/cloud_list_all.sh create mode 100755 scripts/debug_lambdalabs_api.sh create mode 100644 scripts/dynamic-cloud-kconfig.Makefile create mode 100644 scripts/explore_lambda_api.py create mode 100755 scripts/generate_cloud_configs.py create mode 100755 scripts/lambdalabs_api.py create mode 100755 scripts/lambdalabs_credentials.py create mode 100755 scripts/lambdalabs_infer_cheapest.py create mode 100755 scripts/lambdalabs_infer_region.py create mode 100755 scripts/lambdalabs_list_instances.py create mode 100755 scripts/lambdalabs_smart_inference.py create mode 100755 scripts/lambdalabs_ssh_key_name.py create mode 100755 scripts/lambdalabs_ssh_keys.py create mode 100755 scripts/ssh_config_file_name.py create mode 100755 scripts/terraform_list_instances.sh create mode 100644 scripts/test_lambda_ssh.py create mode 100755 scripts/test_lambdalabs_credentials.py create mode 100644 scripts/test_ssh_keys.py create mode 100755 scripts/update_lambdalabs_instance.sh create mode 100755 scripts/update_ssh_config_lambdalabs.py create mode 100755 scripts/upload_ssh_key_to_lambdalabs.py create mode 100644 terraform/lambdalabs/Kconfig create mode 100644 terraform/lambdalabs/README.md create mode 100644 terraform/lambdalabs/SET_API_KEY.sh create mode 120000 terraform/lambdalabs/ansible_provision_cmd.tpl create mode 100755 terraform/lambdalabs/extract_api_key.py create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.compute create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.identity create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.location create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.location.manual create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.smart create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.storage create mode 100644 terraform/lambdalabs/main.tf create mode 100644 terraform/lambdalabs/output.tf create mode 100644 terraform/lambdalabs/provider.tf create mode 120000 terraform/lambdalabs/shared.tf create mode 100644 terraform/lambdalabs/vars.tf -- 2.50.1