From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 239BA21B9C8 for ; Sun, 31 Aug 2025 04:00:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756612808; cv=none; b=QXy6s6J7r2VL+ZIydD3deGcAc1drhqA5Xt35ZHyURwRL7wqKJMy4CRnXhxzyw8WfCWOFem05BtOEZ+SWKz62opD/P3y7BWNduEQDTg/U4nEJmBk0peUqiWBj5Xk9POVsDxEGqux9ybTbwsYB/BX0jZqCqqN3Thw55mKwnjVgr04= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756612808; c=relaxed/simple; bh=eXXQ6Esol1h9R62PZtylgxHWehGvOEQ8MFjkYS6xOMU=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=rZ7AJSaOaSnJxTvuxYIrNObeDD1mr8XDYzHxZV0Jhrq0w9v/mQzRb1ONNXaYQWynN5WQKbQslyUkZHvU+QqnNpZ1blCOQlAiR0USbvNFNvQncRdmHCxCCgPvgvn87oRQjPK/Sh2DNoH7Us+peepEFXvzJItVGTU2Hk9kYdzPoxc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=3obM68H1; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="3obM68H1" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-Type: Content-ID:Content-Description:In-Reply-To:References; bh=Yjg8KRWyhkXiVxZdikeD3uQqFD28gVT3HVNq3A4Ttcc=; b=3obM68H12/XiBKxc6l25BxPsHT /iqR+KnvQ0O8lcz+aJl281jA1fbPVcnDD3W04Zvd54ikLY+o2gacuu8zRJMxJ2B2+MwswUkbCPx2B 0qxAXnJEYWQhkMsG1PybesqvBOanTdAEEOCA1z1OH9MZqZB3sVugvYyCLWNe3zpRj7+Qbj9P9L4FP hTALuQGC1mbcVOcM1o7BnRZ/trEOU0VfzuIokAgW1q0tryQacP0VfqxotEHahrEoCDng5v0LPx8Ex pTM+4Vzy1Sc+waz9SVEE5ZZPHU1jJk+oAPc+DI6PzmMONy2GnnbLz3BNqa8ioPi9XuZuIamk7TuOH LSjUmiHA==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98.2 #2 (Red Hat Linux)) id 1usZEb-000000093rY-2LyG; Sun, 31 Aug 2025 04:00:05 +0000 From: Luis Chamberlain To: Chuck Lever , Daniel Gomez , kdevops@lists.linux.dev Cc: Luis Chamberlain Subject: [PATCH v3 00/10] terraform: add Lambda Labs cloud provider support Date: Sat, 30 Aug 2025 20:59:54 -0700 Message-ID: <20250831040004.2159779-1-mcgrof@kernel.org> X-Mailer: git-send-email 2.49.0 Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: Luis Chamberlain This v3 takes up the idea shared by Chuck to implement a CLI tool and make the dynamic kconfig use it instead. We also add documentation about this. This should hopefully pave the way for other cloud provider support to leverage this more easily, AIs can just read the docs and go to town. Luis Chamberlain (10): gitignore: add entries for Lambda Labs dynamic configuration scripts: add Lambda Labs Python API library scripts: add Lambda Labs testing and debugging utilities scripts: add Lambda Labs credentials management scripts: add Lambda Labs SSH key management utilities kconfig: add dynamic cloud provider configuration infrastructure terraform/lambdalabs: add Kconfig structure for Lambda Labs terraform/lambdalabs: add terraform provider implementation ansible/terraform: integrate Lambda Labs into build system kconfigs: enable Lambda Labs cloud provider in menus .gitignore | 9 + PROMPTS.md | 56 ++ defconfigs/lambdalabs | 15 + defconfigs/lambdalabs-gpu-1x-a10 | 9 + defconfigs/lambdalabs-gpu-1x-a100 | 8 + defconfigs/lambdalabs-gpu-1x-h100 | 8 + defconfigs/lambdalabs-gpu-8x-a100 | 8 + defconfigs/lambdalabs-gpu-8x-h100 | 8 + defconfigs/lambdalabs-shared-key | 11 + defconfigs/lambdalabs-smart | 10 + docs/dynamic-cloud-kconfig.md | 461 +++++++++++++ docs/lambda-cli.1 | 245 +++++++ kconfigs/Kconfig.bringup | 5 + playbooks/roles/gen_tfvars/defaults/main.yml | 23 + .../templates/lambdalabs/terraform.tfvars.j2 | 18 + playbooks/roles/terraform/tasks/main.yml | 85 +++ scripts/cloud_list_all.sh | 152 +++++ scripts/dynamic-cloud-kconfig.Makefile | 44 ++ scripts/dynamic-kconfig.Makefile | 2 + scripts/generate_cloud_configs.py | 113 ++++ scripts/lambda-cli | 639 ++++++++++++++++++ scripts/lambdalabs_api.py | 556 +++++++++++++++ scripts/lambdalabs_credentials.py | 242 +++++++ scripts/lambdalabs_infer_region.py | 61 ++ scripts/lambdalabs_smart_inference.py | 62 ++ scripts/lambdalabs_ssh_key_name.py | 135 ++++ scripts/lambdalabs_ssh_keys.py | 358 ++++++++++ scripts/ssh_config_file_name.py | 79 +++ scripts/terraform.Makefile | 108 ++- scripts/update_ssh_config_lambdalabs.py | 110 +++ terraform/Kconfig.providers | 10 + terraform/Kconfig.ssh | 37 +- terraform/lambdalabs/Kconfig | 33 + terraform/lambdalabs/README.md | 349 ++++++++++ terraform/lambdalabs/SET_API_KEY.sh | 20 + .../lambdalabs/ansible_provision_cmd.tpl | 1 + terraform/lambdalabs/extract_api_key.py | 40 ++ terraform/lambdalabs/kconfigs/Kconfig.compute | 48 ++ .../lambdalabs/kconfigs/Kconfig.identity | 76 +++ .../lambdalabs/kconfigs/Kconfig.location | 73 ++ terraform/lambdalabs/kconfigs/Kconfig.smart | 25 + terraform/lambdalabs/kconfigs/Kconfig.storage | 12 + terraform/lambdalabs/main.tf | 154 +++++ terraform/lambdalabs/output.tf | 51 ++ terraform/lambdalabs/provider.tf | 19 + terraform/lambdalabs/shared.tf | 1 + terraform/lambdalabs/vars.tf | 65 ++ terraform/shared.tf | 14 +- 48 files changed, 4656 insertions(+), 12 deletions(-) create mode 100644 defconfigs/lambdalabs create mode 100644 defconfigs/lambdalabs-gpu-1x-a10 create mode 100644 defconfigs/lambdalabs-gpu-1x-a100 create mode 100644 defconfigs/lambdalabs-gpu-1x-h100 create mode 100644 defconfigs/lambdalabs-gpu-8x-a100 create mode 100644 defconfigs/lambdalabs-gpu-8x-h100 create mode 100644 defconfigs/lambdalabs-shared-key create mode 100644 defconfigs/lambdalabs-smart create mode 100644 docs/dynamic-cloud-kconfig.md create mode 100644 docs/lambda-cli.1 create mode 100644 playbooks/roles/gen_tfvars/templates/lambdalabs/terraform.tfvars.j2 create mode 100755 scripts/cloud_list_all.sh create mode 100644 scripts/dynamic-cloud-kconfig.Makefile create mode 100755 scripts/generate_cloud_configs.py create mode 100755 scripts/lambda-cli create mode 100755 scripts/lambdalabs_api.py create mode 100755 scripts/lambdalabs_credentials.py create mode 100755 scripts/lambdalabs_infer_region.py create mode 100755 scripts/lambdalabs_smart_inference.py create mode 100755 scripts/lambdalabs_ssh_key_name.py create mode 100755 scripts/lambdalabs_ssh_keys.py create mode 100755 scripts/ssh_config_file_name.py create mode 100755 scripts/update_ssh_config_lambdalabs.py create mode 100644 terraform/lambdalabs/Kconfig create mode 100644 terraform/lambdalabs/README.md create mode 100644 terraform/lambdalabs/SET_API_KEY.sh create mode 120000 terraform/lambdalabs/ansible_provision_cmd.tpl create mode 100755 terraform/lambdalabs/extract_api_key.py create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.compute create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.identity create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.location create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.smart create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.storage create mode 100644 terraform/lambdalabs/main.tf create mode 100644 terraform/lambdalabs/output.tf create mode 100644 terraform/lambdalabs/provider.tf create mode 120000 terraform/lambdalabs/shared.tf create mode 100644 terraform/lambdalabs/vars.tf -- 2.50.1