Re: [PATCH v2 02/10] scripts: add Lambda Labs Python API library

[Luis Chamberlain <mcgrof@kernel.org>]

On Thu, Aug 28, 2025 at 02:59:18PM -0400, Chuck Lever wrote:
> On 8/27/25 5:28 PM, Luis Chamberlain wrote:
> > Add a Python library for interacting with the Lambda Labs cloud API.
> > This provides core functionality to query instance types, regions,
> > availability, and manage cloud resources programmatically.
> > 
> > The API library handles:
> > - Instance type enumeration and capacity checking
> > - Region availability queries
> > - SSH key management operations
> > - Error handling and retries for API calls
> > - Parsing and normalizing API responses
> > 
> > This forms the foundation for dynamic Kconfig generation and terraform
> > integration, but doesn't enable any features yet.
> 
> Thanks for splitting this up. Much much easier for humble humans to
> understand now. There are some really interesting ideas in the series;
> kudos to you and Claude.
> 
> The other cloud providers have their own house-brewed command line
> utilities that are typically packaged by distributions (eg, oci, aws,
> and so on), as well as their own Ansible collections.
> 
> I assume that Lambda does not have those (yet).

Right. It sad to not see such a thing, which meant it was actually
harder to do this inference work, and yet it shows its possible. Which
also means, technically in theory, it might even be possible to do the
same with other cloud providers without a CLI tool.

> Can the patch
> description mention that? 

Sure.

> Basically that's the whole purpose for this
> patch and the two following, IIUC.
> 
> I would actually encourage this script to be restructured and renamed so
> that it works like the tooling available from other cloud providers.

So you mean we write Lambda's CLI tool? Sure we can try that, but who do
we follow? And do we need to do that now? Or can I do that later?

> Separate the API calls and retrieval of information (which is rather
> general purpose) from the translation of that information to Kconfig
> menus (which is specific to kdevops).

You mean, to re-use all the knowledge we used for Kconfig dynamic
kconfig to a create a specific tool, which can then be itself leveraged
for both the dynamic kconfig and also a user tool?

> General comment: it might be cleaner and more extensible if the Python
> used Jinja2 templating to generate the Kconfig files.

I *love* Jinja2, even though it originally was an April Fool's joke,
however, my experience with it is its extremely limitting since the
amount of dynamic programability is limited, and also extremely
difficult to read. See playbooks/roles/gen_nodes/templates/gen_drives.j2
as an example.

So I am not sure, I think for cloud with an explosion of varaibility,
it would be super complicatd to manage with jinja2.

> One more remark, far below:
>
> > --- /dev/null
> > +++ b/scripts/lambdalabs_api.py
> > +# NOTE: The Lambda Labs terraform provider (elct9620/lambdalabs v0.3.0) does NOT support
> > +# OS image selection. Lambda Labs automatically deploys Ubuntu 22.04 LTS by default.
> > +#
> > +# The provider only supports these attributes for instances:
> > +# - name (instance name)
> > +# - region_name (deployment region)
> > +# - instance_type_name (GPU type)
> > +# - ssh_key_names (SSH keys)
> > +#
> > +# What's NOT supported:
> > +# - OS/distribution selection
> > +# - Custom user creation
> > +# - User data/cloud-init scripts
> > +# - Storage configuration
> > +#
> > +# SSH User: Always "ubuntu" (the OS default user)
> > +#
> > +# This file is kept as a placeholder for future provider updates.
> 
> IMHO this comment is important for kdevops users to be aware of too.
> Maybe it should go in a top-level Kconfig help section instead of being
> an internal comment.

Sure.

> The storage configuration bit is most interesting to me... since that
> means the volumes submodule is not coming over to the Lamba terraform
> in this iteration. That's OK, that's not what you're testing with this
> one.

Yup, I reached out to them, and they gave me this:

https://cloud.lambda.ai/api/v1/docs#post-/api/v1/filesystems

So it actualy seems that's our outlet, however unfortunately we can't
create our own filesystem, this is just NFS behind the scenes.

So -- at least we can *create* new "volumes".

  Luis