Re: [PATCH v2 02/10] scripts: add Lambda Labs Python API library

[Chuck Lever <cel@kernel.org>]

On 8/28/25 4:03 PM, Luis Chamberlain wrote:
> On Thu, Aug 28, 2025 at 04:00:09PM -0400, Chuck Lever wrote:
>> On 8/28/25 3:33 PM, Luis Chamberlain wrote:
>>> On Thu, Aug 28, 2025 at 02:59:18PM -0400, Chuck Lever wrote:
>>>> On 8/27/25 5:28 PM, Luis Chamberlain wrote:
>>>>> Add a Python library for interacting with the Lambda Labs cloud API.
>>>>> This provides core functionality to query instance types, regions,
>>>>> availability, and manage cloud resources programmatically.
>>>>>
>>>>> The API library handles:
>>>>> - Instance type enumeration and capacity checking
>>>>> - Region availability queries
>>>>> - SSH key management operations
>>>>> - Error handling and retries for API calls
>>>>> - Parsing and normalizing API responses
>>>>>
>>>>> This forms the foundation for dynamic Kconfig generation and terraform
>>>>> integration, but doesn't enable any features yet.
>>>>
>>>> Thanks for splitting this up. Much much easier for humble humans to
>>>> understand now. There are some really interesting ideas in the series;
>>>> kudos to you and Claude.
>>>>
>>>> The other cloud providers have their own house-brewed command line
>>>> utilities that are typically packaged by distributions (eg, oci, aws,
>>>> and so on), as well as their own Ansible collections.
>>>>
>>>> I assume that Lambda does not have those (yet).
>>>
>>> Right. It sad to not see such a thing, which meant it was actually
>>> harder to do this inference work, and yet it shows its possible. Which
>>> also means, technically in theory, it might even be possible to do the
>>> same with other cloud providers without a CLI tool.
>>>
>>>> Can the patch
>>>> description mention that? 
>>>
>>> Sure.
>>>
>>>> Basically that's the whole purpose for this
>>>> patch and the two following, IIUC.
>>>>
>>>> I would actually encourage this script to be restructured and renamed so
>>>> that it works like the tooling available from other cloud providers.
>>>
>>> So you mean we write Lambda's CLI tool? Sure we can try that, but who do
>>> we follow? And do we need to do that now? Or can I do that later?
>>>
>>>> Separate the API calls and retrieval of information (which is rather
>>>> general purpose) from the translation of that information to Kconfig
>>>> menus (which is specific to kdevops).
>>>
>>> You mean, to re-use all the knowledge we used for Kconfig dynamic
>>> kconfig to a create a specific tool, which can then be itself leveraged
>>> for both the dynamic kconfig and also a user tool?
>>
>> I think so: Write a command line tool that makes API queries and spits
>> out JSON. Write a second tool that takes the JSON and turns it into
>> Kconfig menus. Re-use the first tool for other tasks. That is how I
>> was thinking of building dynamic menus for the other providers; their
>> existing command line tools are tool #1 above.
> 
> Yeah I think this is really *it*!
> 
>> But, we can do it later. I won't hold you up with crazy ideas.
> 
> Its not crazy at all, proper architecture trumps all. But if we didn't
> start evaluating this patch series with this inference cloud thing for
> dynamic kconfig thing we wouldn't have ended up here. And so I think we
> have a solid path.
> 
> I'd like to extend your suggestion further: I am not sure if we need a
> different CLI tool per cloud provider. Wouldn't it be nice if we had
> unified way to query such things and output json for each cloud
> provider?

Basically I'm re-using the tools each provider gives us because, well,
I'm a lazy sod. It gives some immediate traction.

But I can see that doing it that way means we have a distinct JSON to
Kconfig translation for each provider.

If handling the API queries is as easy as the Python script here looks,
then the only hard part to building a single tool will be turning the
provider-specific JSON into something common (and YAML output might be
easier for Ansible and Python to deal with than JSON, IMO).


-- 
Chuck Lever