Re: [PATCH v2 0/4] vLLM and the vLLM production stack

[Chuck Lever <cel@kernel.org>]

On 10/4/25 1:14 PM, Chuck Lever wrote:
>> But the rest of the changes are legit, please feel
>> free to cherry pick what you see useful from here:
>>
>> https://github.com/linux-kdevops/kdevops/tree/ci-testing/
>> mcgrof/20251004-cloud-bill
> Great! I will have a look at those.

Have some comments/requests on these, not sure where to post them.
Oldest to newest:

- workflows: Add vLLM workflow for LLM inference and production deployment

- vllm: Add DECLARE_HOSTS support for bare metal and existing infrastructure

- vllm: Add GPU-enabled defconfig with compatibility documentation

- defconfigs: Add composable fragments for Lambda Labs vLLM deployment

No comments on these.


- aws: prevent SSH key conflicts across multiple kdevops directories

This one was posted before, and my comment still stands: this is badly
needed IMO, but it should work for all cloud providers, not just aws,
and the new Kconfig options should go in the existing kconfig ssh menu
for terraform, probably.

Do you want me to work on adapting this one, or do you want to give
Claude another crack at it?


- Add static GPU Kconfig support for AWS

Wondering if my dynamic instance type menu already brings in these new
GPU-enabled instance types.


- Add make cloud-bill target for AWS cost tracking

Nit: I'd like to see provider-specific scripts go into

  terraform/<provider>/scripts/

I'm sorry that I had to drop the pricing information from my dynamic
menu patches. I just pushed that out of the MVP "just get it working"
patches, and I do plan to come back to it. I do follow running costs,
but not as closely as these patches suggest that you do.


- terraform/aws: use default VPC to avoid VPC limit issues

I think we can make this work, and IIRC some of the other providers
also provision default VPCs. Making it switchable (use the default,
or create one for me) makes sense. We might consider following the
precedent that OCI has set here (use an existing VPC).

There are some other resources that have similar limits.


- terraform/aws: fix EBS volume availability zone mismatch

Fair catch, but why not use the AZ that the instance is in rather
than the AZ that the subnet is in?


- terraform/aws: enable public IP assignment for instances

- terraform/aws: prefer subnets with public IP auto-assignment

As above, might need some work, but these two look do-able. Probably
should be squashed into "terraform/aws: use default VPC to avoid VPC
limit issues"


- terraform/aws: fix GPU AMI selection in terraform templates

No comment on this one. I need to first go and merge in your original
GPU AMI patches. I'd like to see that integrated into the existing AWS
Kconfig compute menu.


- ansible: map GPU instance configurations to terraform instance types

- defconfigs: fix GPU instance choice configuration

Wondering if these two are still necessary with my dynamic menu patches.


- slack-billing: add AWS cost notifications to Slack

Clever, but isn't this something that should be configured via the
cloud console? Not really sure.


- kconfig: fix Slack notification configuration syntax errors

Squash-me.


- defconfigs: add AWS P5.4xlarge GPU instance support

No comment.


HTH.


-- 
Chuck Lever