From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB0D12D05D for ; Wed, 8 Oct 2025 17:46:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759945579; cv=none; b=s/SlTdujju+GuN6BeEAtro+EpDlA6kn325waO8MmPw58WcVmW6NwgzHPoyCGqK3TtnI4WPPTuceTxIPC19EuIzN/b454WJzHHo7ZDre6W94XTONqgCusCaHAOqzYP3nOZ8NDW1isUbBYOQK1RnIVaOqSiIaPdEkH/aUeAjns3VM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1759945579; c=relaxed/simple; bh=rm310+arp3qHmmq/CGVvredejSw397g68ynTXCiw6Vw=; h=Message-ID:Date:MIME-Version:Subject:From:To:Cc:References: In-Reply-To:Content-Type; b=eU0fDIZyxOcRsVvnHFFrPNG/QIGiZP9o4XlaViNWFGVHe7BPu/I6BXzYXp40u33J+tnCSbO49XA6vzCtOwU7LUIPkSUPlijBwG9BOBTV24Dzbi7riDOVdSzXWYCZ1BQIGgJxkf4Cg6lfAeMgjbDQSe3DGhUw0OzPC8NnTsaFMKw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kpx/ibao; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kpx/ibao" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B4180C4CEE7; Wed, 8 Oct 2025 17:46:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1759945579; bh=rm310+arp3qHmmq/CGVvredejSw397g68ynTXCiw6Vw=; h=Date:Subject:From:To:Cc:References:In-Reply-To:From; b=kpx/ibaoV85zvheuyI7TtRf7+3fzgD9TaXd4XWvOTh0HnRTbzISQSacU/Cplb5CeG Pm0qtNZkcE7FRivZkjhVpibGYfxT5XXDz4gPo2IYTdUwMrkjYmgT6G/G2lfr7sOZm3 TNn25GPEZzZ3cvnG2uu8Ivz8etd2lztmdfPyZyeO6fT4kG5CSy7Ot4RVBK6M54GXMJ ObaOt9DMOpDsE5MKSK7kWk4Y5uVORY+KjK1+jn+fel7/GhqZntJNtvNm+i+G+vAx1P 9V9oLlPnvVIedQTX9UEJzWKdcdArcLCZPU/SuRxfPj++bSUJvt+YpaJPDfUQXcQ1NH 7u1GDnSjZpsXg== Message-ID: Date: Wed, 8 Oct 2025 13:46:15 -0400 Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 0/4] vLLM and the vLLM production stack From: Chuck Lever To: Luis Chamberlain Cc: Daniel Gomez , kdevops@lists.linux.dev, Devasena Inupakutika , DongjooSeo , Joel Fernandes References: <20251004163816.3303237-1-mcgrof@kernel.org> <9821a951-e5a1-4e24-868f-f1a874509d5b@kernel.org> Content-Language: en-US Organization: kernel.org In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 10/4/25 1:14 PM, Chuck Lever wrote: >> But the rest of the changes are legit, please feel >> free to cherry pick what you see useful from here: >> >> https://github.com/linux-kdevops/kdevops/tree/ci-testing/ >> mcgrof/20251004-cloud-bill > Great! I will have a look at those. Have some comments/requests on these, not sure where to post them. Oldest to newest: - workflows: Add vLLM workflow for LLM inference and production deployment - vllm: Add DECLARE_HOSTS support for bare metal and existing infrastructure - vllm: Add GPU-enabled defconfig with compatibility documentation - defconfigs: Add composable fragments for Lambda Labs vLLM deployment No comments on these. - aws: prevent SSH key conflicts across multiple kdevops directories This one was posted before, and my comment still stands: this is badly needed IMO, but it should work for all cloud providers, not just aws, and the new Kconfig options should go in the existing kconfig ssh menu for terraform, probably. Do you want me to work on adapting this one, or do you want to give Claude another crack at it? - Add static GPU Kconfig support for AWS Wondering if my dynamic instance type menu already brings in these new GPU-enabled instance types. - Add make cloud-bill target for AWS cost tracking Nit: I'd like to see provider-specific scripts go into terraform//scripts/ I'm sorry that I had to drop the pricing information from my dynamic menu patches. I just pushed that out of the MVP "just get it working" patches, and I do plan to come back to it. I do follow running costs, but not as closely as these patches suggest that you do. - terraform/aws: use default VPC to avoid VPC limit issues I think we can make this work, and IIRC some of the other providers also provision default VPCs. Making it switchable (use the default, or create one for me) makes sense. We might consider following the precedent that OCI has set here (use an existing VPC). There are some other resources that have similar limits. - terraform/aws: fix EBS volume availability zone mismatch Fair catch, but why not use the AZ that the instance is in rather than the AZ that the subnet is in? - terraform/aws: enable public IP assignment for instances - terraform/aws: prefer subnets with public IP auto-assignment As above, might need some work, but these two look do-able. Probably should be squashed into "terraform/aws: use default VPC to avoid VPC limit issues" - terraform/aws: fix GPU AMI selection in terraform templates No comment on this one. I need to first go and merge in your original GPU AMI patches. I'd like to see that integrated into the existing AWS Kconfig compute menu. - ansible: map GPU instance configurations to terraform instance types - defconfigs: fix GPU instance choice configuration Wondering if these two are still necessary with my dynamic menu patches. - slack-billing: add AWS cost notifications to Slack Clever, but isn't this something that should be configured via the cloud console? Not really sure. - kconfig: fix Slack notification configuration syntax errors Squash-me. - defconfigs: add AWS P5.4xlarge GPU instance support No comment. HTH. -- Chuck Lever