public inbox for kdevops@lists.linux.dev
 help / color / mirror / Atom feed
From: Luis Chamberlain <mcgrof@kernel.org>
To: Chuck Lever <cel@kernel.org>, Daniel Gomez <da.gomez@kruces.com>,
	kdevops@lists.linux.dev
Cc: Luis Chamberlain <mcgrof@kernel.org>
Subject: [PATCH v4 5/8] aws: add cloud billing support with make cloud-bill
Date: Tue, 16 Sep 2025 17:34:46 -0700	[thread overview]
Message-ID: <20250917003451.2318229-6-mcgrof@kernel.org> (raw)
In-Reply-To: <20250917003451.2318229-1-mcgrof@kernel.org>

Add AWS cost tracking functionality to quickly check monthly spending.
This includes my personal hack scripts for monitoring AWS costs:

- scripts/aws-costs.sh: Queries AWS Cost Explorer for current month
- scripts/aws-parse-costs.py: Parses and displays costs by service
- make cloud-bill: Show cloud provider costs (currently AWS only)
- make cloud-bill-aws: Show detailed AWS costs for current month

The scripts provide:
- Total monthly cost to date
- Breakdown by AWS service
- Daily average spending
- Projected monthly cost (when mid-month)

This is useful for monitoring cloud spending during development and
testing, especially when running expensive instances or long tests.

Generated-by: Claude AI
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 scripts/aws-costs.sh                   | 39 ++++++++++
 scripts/aws-parse-costs.py             | 98 ++++++++++++++++++++++++++
 scripts/dynamic-cloud-kconfig.Makefile | 13 +++-
 3 files changed, 149 insertions(+), 1 deletion(-)
 create mode 100755 scripts/aws-costs.sh
 create mode 100755 scripts/aws-parse-costs.py

diff --git a/scripts/aws-costs.sh b/scripts/aws-costs.sh
new file mode 100755
index 00000000..e2298008
--- /dev/null
+++ b/scripts/aws-costs.sh
@@ -0,0 +1,39 @@
+#!/bin/bash
+# SPDX-License-Identifier: copyleft-next-0.3.1
+#
+# AWS cost tracking script - quick hack to check AWS spending
+# This queries AWS Cost Explorer to get current month's costs
+
+set -e
+
+# Get the first and last day of the current month
+FIRST_DAY=$(date +%Y-%m-01)
+LAST_DAY=$(date -d "$FIRST_DAY +1 month -1 day" +%Y-%m-%d)
+TODAY=$(date +%Y-%m-%d)
+
+# If we're still in the current month, use today as the end date
+if [[ "$TODAY" < "$LAST_DAY" ]]; then
+    END_DATE="$TODAY"
+else
+    END_DATE="$LAST_DAY"
+fi
+
+echo "Fetching AWS costs from $FIRST_DAY to $END_DATE..." >&2
+
+# Query AWS Cost Explorer
+aws ce get-cost-and-usage \
+    --time-period Start=$FIRST_DAY,End=$END_DATE \
+    --granularity MONTHLY \
+    --metrics UnblendedCost \
+    --group-by Type=DIMENSION,Key=SERVICE \
+    --output json > cost.json
+
+# Parse and display the results
+if [ -f cost.json ]; then
+    echo "Cost data saved to cost.json" >&2
+    echo "Parsing costs..." >&2
+    python3 scripts/aws-parse-costs.py cost.json
+else
+    echo "Error: Failed to retrieve cost data" >&2
+    exit 1
+fi
\ No newline at end of file
diff --git a/scripts/aws-parse-costs.py b/scripts/aws-parse-costs.py
new file mode 100755
index 00000000..7f3256a7
--- /dev/null
+++ b/scripts/aws-parse-costs.py
@@ -0,0 +1,98 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: copyleft-next-0.3.1
+#
+# Parse AWS Cost Explorer JSON output and display costs
+
+import json
+import sys
+from datetime import datetime
+
+
+def parse_costs(filename):
+    """Parse AWS Cost Explorer JSON output."""
+    try:
+        with open(filename, 'r') as f:
+            data = json.load(f)
+    except FileNotFoundError:
+        print(f"Error: File {filename} not found", file=sys.stderr)
+        sys.exit(1)
+    except json.JSONDecodeError as e:
+        print(f"Error parsing JSON: {e}", file=sys.stderr)
+        sys.exit(1)
+
+    # Extract time period
+    if 'ResultsByTime' in data and data['ResultsByTime']:
+        result = data['ResultsByTime'][0]
+        time_period = result.get('TimePeriod', {})
+        start = time_period.get('Start', 'Unknown')
+        end = time_period.get('End', 'Unknown')
+
+        print(f"\nAWS Cost Report")
+        print(f"Period: {start} to {end}")
+        print("=" * 60)
+
+        # Get total cost
+        total = result.get('Total', {})
+        if 'UnblendedCost' in total:
+            total_amount = float(total['UnblendedCost'].get('Amount', 0))
+            currency = total['UnblendedCost'].get('Unit', 'USD')
+            print(f"\nTotal Cost: ${total_amount:.2f} {currency}")
+
+        # Get costs by service
+        groups = result.get('Groups', [])
+        if groups:
+            print("\nCosts by Service:")
+            print("-" * 40)
+
+            # Sort by cost (descending)
+            sorted_groups = sorted(groups,
+                                   key=lambda x: float(x['Metrics']['UnblendedCost']['Amount']),
+                                   reverse=True)
+
+            for group in sorted_groups:
+                service = group['Keys'][0] if group.get('Keys') else 'Unknown'
+                metrics = group.get('Metrics', {})
+                if 'UnblendedCost' in metrics:
+                    amount = float(metrics['UnblendedCost'].get('Amount', 0))
+                    if amount > 0.01:  # Only show services with costs > $0.01
+                        print(f"  {service:30} ${amount:10.2f}")
+
+            # Show total at the bottom
+            print("-" * 40)
+            if 'UnblendedCost' in total:
+                print(f"  {'TOTAL':30} ${total_amount:10.2f}")
+
+        # Calculate daily average
+        try:
+            start_date = datetime.strptime(start, '%Y-%m-%d')
+            end_date = datetime.strptime(end, '%Y-%m-%d')
+            days = (end_date - start_date).days
+            if days > 0 and 'UnblendedCost' in total:
+                daily_avg = total_amount / days
+                print(f"\nDaily Average: ${daily_avg:.2f}")
+
+                # Project monthly cost if we're mid-month
+                today = datetime.now()
+                if end_date.date() == today.date() and start_date.day == 1:
+                    days_in_month = 30  # Approximate
+                    projected = daily_avg * days_in_month
+                    print(f"Projected Monthly: ${projected:.2f}")
+        except ValueError:
+            pass
+
+    else:
+        print("No cost data available in the response", file=sys.stderr)
+        sys.exit(1)
+
+
+def main():
+    """Main function."""
+    if len(sys.argv) != 2:
+        print("Usage: aws-parse-costs.py <cost.json>", file=sys.stderr)
+        sys.exit(1)
+
+    parse_costs(sys.argv[1])
+
+
+if __name__ == "__main__":
+    main()
\ No newline at end of file
diff --git a/scripts/dynamic-cloud-kconfig.Makefile b/scripts/dynamic-cloud-kconfig.Makefile
index ed2d5366..0ec14966 100644
--- a/scripts/dynamic-cloud-kconfig.Makefile
+++ b/scripts/dynamic-cloud-kconfig.Makefile
@@ -72,6 +72,8 @@ cloud-config-help:
 	@echo "cloud-update-aws        - refreshes AWS data (clears cache and regenerates)"
 	@echo "clean-cloud-config      - removes all generated cloud kconfig files"
 	@echo "cloud-list-all          - list all cloud instances for configured provider"
+	@echo "cloud-bill              - show current month's cloud provider costs"
+	@echo "cloud-bill-aws          - show AWS costs for current month"
 
 HELP_TARGETS += cloud-config-help
 
@@ -88,6 +90,15 @@ cloud-list-all:
 	$(Q)chmod +x scripts/cloud_list_all.sh
 	$(Q)scripts/cloud_list_all.sh
 
+# Cloud billing targets
+cloud-bill-aws:
+	$(Q)chmod +x scripts/aws-costs.sh
+	$(Q)scripts/aws-costs.sh
+
+cloud-bill: cloud-bill-aws
+	$(Q)echo ""
+	$(Q)echo "Note: Only AWS billing is currently supported"
+
 PHONY += cloud-config cloud-config-lambdalabs cloud-config-aws cloud-update cloud-update-aws
 PHONY += clean-cloud-config clean-cloud-config-lambdalabs clean-cloud-config-aws
-PHONY += cloud-config-help cloud-list-all
+PHONY += cloud-config-help cloud-list-all cloud-bill cloud-bill-aws
-- 
2.51.0


  parent reply	other threads:[~2025-09-17  0:34 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-09-17  0:34 [PATCH v4 0/8] aws: add dynamic kconfig support Luis Chamberlain
2025-09-17  0:34 ` [PATCH v4 1/8] aws: prevent SSH key conflicts across multiple kdevops directories Luis Chamberlain
2025-09-17  3:36   ` Chuck Lever
2025-09-17  0:34 ` [PATCH v4 2/8] terraform/aws: Add scripts to gather provider resource information Luis Chamberlain
2025-09-17  0:34 ` [PATCH v4 3/8] aws: add optimized Kconfig generator using Chuck's scripts Luis Chamberlain
2025-09-17  3:58   ` Chuck Lever
2025-09-17  0:34 ` [PATCH v4 4/8] aws: integrate dynamic Kconfig generation with make targets Luis Chamberlain
2025-09-17  3:40   ` Chuck Lever
2025-09-17  7:05     ` Luis Chamberlain
2025-09-17  0:34 ` Luis Chamberlain [this message]
2025-09-17  0:34 ` [PATCH v4 6/8] aws: replace static Kconfig files with dynamically generated ones Luis Chamberlain
2025-09-17  0:34 ` [PATCH v4 7/8] aws: add GPU instance defconfigs for AI/ML workloads Luis Chamberlain
2025-09-17  0:34 ` [PATCH v4 8/8] docs: add documentation for dynamic cloud configuration Luis Chamberlain

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250917003451.2318229-6-mcgrof@kernel.org \
    --to=mcgrof@kernel.org \
    --cc=cel@kernel.org \
    --cc=da.gomez@kruces.com \
    --cc=kdevops@lists.linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox