Just built a dashboard to track our team's SaaS spending. Open to feedback.

infra_ops_guru · 2026-07-26T09:51:51Z

Having observed a concerning lack of granular visibility into our burgeoning cloud and SaaS expenditure, I recently undertook a side-project to instrument and visualize these costs. The goal was to move beyond monthly invoice shock and towards predictive, service-owner-accountable cost tracking. The architecture is built on a principle of immutable, declarative infrastructure, even for internal tooling. The core pipeline is event-driven and deployed via Terraform and Kubernetes manifests. A brief overview of the components: * **Extraction Layer:** Scheduled Kubernetes `CronJobs` run lightweight Python containers that utilize vendor APIs (AWS Cost Explorer, GitHub, Datadog, etc.) to pull spend data. Credentials are managed via a secret store (Vault) with short-lived tokens. * **Transformation & Storage:** Raw data is normalized into a common schema and written to a time-series database (TimescaleDB). This is where business logic tags (e.g., `team:platform`, `project:observability-dashboard`) are applied, which is critical for meaningful allocation. * **Visualization & Alerting:** Grafana serves as the front-end, with dashboards organized by cost center and service. Prometheus alerts are configured for anomalous spend increases against forecasted baselines. ```hcl # Example Terraform module for a cost exporter deployment module "github_spend_exporter" { source = "./modules/cron-exporter" name = "github-spend-exporter" schedule = "0 8 * * *" # Daily at 0800 UTC image = "internal-registry/cost-exporters:github-latest" secret_env = { GITHUB_TOKEN = vault_generic_secret.gh_cost_token.path DB_CONNECTION = module.timescale_db.connection_uri } k8s_namespace = kubernetes_namespace.cost_tools.metadata[0].name } ``` I am particularly interested in critiques of the data model and the trade-offs of this approach versus a centralized SaaS platform like Apptio or CloudHealth. While the latter offer richer out-of-the-box reports, this system provides unparalleled flexibility in tagging and integrates directly with our existing observability stack for correlated insights (e.g., cost per million requests). The primary weaknesses I've identified are the maintenance burden of API integration breakage and the initial lack of advanced forecasting algorithms. I am open to feedback on the architectural pattern, especially regarding long-term data retention strategies and whether a stream-processing model (Kafka, Flink) would be justified over the current batch approach as we scale to incorporate more real-time cloud provider billing streams. --from the trenches

Page 3 / 3 Prev

Off-Topic

Last Post by Emma P. 1 hour ago

31 Posts

31 Users

0 Reactions

9 Views

RSS

Emma P.

(@emmap)

Trusted Member

Joined: 1 week ago

Posts: 36

29/07/2026 8:14 am

Love the initiative on this! Building something in-house means you can bake in exactly the logic your team needs, which most generic tools can't touch.

The part that really jumped out at me is tagging at the transformation layer. Getting that right is everything for accountability, but people's roles and teams shift constantly, especially in remote setups. Are you pulling team membership from your HRIS (like BambooHR or Workday) to auto-update those `team:` tags? If that's a manual YAML file, it'll be outdated in a month.

Also, for predictive tracking, don't forget to factor in per-employee costs for things like learning platforms or wellness apps. Those are easy to miss but add up fast, and they scale directly with headcount changes.

ReplyQuote

Page 3 / 3 Prev