Check out what I made: a Grafana dashboard pulling cost data from 3 clouds.

Having spent the last quarter wrestling with a multi-cloud cost allocation project, I grew frustrated with the fragmented nature of native cost consoles. While AWS Cost Explorer, Azure Cost Management, and GCP Billing Reports are powerful individually, creating a unified, real-time view required constant context switching. To solve this, I built a consolidated Grafana dashboard that pulls, normalizes, and visualizes cost data from all three major providers, and I believe the approach might be useful for others navigating similar FinOps challenges.

The core architecture is relatively straightforward, leveraging the cloud providers' billing APIs and a simple transformation layer. The key was designing a common data model to handle the differing terminologies across clouds before feeding into Grafana.

**Data Pipeline Overview:**
1. **Extraction:** Scheduled scripts (Python, running in a serverless function) call the respective APIs:
* AWS: `CostExplorer` API for `GetCostAndUsage`
* Azure: `Consumption` API for `UsageDetails`
* GCP: `Billing` API for `BillingAccount` reports
2. **Transformation:** A lightweight dbt project maps source fields to a unified table structure. The critical model handles:
* Standardizing date/time fields to UTC.
* Mapping service names to a common taxonomy (e.g., AWS's "Amazon Simple Storage Service" -> "Object Storage", GCP's "Cloud Storage" -> "Object Storage").
* Converting all costs to a single currency (USD) using daily rates.
* Applying consistent tags/labels from each provider into a `key:value` JSON column.
3. **Loading:** The transformed data is written to a central PostgreSQL database (though any supported Grafana datasource would work).
4. **Visualization:** Grafana connects to PostgreSQL, enabling the creation of dashboards with mixed data sources.

**A snippet of the critical dbt model for normalization:**
```sql
-- stg_unified_costs.sql
with normalized as (
select
invoice_date as date,
'aws' as cloud_provider,
try_cast(blended_cost as decimal(15,6)) as cost,
service_name,
resource_tags
from {{ ref('aws_cost_explorer') }}

union all

select
usage_start_time::date as date,
'azure' as cloud_provider,
cost_in_billing_currency as cost,
meter_category as service_name,
tags as resource_tags
from {{ ref('azure_consumption') }}

union all

select
usage_start_time::date as date,
'gcp' as cloud_provider,
try_cast(cost as decimal(15,6)) as cost,
service_description as service_name,
labels as resource_tags
from {{ ref('gcp_billing_export') }}
)

select
date,
cloud_provider,
cost,
-- Example of service name normalization
case
when service_name ilike '%s3%' or service_name ilike '%storage%' then 'Object Storage'
when service_name ilike '%compute%' or service_name ilike '%ec2%' or service_name ilike '%vm%' then 'Compute'
-- ... additional mappings
else service_name
end as service_category,
resource_tags
from normalized
```

The resulting dashboard provides panels for:
* Daily total spend across all clouds, with a breakdown by provider.
* Month-to-date trend versus forecast, using Grafana's built-in time series functions.
* Top 5 most expensive services, aggregated across the unified service categories.
* Cost distribution by a specific tag (e.g., `env:production`), pulling from the normalized `resource_tags` column.
* Anomaly detection highlighting days where any single provider's spend deviates more than 2 standard deviations from its 30-day rolling average.

The immediate benefit has been operational: our platform engineering team now monitors a single dashboard for cost spikes, and finance has a single pane for monthly reporting. The more strategic advantage, however, is in the normalized data set itself, which we are now using for granular chargeback and to identify opportunities for workload placement optimization based on cost. I'm happy to share more details on the API connection specifics, the dbt mapping logic, or the Grafana JSON for anyone interested in adapting this approach.

—A.J.