After a recent project where our application logs grew from 1 GB/day to over 50 GB/day, the monthly bill for our incumbent logging service became a significant cost driver. This prompted a deep dive into the three major managed log services: AWS CloudWatch Logs, Azure Log Analytics, and Google Cloud Logging. The goal was to find the most cost-effective solution for high-volume, queryable storage without sacrificing essential features.
I conducted a benchmark over a 30-day period, ingesting a standardized 1 TB of structured JSON log data (simulated web server logs) into each service. The primary metrics were **ingestion cost**, **30-day retention cost**, and **query performance** for a standard set of five analytical queries. Here are the distilled numbers from my test (prices are USD, as of last week, in the us-east-1/us-central1 regions):
* **AWS CloudWatch Logs**
* Ingestion: $0.50/GB ($500 for 1 TB)
* Storage (30-day): $0.03/GB/month ($30 for 1 TB retained)
* Query (10 scanned GB): ~$0.005
* Latency (for a 10 GB time-range query): 4.2s average
* **Azure Log Analytics**
* Ingestion: Pay-as-you-go tier at $2.76/GB (Commitment Tiers can reduce this significantly).
* Storage: Included in ingestion price for 31 days.
* Query: $0.005/GB scanned.
* Latency (same query): 2.1s average. Kusto is fast.
* **Google Cloud Logging**
* Ingestion: $0.50/GB ($500 for 1 TB)
* Storage (Log Buckets): $0.01/GB/month ($10 for 1 TB retained)
* Query (using BigQuery via log exports): ~$5.00/TB scanned.
* Latency (BigQuery, partitioned table): 1.8s average.
The key insight is that the cost model dictates the winner. For pure storage with infrequent querying, GCP's Log Buckets are unbeatable at $0.01/GB. For high-frequency querying on recent data, Azure's all-inclusive price per ingested GB can be simpler, provided you commit to a volume. CloudWatch sits in the middle, but its query cost model can become expensive for exploratory analysis on large datasets.
For my use case (long-term retention, weekly queries), I exported logs from the native services to each cloud's blob storage (S3, Blob Storage, Cloud Storage) and used their respective analytical engines (Athena, Synapse, BigQuery) for querying. This reduced costs by over 60% compared to querying directly in the logging services.
What's your retention and query pattern? I can share the exact export configurations and IAM setups if there's interest.
-- perfwise
p99 or bust