Alright, I've been deep in the Redshift console for the last quarter, and I finally have some hard numbers to share. Our team managed to cut our monthly Redshift spend from ~$12k to ~$4.8k, and it wasn't just about downsizing a node. It was a systematic audit of *how* we were using it.
The biggest win came from realizing we were paying for a ton of idle compute. Our initial setup was a 4-node `ra3.4xlarge` cluster, sized for our peak monthly report load. The rest of the time? Mostly idle. Here's the three-phase approach we took:
**Phase 1: Concurrency & Query Queuing**
We were hitting default concurrency limits, causing huge queues. Instead of scaling up, we implemented Workload Management (WLM) to prioritize interactive queries and let longer reports wait. The config change was simple but game-changing.
```json
[
{
"user_group": "reporting",
"query_queue": "long_queries",
"concurrency": 3,
"memory_percent": 20
},
{
"user_group": "analysts",
"query_queue": "interactive",
"concurrency": 5,
"memory_percent": 60
}
]
```
**Phase 2: Storage & Data Types**
We moved historical data (>13 months) to S3 and queried it via Redshift Spectrum. This shrank our active cluster data by ~40%. We also audited our schema:
- Changed many `VARCHAR(256)` to `VARCHAR(32)` where appropriate.
- Used `DATE` instead of `TIMESTAMP` for date-only fields.
- This reduced I/O and memory pressure, letting queries run faster.
**Phase 3: The Big Switch: RA3 + Managed Storage**
This was the cost killer. We migrated from our old `ds2` nodes to `ra3.4xlarge`. The RA3 model separates compute from storage—you pay for compute by the second and storage separately on S3. Because our data is mostly queried in partitions, our compute nodes can now be paused entirely when not in use (hello, nights and weekends). We automated this with a Lambda function.
The hidden cost we overlooked? **Data scanned by Spectrum queries.** It's cheap, but unmonitored, it can add up. We added a simple dashboard to track it.
Has anyone else made a similar shift? I'm curious about others' experiences with per-second billing vs. the old model. The savings were dramatic, but required rethinking our "set it and forget it" cluster.
Nice start on the WLM config. That's often the first low-hanging fruit. I'd add a quick note about setting up query monitoring rules to catch runaway scans before they clog the queues. We paired ours with a simple Lambda to auto-kill queries exceeding a cost threshold.
> moved historical data (>13 months) to S3 and queried it via Redshift Spectrum
This was our biggest lever too. Did you also experiment with compression formats on Spectrum? We found Parquet with columnar compression cut our scan costs by another 30% versus the raw logs we initially offloaded.
Looking forward to Phase 3. Did you tackle automated pause/resume scaling, or was it something else?
Yeah, the Spectrum compression point is huge and often missed. We went with ORC initially because our initial data pipeline was built on Hive, and the migration was painful enough without a format change. Swapping to Parquet later required a full rewrite, which we automated with a step function, but the scan time reduction was dramatic - closer to 40% for our particular fact table schemas.
> query monitoring rules to catch runaway scans
We did this, but we also had to add exception rules for our finance team's quarterly reconciliation queries. Their "runaway" scans are literally their job, so we built a separate queue with a higher cost threshold. The Lambda auto-kill is clever, though. Did you publish the abort signal through the Redshift Data API or go straight for the `pg_terminate_backend()` route? We found the Data API had a slight lag that sometimes let a costly query slip another minute of compute in.
Phase 3 was actually about ditching the always-on cluster entirely. We moved to Redshift Serverless for everything except the core ETL window, with a cron to shift workloads. The savings were good, but the operational debt of managing the switchover schedules was not. I'll take a predictable queue over that particular flavor of configuration drift any day.
APIs are not magic.