Alright, gather 'round the campfire of cost savings, folks. Just finished a six-month migration saga that moved our on-demand PDF generation service from a pair of heroku dynos (1x standard-2x, 1x performance-m) to a fully serverless stack on AWS. The result? Our monthly bill for this service went from a consistent ~$450 to a hilariously variable $70-$90, depending on volume. That's not a typo.
The old setup was the classic "just throw a dyno at it." We had a Rails app with Prawn that would generate complex, multi-page invoices and reports. It worked, but it was **always on**, chewing through dyno hours even at 3 AM when the only PDFs being generated were in my anxiety dreams about vendor lock-in. The performance-m dyno was there to handle the "big" reports, but 90% of the time it was overkill and 100% of the time it was expensive.
The migration wasn't about swapping a library; it was a full architectural disembowelment. Here's the new flow, gotchas included:
**The New Serverless Stack:**
1. **API Gateway** -> receives the request with generation parameters.
2. **Lambda (Node.js)** -> acts as the orchestrator. It fetches the raw data from our core API (still on Heroku, for now 😬), validates it.
3. **Lambda Container (Python)** -> this is the workhorse. We use a Lambda function packaged as a container image because we needed specific, older versions of `WeasyPrint` and `cairo` dependencies that the standard Lambda runtime couldn't easily satisfy. This image is kept warm-ish by low, consistent traffic.
4. **S3** -> generated PDF is dumped here, a signed URL is returned.
5. **CloudWatch & X-Ray** -> for the love of data, instrument everything.
The key config bit? The Lambda container. The `Dockerfile` is mundane, but the Lambda setup is critical:
```yaml
# SAM template snippet
PdfGeneratorFunction:
Type: AWS::Serverless::Function
Properties:
PackageType: Image
MemorySize: 1769 # WeasyPrint is memory-hungry. Found this sweet spot via load testing.
Timeout: 30
EphemeralStorage:
Size: 512 # Crucial! Default is 512MB, but large HTML/CSS assets for reports blew this.
Environment:
Variables:
FONTCONFIG_PATH: /var/task/fonts
Policies:
- S3CrudPolicy:
BucketName: !Ref PdfBucket
```
**Gotchas & Timelines:**
* **Month 1-2: The "This is Easy" Phase.** Wrote the container, ran great locally. Deployed. Immediate 5-second cold starts. The first "oh right" moment.
* **Month 3: The Tuning Grind.** Implemented a provisioned concurrency of 1 for the main business hours (9 AM-5 PM ET). Cold starts for the *orchestrator* Lambda became a non-issue. The container Lambda, due to its size, still had a ~1.5s init on true cold start, but it was acceptable.
* **Month 4: The Font Crisis.** WeasyPrint needs system fonts. Our corporate fonts weren't in the container. Had to bake them in and set `FONTCONFIG_PATH`. This caused more failed renders than I care to admit.
* **Month 5: The Ephemeral Storage Wall.** Generating a 50-page report with heavy CSS and base64 images? It would mysteriously fail. Turns out the `/tmp` space default wasn't enough. The `EphemeralStorage` setting in the config above was the hero.
* **Month 6: Observability & Cutover.** Shadow-prodded the new endpoint for 10% of traffic, compared outputs byte-for-byte with the old system. Drift was zero. Flipped the switch.
**The Tradeoffs:** Yes, we're now deeper in AWS's embrace. Yes, debugging a failed PDF generation in CloudWatch Logs is marginally more annoying than tailing a Heroku log. But the cost curve is undeniably beautiful. We scale to zero. The performance is more consistent under spike loads because Lambda just spins up more copies instead of queueing on a single overloaded dyno.
Was it worth six months of my life? For an 80% cost reduction on a critical path service, absolutely. The real lesson? Don't just lift-and-shift your app into a serverless function. Dissect it, isolate the compute-heavy bit, and feed it only when it's hungry.
Migrated and lived to tell.
MrMigration