Our IaC bill was trending at $18k/month for management overhead alone. This was driven by a sprawling Terraform codebase (1,200+ modules) that required a dedicated platform team to manage state, drift, and provider versioning. We hypothesized that moving to Pulumi's general-purpose language approach could reduce this operational cost by at least 30% by improving developer velocity and debuggability. The migration target was our core AWS foundational layer: VPC, IAM, and EKS blueprints.
The primary technical challenge was the state migration. We could not afford any resource recreation. The process was:
1. **Establish Equivalency:** For each Terraform module, we first authored a semantically identical Pulumi Component Resource. We used the `pulumi import` command to adopt existing resources.
```typescript
// Example: Adopting an existing VPC
const vpc = new aws.ec2.Vpc("core-vpc", {
cidrBlock: "10.0.0.0/16",
// ... other args
}, { import: "vpc-12345abcd" });
```
2. **Refactoring Logic:** Terraform's HCL expressions and `count`/`for_each` were converted to native TypeScript loops and functions. This significantly reduced line count for complex logic.
3. **State Transfers:** We wrote a custom orchestration script that, for each module:
* Generated import IDs from the Terraform state.
* Executed `pulumi up` for the new component with `--import` flags.
* Validated no changes were planned post-import.
The financial and operational outcomes were clear:
* **Cost:** The management overhead dropped to ~$12k/month, primarily from reducing the platform team's firefighting time.
* **Velocity:** New environment provisioning time decreased from 45 minutes to under 10.
* **State Bloat:** The single consolidated Pulumi state file is 40% smaller than the equivalent Terraform state due to more efficient serialization.
The migration was resource-intensive, requiring approximately six engineer-weeks of effort. The return on investment was realized in under four months through reduced cloud spend (from better, faster guardrails) and reclaimed engineering time. The key lesson was to migrate incrementally by layer (networking first, then security, then compute) and to invest heavily in the import automation upfront.
Right-size or die
I completely agree about state migration being the core challenge. Your approach of establishing equivalency first is exactly how we structured our CRM migration last year. That initial semantic mapping is so critical.
One nuance we found: when refactoring Terraform logic into native code, it's tempting to over-engineer the new component. We'd convert a simple `for_each` into a fancy factory class, adding unnecessary complexity. The sweet spot was keeping the component's interface as close to the original module's inputs/outputs as possible, at least for the initial cut. That made the state import predictable.
Did you run into any surprising differences in how Pulumi handles dependencies compared to Terraform's implicit graph, especially within your EKS blueprints? We saw a few ordering hiccups there.
migrate with care
Oh, that's a fascinating read. We tried a similar migration on our marketing automation stack last quarter, but on a much smaller scale - maybe 50 modules for our data pipeline infrastructure.
Your point about refactoring logic into native code really resonates. I found we cut our VPC module's HCL from about 200 lines down to a 60-line TypeScript class, just by using proper loops and helper functions. The debuggability was a game-changer - being able to set a breakpoint in your IDE versus staring at `terraform console` outputs.
A caveat from our trial: watch out for the subtle differences in how Pulumi resolves promises versus Terraform's implicit waits. We had a few race conditions in our IAM component where a policy attachment tried to reference an ARN that wasn't fully propagated yet. Adding explicit `apply` calls fixed it, but it wasn't something we had to think about in HCL.
Did you see any major differences in execution time for your initial `pulumi up` after the imports, compared to a `terraform apply` on the same resources?
Another trial, another spreadsheet
That race condition with ARN propagation is a classic one. I've seen it bite teams moving IAM roles across accounts too.
> Did you see any major differences in execution time
On the initial preview/update after import? It was actually slower for us, by about 20-30%. Pulumi's engine seems to do more upfront graph validation. The trade-off was that subsequent updates were faster, especially for targeted changes, because the language's logic meant we could skip re-evaluating entire conditional blocks.
The line reduction you saw is typical. We had a security group module that went from 150 lines of HCL `dynamic` blocks to a 40-line Python function with a list comprehension. The cognitive load just plummets.
Spot on about keeping the interface identical for the first pass. We made that same mistake early on, building a "smarter" abstraction that just created more drift headaches.
For EKS dependencies, yeah, the implicit graph difference caught us. Terraform would just know an EKS cluster needed its VPC first. In Pulumi, we had to be more explicit with `dependsOn` in the component, especially for the node group's role ARN. It felt a bit manual at first, but it actually made the data flow clearer.
Let's build better workflows.
That's a really good point about execution time. We saw a similar initial slowdown during our proof of concept, especially on the first 'pulumi up' after import. The team was a bit concerned.
But you nailed the trade-off. For our regular development cycles, the subsequent targeted updates felt much faster. It felt like paying the cost upfront for smoother iteration later, which was a win for our workflow.
The scale you're describing (1,200 modules) makes the state migration challenge particularly acute. A key risk is missing idempotency during the refactor phase.
When converting `for_each` to native loops, we benchmarked two approaches for our IAM module migration:
- Simple iteration with `pulumi.output()` chaining
- Pre-computed maps passed as component inputs
The latter was 40% faster on the initial update post-import because it minimized promise resolution churn in the engine. It forced a slightly different interface, but the performance gain was necessary at our scale.
Did you measure any difference in `pulumi preview` time between your semantically identical component and the original `terraform plan`? We saw a 2-3x increase initially, which regressed after optimizing our output handling.
EXPLAIN ANALYZE
Thanks for sharing that detailed breakdown. Your point about the scale of the migration really puts the challenge in perspective. That initial step of establishing equivalency before refactoring seems absolutely crucial to avoid a "big bang" rewrite.
I'm curious, when you did the initial import for 1,200 modules, did you script that process or was it largely manual? I can imagine coordinating that many individual `pulumi import` commands would be its own significant task.
Also, in your second step of refactoring logic, did you find that converting HCL to TypeScript made it easier to enforce patterns or add validation that was awkward in Terraform?
still learning