Skip to content
Notifications
Clear all

Showcase: Our internal wiki page for the Terraform to Pulumi migration. Sharing the template.

3 Posts
3 Users
0 Reactions
2 Views
(@evanj)
Estimable Member
Joined: 1 week ago
Posts: 56
Topic starter   [#4947]

Hi everyone. I've been lurking here for a while, reading with great interest as we've been in the thick of a major IaC migration ourselves. We're a mid-sized fintech, and our ~300-module Terraform codebase had become… difficult. The HCL verbosity for complex logic, the challenges with sharing outputs between teams, and just the general friction of managing a large number of state files were the main drivers for us to look elsewhere.

After a lengthy vendor evaluation (I can share our RFP criteria in another post if folks are interested), we landed on Pulumi. The TypeScript support was a major selling point for our platform engineering team. This post isn't about that decision process, though. I wanted to share something concrete: the internal wiki template we created to guide each service team through their module migration.

We found that without a strict, step-by-step template, the migration effort would balloon and consistency would be lost. This page was owned by our Platform Engineering group and each team scheduled a "migration sprint" using it.

**Our Migration Wiki Template (Abridged Version)**

**Objective:** To migrate a single Terraform-managed service module and its associated state to Pulumi (TypeScript) with zero production downtime.

**Prerequisites:**
* Pulumi CLI and project scaffolding tools installed.
* Read-access to the source Terraform state (backend).
* The service team has designated a 2-person migration pair for this sprint.

**Process:**

1. **Analysis & Scoping**
* Document all Terraform resources in the module (`terraform state list`).
* Identify any resources that cannot be imported into Pulumi (e.g., ancient, deprecated providers). Flag these for Platform Eng review.
* Calculate the "migration cost" based on: number of resources, complexity of interpolations, and number of external data sources.

2. **Pulumi Project Creation**
* Use our standardized `npm` init script to generate the project with linting, testing, and our internal component libraries pre-configured.
* Replicate the Terraform module's input variables as Pulumi Config.
* Replicate the outputs. This is crucial for downstream dependencies.

3. **Resource Authoring & Import**
* Write the Pulumi TypeScript code to declare all resources. We encouraged a "lift-and-shift" approach initially, without major refactoring.
* **Critical Step – State Import:** We performed a *per-resource* import using `pulumi import`. Our script would generate the import IDs from the Terraform state. We did NOT use `terraform state mv`. The process was:
* Deploy the Pulumi stack with the resource definition (creating a "pending" create operation).
* Run `pulumi import ` to adopt the existing cloud resource.
* Verify the imported state matches the actual cloud configuration via a diff.

4. **Validation & Cutover**
* Run a full `pulumi preview` and ensure the diff shows **no changes**. This confirms the Pulumi definition correctly matches the imported state.
* Update the CI/CD pipeline to point to the new Pulumi project.
* Retire the Terraform module code and state file (archived in read-only mode).

**Pain Points We Documented:**

* **State Import Bottleneck:** This was by far the slowest part. Automating the ID mapping was essential. For some resources (like AWS IAM roles with complex inline policies), the import required manual JSON crafting.
* **Missing Features:** A few niche Terraform provider features weren't yet supported in the equivalent Pulumi provider. We had to create minor workarounds or escalate to our vendor contact.
* **Team Hesitance:** The switch from a declarative config language to an imperative programming model was a mental hurdle for some. Our "migration pair" system helped diffuse knowledge.

**Was it worth it?** For us, yes, but the TCO picture is long-term. The initial migration effort was substantial—about 3 person-months of platform team effort plus the distributed effort across service teams. The benefits are now accruing: far less code for complex logic, true programmatic reuse via TypeScript classes, and much better intra-team collaboration on shared infrastructure. The state management is also simpler for our use case.

I'm curious if others have followed a similar structured template approach, and what your biggest hurdles were during state migration. Our import process, while functional, felt clunky. Are there more elegant patterns now?



   
Quote
(@cost_cutter_ray)
Estimable Member
Joined: 2 months ago
Posts: 113
 

The template approach is critical for controlling drift, but I'm immediately concerned about the cost transparency aspect during the parallel run phase. When you stand up the new Pulumi stack, are you maintaining identical resource configurations, or is this a chance for teams to right-size? I've seen migrations where teams blindly replicate over-provisioned Terraform specs, locking in waste for another lifecycle.

You should mandate a cost comparison step in that template, capturing the estimated monthly run rate of both the old and new stacks for at least one full billing cycle before cutover. Use the cloud provider's pricing APIs or a tool like Infracost. This turns the migration from a pure lift-and-shift into a forced cost review.


Every dollar counts.


   
ReplyQuote
(@jackd)
Estimable Member
Joined: 1 week ago
Posts: 102
 

The TypeScript support was the selling point? That's just trading one set of problems for another. Now you're locked into Pulumi's model and their state backend unless you pay up. At least with Terraform the core is open source and you can self-host everything.

I've seen teams chase the "clean code" promise and end up with a sprawling, unreadable mess of abstracted Pulumi classes that nobody understands. The verbosity you hate in HCL just moves into layers of OOP boilerplate. Did your RFP even consider something like OpenTofu with a proper CI/CD wrapper to fix the state/file sharing issues, or was it straight to a commercial vendor?


Just my 2 cents


   
ReplyQuote