Having recently concluded a multi-phase migration of our organization's secret storage from AWS Secrets Manager to HashiCorp Vault, I felt compelled to document the procedural nuances and comparative insights gained. This was driven not by a failure of Secrets Manager, which is a robust service, but by our need for a unified, cloud-agnostic secrets management plane that could seamlessly integrate with our on-premises infrastructure and provide more granular access controls via identity-based authentication (e.g., JWT/OIDC) rather than solely IAM. The migration, while methodical, presented several interesting decision points that highlight the fundamental operational differences between these two systems.
The following step-by-step outline formed our core migration strategy, executed in a staged manner to minimize risk:
* **Phase 1: Vault Infrastructure and Policy Foundation**
* Established a highly available Vault cluster (using the Integrated Storage backend) in our primary AWS region, with a disaster recovery cluster in a secondary region.
* Defined and tested a comprehensive policy (`.hcl` files) framework *before* any secret ingestion. This included policies for different application tiers (e.g., `app-read-only`, `infra-admin`), team-based access, and automated provisioning roles.
* Configured authentication methods, primarily focusing on AWS IAM auth for the migration tooling and AppRole auth for our CI/CD pipelines, with a roadmap to integrate Kubernetes service account auth.
* **Phase 2: Secret Discovery and Inventory**
* Used a combination of AWS CLI scripts and AWS Config rules to inventory all Secrets Manager secrets across accounts and regions. Critical metadata included: secret name, ARN, rotation status, associated tags, and last accessed date (which helped us prune stale secrets).
* Created a normalized inventory spreadsheet mapping source (AWS SM) to destination (Vault) paths. We adopted a logical path structure in Vault: `environments/{env}/{aws_account}/{application}/{secret_name}`.
* **Phase 3: The Migration Execution Loop**
* Developed idempotent Python scripts leveraging Boto3 and the `hvac` Vault client. The logic for each secret was:
1. Fetch the secret value (and metadata) from AWS Secrets Manager.
2. Transform the data if necessary (e.g., flattening JSON structures for legacy app compatibility).
3. Write the secret to the predetermined Vault path using the `kv-v2` engine.
4. Apply specific Vault metadata (e.g., `max_versions`, `custom_metadata` replicating AWS tags).
* **Crucially, we did NOT delete the AWS secret at this stage.** We updated the secret value in AWS Secrets Manager to a JSON payload containing a pointer, e.g., `{"vault_path": "environments/prod/1234567/app/db_password", "migrated_on": "2024-01-15"}`. This provided an immediate rollback mechanism and informed any yet-unupdated consumers of the new location.
* **Phase 4: Client Application Cutover**
* This was the most protracted phase. We updated our internal libraries and configuration management code to prefer Vault clients, with fallback logic to check the legacy AWS secret for the pointer.
* Application teams were given a clear timeline to update their services. We used distributed tracing (via OpenTelemetry) to monitor for any lingering calls to Secrets Manager APIs post-migration, which helped identify undocumented dependencies.
**Key Comparative Observations:**
* **Cost Model:** AWS Secrets Manager charges per secret per month, which becomes costly at scale. Vault's operational cost is primarily the infrastructure/compute, which can be more economical for large inventories, though this adds administrative overhead.
* **Access Pattern:** Secrets Manager access is inherently tied to AWS IAM. Vault's identity-based auth provides a layer of abstraction, allowing the same secret store to service workloads from AWS, Azure, GCP, and data centers uniformly.
* **Secret Engine Richness:** While Secrets Manager handles key-value secrets and RDS credential rotation well, Vault's dynamic secrets (e.g., database credentials, AWS IAM credentials) and transit engine for encryption-as-a-service presented new opportunities for us to enhance our security posture post-migration.
The primary pitfalls to avoid are underestimating the policy design phase and attempting a "big bang" secret deletion from AWS. A phased, observable cutover is essential for platform stability. The migration solidified Vault as our central secret zero-trust boundary, though it did increase the complexity of our operational footprint compared to the fully managed Secrets Manager service.
— Billy
You've hit on the most critical part right away with your focus on policy. So many teams rush to move data and treat the policy framework as an afterthought, which creates security gaps immediately. Your approach of defining the .hcl files first is exactly right.
One nuance we found helpful was to author those policies using the actual application team names as the policy names initially, not role-based names. This made the initial mapping from IAM principals to Vault entities much clearer during the parallel run phase. Did you find any friction translating your existing IAM permission patterns into Vault's path-based policy language?
Keep it civil, keep it real
Translating IAM patterns was the hardest part. IAM's resource-level permissions don't map cleanly to Vault's path hierarchy. We had to reframe everything around the secret's location, not the secret's AWS ARN.
We used a script to analyze existing IAM policies and generate a proposed Vault path structure. The biggest friction point was wildcards. In IAM you can restrict actions on a specific secret name. In Vault, if you grant `read` on `apps/*`, you can read everything under it. We had to introduce more sub-paths to get the same granularity.
Naming policies after teams first is smart. We did the opposite and regretted it. Starting with role-based names meant we spent days untangling the mapping later. Your approach would have saved us time.
Ship it right
That's a really good point about naming policies after teams first. We're in the planning stages for a similar migration, and I'm already seeing the logic there. My instinct was to mirror our AWS IAM role naming convention, which is all about functions like `app-readonly` or `db-credential-rotator`. But that immediately feels disconnected from who actually *has* that access.
> One nuance we found helpful was to author those policies using the actual application team names as the policy names initially
This seems like it would create a much clearer audit trail right from the start. When you later mapped these team-named policies to actual Vault entities, did you find you could mostly keep the policy name as-is, or did you have to refactor them again for the production identity integration? I'm worried about doing the work twice.
You're right to worry about doing the work twice, but starting with team-based policy names actually prevented that in our case. We kept the policy names as-is through the entire production integration.
The key was treating the team-named policy as the stable authorization boundary. When we later integrated with our OIDC provider, we simply attached that same policy to the corresponding Vault identity group for that team. The policy itself, with its path rules and capabilities, didn't need to change. The refactoring would only happen if we later decided to split a team's access into more granular roles, at which point we'd create new, role-based policies (like `team-name-db-rotator`) and assign them accordingly. Starting with the team boundary gave us a solid, semantically clear foundation to build those roles upon.
Data over opinions