The prevailing pattern of templating Kubernetes Secrets directly in Helm charts is fundamentally broken. It's a security anti-pattern that leaves sensitive data in plaintext in your `values.yaml`, release history, and potentially in your version control system. While `helm-secrets` with SOPS addresses the immediate encryption need, it shifts the secret lifecycle management burden onto the Helm/CI process and doesn't integrate cleanly with cloud-native secret backends.
The correct approach is to decouple secret *provisioning* from secret *consumption*. Helm should only define the *demand* for a secret—a Kubernetes Secret or ExternalSecret manifest—while an external system fulfills the *supply*. The most robust method I've implemented uses the External Secrets Operator (ESO) with Helm's `post-renderer` feature.
Here is the pattern:
1. **Define an `ExternalSecret` manifest in your chart's `templates/` directory.** This is a CRD that ESO watches. It specifies the source (e.g., AWS Secrets Manager, HashiCorp Vault) and the mapping to a Kubernetes Secret.
```yaml
# templates/externalsecret-database.yaml
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: {{ .Values.externalSecretName }}
spec:
refreshInterval: 1h
secretStoreRef:
name: vault-backend
kind: ClusterSecretStore
target:
name: {{ .Values.application.secretName }}
creationPolicy: Owner
data:
- secretKey: database-password
remoteRef:
key: {{ .Values.externalSecretPath }}
property: password
```
2. **In your `values.yaml`, provide only non-sensitive references.** No actual credentials.
```yaml
# values.yaml
application:
secretName: "app-db-secret"
externalSecretName: "app-db-externalsecret"
externalSecretPath: "production/database"
```
3. **Use a Kustomize `post-renderer` to strip the raw Secret.** Since ESO will create the final Secret, you must prevent Helm from also creating an empty Secret from a template. A Kustomize patch applied as a `post-renderer` removes the vanilla Secret manifest if it exists.
```yaml
# kustomization.yaml (used as post-renderer)
resources:
- all.yaml
patches:
- patch: |
- op: remove
path: /spec/template/spec/containers/0/envFrom/0/secretRef
target:
kind: Deployment
name: my-app
```
This achieves a clean separation of concerns. The Helm chart defines *what* secret is needed and its structure. The External Secrets Operator, configured at the cluster level, handles the secure retrieval from a managed vault. The CI/CD pipeline requires no special decryption keys, and the Helm release history contains only references.
The critical trade-off is operational complexity: you now depend on ESO being installed and a `SecretStore` being correctly configured. However, this is a one-time cluster-level cost that pays dividends in auditability and security posture. I'm interested in hearing about failure scenarios others have encountered—particularly around ESO's permission model or handling secret rotation during Helm upgrades.