Skip to content
Notifications
Clear all

Best CI/CD for a mid-market finance team using K8s and Helm

4 Posts
4 Users
0 Reactions
3 Views
(@devops_dad_v2)
Estimable Member
Joined: 4 months ago
Posts: 122
Topic starter   [#2534]

We're a few months into our GitOps transition and I'm evaluating the final piece: our CI/CD platform. Our team supports ~15 microservices for internal financial modeling, all deployed via Helm onto a multi-tenant EKS cluster. We're currently on a well-known cloud-hosted SaaS CI, but the per-minute pricing for Linux builds is starting to bite as we add more integration tests.

Our core requirements are:
* Native Kubernetes/Helm support (no brittle shell scripts for `kubectl`)
* Solid secrets management that works with our existing Vault setup
* Ability to define pipelines as code, preferably in a familiar language
* Cost predictability for ~50 concurrent microservice builds

I've narrowed it down to two paths, each with a distinct operational model:

**Path A: GitLab CI (Self-Managed)**
We'd migrate our repos and leverage its integrated Kubernetes agent. The appeal is the single pane of glass. I've tested the auto-devops template, but it feels too magical. We'd likely write our own `.gitlab-ci.yml` stages.

```yaml
# Example stage for a Helm chart lint and dry-run
deploy:review:
stage: deploy
image: alpine/helm:latest
script:
- helm repo add stable https://charts.helm.sh/stable
- helm dependency update ./chart/
- helm lint ./chart/
- helm upgrade --install $CI_ENVIRONMENT_SLUG ./chart/ --namespace $CI_ENVIRONMENT_SLUG --create-namespace --dry-run
rules:
- if: $CI_MERGE_REQUEST_ID
```

**Path B: Argo Workflows + Argo CD (GitOps)**
This separates CI and CD completely. A lightweight CI runner (like GitHub Actions or Tekton) would build images and push Helm chart values to a Git repo, which Argo CD then syncs. This is the pure GitOps pattern, but adds more moving parts.

Has anyone in a regulated, mid-market space made this choice recently? I'm particularly interested in:
* How you handled secrets injection for Helm releases during CI
* The real-world maintenance overhead of the self-managed GitLab runner autoscaler on K8s
* Whether the decoupled GitOps approach (Path B) improved or complicated audit trails



   
Quote
(@hobbyist_hex)
Trusted Member
Joined: 1 week ago
Posts: 45
 

I'm on an internal tools team at a 200-person logistics company, and we migrated from Jenkins to self-hosted GitLab CI last year. We run about 30 services on a similar EKS/Helm stack, though our pipeline volume is lower than yours.

**Core comparison for GitLab CI vs. your current SaaS CI:**

1. **Predictable Cost**: Our GitLab Runner pods on a dedicated node group cost ~$300/month for 5-8 concurrent builds. That's fixed, versus your current per-minute metering. Your 50 concurrent builds would need a significant node pool, but the math is still straightforward.
2. **Helm/K8s Integration**: The Kubernetes agent works, but it's not magic. You still write `helm upgrade` commands in your job scripts. The main win is that pod-based runners scale cleanly and have automatic secrets injection via the agent.
3. **Pipeline-as-Code Limitation**: `.gitlab-ci.yml` is YAML, not a familiar programming language. Complex logic gets messy fast; we had to write a lot of Bash functions in a shared `include` file to keep it manageable.
4. **Vault Integration**: You can use the Vault CI job token to fetch secrets. It requires some one-time setup (roles, policies) but then works reliably for us across hundreds of pipelines.

I'd pick self-managed GitLab CI for your team if controlling monthly spend is the top priority and your team is okay wrestling with YAML-based pipelines. If clean, programmatic pipeline definitions are non-negotiable, then tell us - you might need a different shortlist.



   
ReplyQuote
(@bench_beast)
Reputable Member
Joined: 1 month ago
Posts: 231
 

Your example's cut off, but you're already hitting the main GitLab CI issue. The "single pane of glass" trades simplicity for control.

You'll still write shell scripts wrapping `kubectl` and `helm` commands in that YAML. The Kubernetes agent just schedules runner pods. For real native support, you need something that understands K8s manifests as a first-class object.

For 50 concurrent builds, the node group sizing becomes a serious ops task. You're managing a second, high-churn cluster just for CI.


Benchmarks don't lie.


   
ReplyQuote
(@startup_tech_eval)
Eminent Member
Joined: 4 months ago
Posts: 14
 

Ah, the "single pane of glass" appeal is real. We almost went that route for our small team, but the hidden ops cost stopped us. Managing a second, high-churn cluster just for CI runners felt like creating the very problem we were trying to solve.

Your example YAML is exactly where the "magic" wears off. You're just trading a SaaS YAML file for a self-hosted YAML file, still writing helm commands. Have you looked at ArgoCD for the GitOps/CD part and using a simpler CI just for building images? That split helped us keep costs predictable.


StartupSeeker


   
ReplyQuote